Model Online / On-Site
Duration
Modules 4 modules

Course Objective

To empower organizations seeking data sovereignty, regulatory compliance, and predictable costs to architect and operate an end-to-end open-source and self-hosted enterprise AI stack without proprietary cloud lock-in.

Participants learn how to serve open-weight models from the Llama, Mistral, Qwen, and DeepSeek families at high throughput with vLLM, enforce structured output, and integrate them with open-source agent frameworks.

Target Audience and Prerequisites

  • Target Audience: AI infrastructure engineers, MLOps/Platform engineers, senior software developers, and on-premise/VPC AI architects.
  • Prerequisites: Working knowledge of Linux, Docker/Kubernetes containers, Python programming, and foundational GPU/LLM concepts.

Key Takeaways and Focus Areas

  • Applying PagedAttention, continuous batching, and quantization (AWQ, GPTQ, FP8) optimizations using vLLM, SGLang, and TGI.
  • Implementing reliable tool calling, grammar-constrained JSON generation (guided decoding), and MCP (Model Context Protocol) servers on open-weight models.
  • Assembling a sovereign stack with LiteLLM gateways, open-source vector databases (Qdrant/Milvus), and Langfuse/Phoenix observability.

Curriculum

Open-Weight Model Families and Enterprise Selection Matrix
  • Architectural, licensing, and multilingual performance analysis of Llama, Mistral, Qwen, Gemma, and DeepSeek
  • VRAM sizing, parameter counts, Mixture-of-Experts (MoE), and quantization (FP8, AWQ, GGUF) trade-offs
vLLM and High-Performance Model Serving Infrastructure
  • PagedAttention, continuous batching, prefix caching, and speculative decoding with vLLM and SGLang
  • OpenAI-compatible API deployment, multi-LoRA serving, and enterprise AI Gateway routing with LiteLLM
Open-Source Agent Frameworks, Tool Calling, and Guided Decoding
  • Guaranteed JSON schema generation with Outlines/XGrammar and reliable function calling on local models
  • Building open-source agents with LangGraph, CrewAI, PydanticAI, and the Model Context Protocol (MCP)
Self-Hosted RAG, Observability, and Security Stack
  • Qdrant/Milvus vector databases, open-source embedding/reranker models, and local document parsing (Docling)
  • Open-source tracing and evaluation with Langfuse and Arize Phoenix, plus local guardrail (Llama Guard) integration

Other Courses in This Category

  • Enterprise AI Development with Microsoft Azure AI Foundry

    Model selection, enterprise RAG and agent development, content safety filters, automated evaluation, OpenTelemetry tracing and secure deployment on Microsoft Azure AI Foundry.

    Enterprise AI Platforms & Agent Cloud Ecosystem Corporate
    Live Course
    5 modules
    View
  • Building Multi-Agent Systems with Microsoft Agent Framework

    Enterprise single- and multi-agent orchestration, MCP tool integration, memory and stateful process workflows using Microsoft Agent Framework (unifying Semantic Kernel and AutoGen).

    Enterprise AI Platforms & Agent Cloud Ecosystem Corporate
    Live Course
    5 modules
    View
  • GitHub Copilot and Autonomous Coding Agents for Engineering Teams

    Accelerating the software development lifecycle with GitHub Copilot Agent Mode, Copilot Coding Agent, custom instructions, IDE MCP servers and automated PR review.

    Enterprise AI Platforms & Agent Cloud Ecosystem Corporate
    Live Course
    5 modules
    View
  • Enterprise Agent Infrastructure with Amazon Bedrock AgentCore

    Deploying agents built with any framework (Strands, LangGraph, CrewAI) securely and at scale using Amazon Bedrock AgentCore: Runtime, Memory, Identity, Gateway and Observability.

    Enterprise AI Platforms & Agent Cloud Ecosystem Corporate
    Live Course
    5 modules
    View
  • Developing Production AI Agents with AWS Strands Agents SDK

    Building model-driven agents with the open-source AWS Strands Agents SDK: @tool decorators, native MCP integration, multi-provider model support, multi-agent patterns and OpenTelemetry tracing.

    Enterprise AI Platforms & Agent Cloud Ecosystem Corporate
    Live Course
    5 modules
    View
  • Amazon Bedrock Knowledge Bases and Guardrails

    Managed enterprise RAG with Amazon Bedrock Knowledge Bases (advanced parsing, chunking, hybrid search, reranking) and safety enforcement with Bedrock Guardrails (PII, grounding, content filters).

    Enterprise AI Platforms & Agent Cloud Ecosystem Corporate
    Live Course
    5 modules
    View

Get in touch