Model Online / On-Site
Duration
Modules 6 modules

Course Objective & Scope

Sending every request in an enterprise application to the largest frontier model quickly burns budget, while sending everything to a small model hurts quality on hard queries. This course covers intent classification, difficulty estimation, semantic and rule-based routers, model cascades (fast model first, escalating when needed) and multi-provider fallback architectures.

Using LiteLLM and custom router components, participants unify closed and open-weight models behind a single gateway and build a model selection matrix and routing policy tested on workload samples from their organisation.

Who Should Attend?

  • AI and backend engineers scaling products that use multiple LLMs
  • Software architects designing enterprise AI gateways and platform infrastructure
  • Technical leads targeting unit-cost reduction, SLAs and provider independence
  • Prerequisite: experience building services with LLM APIs and basic evaluation concepts

Key Takeaways

  • Build a multi-model selection matrix based on task type, complexity and SLA targets
  • Implement rule-based, embedding-based and small-classifier routers
  • Design model cascades using confidence signals and verifier checks
  • Configure load balancing, quotas and fallbacks on an LLM gateway such as LiteLLM
  • Benchmark routed architectures against a single frontier model on quality, cost and p95 latency

Curriculum

Multi-Model Architecture and the Selection Matrix
  • Matching reasoning, general-purpose, fast and open-weight models to tasks
  • Criteria across quality, unit cost, TTFT, throughput and data residency
Model Routing Approaches
  • Routing via deterministic rules, embedding similarity and lightweight classifiers
  • Training and calibrating RouteLLM and custom router models
Model Cascades and Verifier Patterns
  • Escalation criteria from small models to frontier models
  • Quality gates using schema validation, self-checks and judge signals
LLM Gateways and Provider Resilience
  • Unified APIs, rate limits, budget controls and key rotation with LiteLLM
  • Automated cross-provider fallbacks on regional outages and quota exhaustion
Prompt Portability Across Model Families
  • Handling prompt and tool-calling differences between providers
  • Model-agnostic templates and eval regression suites
Hands-On: Enterprise Router and Gateway Design
  • Building a two-tier router on representative internal requests
  • Measuring cost savings and quality retention

Other Courses in This Category

  • Building Multimodal LLM Applications: Vision, Documents and Audio

    Hands-on programme for building multimodal LLM applications that process images, scanned documents, charts, audio and video using frontier VLM APIs, open-weight vision models and structured extraction.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • Mixture of Experts (MoE) LLMs: Architecture, Serving and Fine-Tuning

    Advanced programme on sparse Mixture of Experts (MoE) LLMs, covering gating networks, active vs total parameters, VRAM sizing, expert parallelism in vLLM/SGLang and MoE fine-tuning.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • Streaming and Real-Time AI Application Engineering

    Building low-latency streaming text, partial structured output and bidirectional voice AI applications using SSE, WebSockets and WebRTC, with TTFT optimisation, barge-in handling and concurrency control.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • Small Language Models (SLMs) for Enterprise Applications

    Building fast, low-cost enterprise solutions with 1B–14B Small Language Models (Llama, Qwen, Gemma, Phi, Ministral) for classification, extraction and routing, including distillation and task adaptation.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • Open-Weight LLM Integration and Self-Hosted Serving

    Deploying and integrating open-weight LLMs (Llama, Qwen, Mistral, DeepSeek) on-premises or in private cloud as OpenAI-compatible APIs using vLLM, SGLang and Ollama, including GPU sizing and quantization.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • LLM Cost and Latency Optimisation: Prompt Caching, Batching and FinOps

    Engineering techniques to cut token spend and p95 latency in production LLM applications: prompt and semantic caching, Batch APIs, output length control, tiered models and unit economics.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View

Get in touch