Model Online / On-Site
Duration
Modules 6 modules

Course Objective & Scope

Token consumption and multi-second latencies that seem harmless in a prototype quickly turn into budget overruns and sluggish user experiences at production scale. This course covers the anatomy of LLM cost and latency (input, output and reasoning tokens, TTFT and TPOT), provider prompt caching, semantic caching, asynchronous Batch APIs and controlling output verbosity.

Participants profile an existing LLM workflow and apply concrete optimisations that lower unit cost and p95 latency while preserving eval quality.

Who Should Attend?

  • AI and backend engineers responsible for the performance and budget of production LLM services
  • Platform teams managing AI infrastructure and FinOps metrics
  • Technical product managers and architects improving AI unit economics
  • Prerequisite: experience building applications with LLM APIs

Key Takeaways

  • Profile cost per transaction and latency breakdown (network, TTFT, generation) for an LLM feature
  • Structure static instructions, tool schemas and examples to maximise prompt cache hit rates
  • Design a safe semantic cache layer for repeated or near-duplicate queries
  • Move non-interactive workloads to Batch APIs and queues for major cost reductions
  • Run controlled experiments proving cost and latency gains without hurting eval accuracy

Curriculum

LLM Unit Economics and Latency Anatomy
  • Analysing input, output, cached and reasoning token pricing
  • How prefill (TTFT) and decode (TPOT) stages drive end-to-end latency
Prompt Caching and Prefix Optimisation
  • Prompt caching mechanics and TTL management in OpenAI, Anthropic and Gemini
  • Placing dynamic variables at the tail of the prompt to avoid cache invalidation
Exact and Semantic Caching in the Application Layer
  • Redis and vector-based caching, similarity thresholds and false-positive risks
  • Safe cache keys across user permissions, time-sensitive data and personalisation
Token Reduction and Output Length Control
  • Cutting output tokens (the main latency driver), compact schemas and stop sequences
  • Tuning reasoning effort and thinking budgets in reasoning models
Parallelism, Batch APIs and Execution Patterns
  • Running independent LLM steps concurrently and speculative execution
  • Using asynchronous Batch APIs to cut cost on offline workloads
Hands-On: Cost / Latency / Quality Pareto Analysis
  • Step-by-step optimisation of a sample LLM service
  • Producing a before/after table of unit cost, p95 latency and eval quality

Other Courses in This Category

  • LangChain Fundamentals for LLM Application Development

    Hands-on programme on building modular LLM applications with modern LangChain (LCEL, ChatModel abstractions, prompt templates, structured outputs, tool calling, retrievers and LangSmith tracing).

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • LLM Engineering for Java & .NET (Spring AI, LangChain4j, Semantic Kernel)

    Building type-safe LLM, RAG, tool-calling and MCP integrations directly in enterprise Java (Spring AI, LangChain4j) and .NET (Microsoft.Extensions.AI, Semantic Kernel) stacks without switching to Python.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • Building Enterprise LLM Applications on Amazon Bedrock

    Hands-on programme for building enterprise generative AI solutions on AWS using the Amazon Bedrock Converse API, Knowledge Bases, Agents, Guardrails, IAM/VPC security and cost controls.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • Enterprise LLM Development with Azure OpenAI and Azure AI Foundry

    Building enterprise AI applications on Azure AI Foundry and Azure OpenAI Service with GPT and reasoning models, Azure AI Search, Entra ID authentication, Private Endpoints, Content Safety and PTU/quota management.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • Building Enterprise LLM Applications on Google Cloud Vertex AI

    Hands-on programme for building enterprise AI applications on Google Cloud Vertex AI using Gemini models, the Google Gen AI SDK, Context Caching, Grounding, Vertex AI Search, BigQuery and VPC-SC security.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • Production Architecture Patterns for Enterprise LLM Systems

    Architectural patterns for turning LLM prototypes into resilient enterprise systems: AI gateways, async queues, circuit breakers, fallback chains, idempotency, guardrails and OpenTelemetry tracing.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View

Get in touch