Model Online / On-Site
Duration
Modules 6 modules

Course Objective & Scope

Even when models offer 128K to 1M+ token context windows, stuffing entire document sets into the prompt causes high cost, slow time-to-first-token (TTFT) and quality degradation from the "lost in the middle" effect. This programme covers attention behaviour over long sequences, KV cache growth, context pruning and compression, hierarchical summarisation and long-term conversation memory design.

Participants benchmark full-context prompting against RAG and hybrid approaches on large codebases or multi-document corporate corpora, producing a cost, latency and quality profile tailored to their organisation.

Who Should Attend?

  • Software engineers building LLM applications over long documents, contracts, regulations or codebases
  • AI engineers designing memory architectures for multi-turn assistants and agents
  • Platform teams responsible for cost and latency in high-volume LLM services
  • Prerequisite: working experience with LLM APIs and core token/context concepts

Key Takeaways

  • Measure positional sensitivity and the lost-in-the-middle effect across long inputs
  • Choose between long context, RAG and hybrid designs based on cost, latency and accuracy
  • Implement sliding-window, hierarchical summarisation and semantic context compression
  • Use provider prompt/context caching effectively on repeated long prefixes
  • Build needle-in-a-haystack and multi-hop synthesis evaluations for enterprise documents

Curriculum

Long-Context Architecture and Its Limits
  • RoPE scaling, KV cache memory footprint and impact on TTFT
  • The lost-in-the-middle phenomenon and effective context length
Long Context vs RAG vs Hybrid Designs
  • Decision matrix based on query volume, document churn and token cost
  • Patterns for combining retrieval with long-context synthesis
Context Compression and Pruning Techniques
  • Section selection, relevance-based ordering and boilerplate removal
  • Hierarchical summarisation and map-reduce document pipelines
Multi-Turn Conversation and Memory Management
  • Sliding windows, summary memory and structured state tables
  • Separating episodic and semantic memory layers
Prefix and Context Caching
  • Prompt and context caching mechanisms in OpenAI, Anthropic and Gemini
  • Prompt ordering rules that maximise cache hit rates
Lab: Long-Document Benchmarking
  • Needle-in-a-haystack and synthesis tests on internal documents
  • Producing a cost, latency and accuracy profile

Other Courses in This Category

  • Model Routing and Multi-Model Architecture Design

    Engineering programme on dynamic model routing, cascades, fallbacks and LLM gateways to balance quality, cost and latency across frontier and small models without single-provider lock-in.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • Building Multimodal LLM Applications: Vision, Documents and Audio

    Hands-on programme for building multimodal LLM applications that process images, scanned documents, charts, audio and video using frontier VLM APIs, open-weight vision models and structured extraction.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • Mixture of Experts (MoE) LLMs: Architecture, Serving and Fine-Tuning

    Advanced programme on sparse Mixture of Experts (MoE) LLMs, covering gating networks, active vs total parameters, VRAM sizing, expert parallelism in vLLM/SGLang and MoE fine-tuning.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • Streaming and Real-Time AI Application Engineering

    Building low-latency streaming text, partial structured output and bidirectional voice AI applications using SSE, WebSockets and WebRTC, with TTFT optimisation, barge-in handling and concurrency control.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • Small Language Models (SLMs) for Enterprise Applications

    Building fast, low-cost enterprise solutions with 1B–14B Small Language Models (Llama, Qwen, Gemma, Phi, Ministral) for classification, extraction and routing, including distillation and task adaptation.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • Open-Weight LLM Integration and Self-Hosted Serving

    Deploying and integrating open-weight LLMs (Llama, Qwen, Mistral, DeepSeek) on-premises or in private cloud as OpenAI-compatible APIs using vLLM, SGLang and Ollama, including GPU sizing and quantization.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View

Get in touch