Model Online / On-Site
Duration
Modules 6 modules

Course Objective & Scope

Decisions about model selection, inference optimisation or fine-tuning are hard to make well without understanding what happens inside a transformer. This course walks through the decoder-only architecture layer by layer: BPE tokenization, the embedding layer, multi-head self-attention, positional encoding (RoPE), feed-forward blocks and how the KV cache affects latency.

It then covers the pretraining objective, scaling laws, SFT, RLHF and DPO, and how reasoning models are trained. Using PyTorch and Hugging Face Transformers, participants run a small model step by step and inspect attention weights and token probabilities themselves.

Who Should Attend?

  • Machine learning and AI engineers who will work deeply with LLMs
  • Data scientists preparing for fine-tuning or model serving projects
  • Architects who need technical grounds for model selection decisions
  • Prerequisite: Python skills and familiarity with basic linear algebra and probability

Key Takeaways

  • Explain self-attention and multi-head attention computation step by step
  • Interpret how tokenization, context length and the KV cache affect cost and latency
  • Distinguish the contribution of pretraining, SFT, RLHF and DPO to model behaviour
  • Load a model with Hugging Face Transformers and analyse its logits and attention outputs
  • Evaluate architectural details on model cards (parameter count, context, GQA, MoE)

Curriculum

Tokenization and the Embedding Layer
  • BPE and SentencePiece, token efficiency for Turkish text
  • The embedding matrix and the impact of vocabulary size
Self-Attention and the Transformer Block
  • Query, key, value computation and multi-head attention
  • Residual connections, layer normalisation, feed-forward layers
Positional Information, Long Context and the KV Cache
  • RoPE and context extension techniques
  • KV cache, grouped-query attention and FlashAttention
Pretraining and Scaling Laws
  • The next-token prediction objective and data mixtures
  • The relationship between parameters, data and compute
Post-Training: SFT, RLHF, DPO and Reasoning Models
  • Instruction tuning and preference optimisation methods
  • Test-time compute in reasoning models
Lab: Looking Inside a Model
  • Logit and attention analysis with Hugging Face Transformers
  • Experimental comparison of sampling strategies

Other Courses in This Category

  • Building Applications with LLM APIs

    Hands-on programme for building a near-production LLM application with the OpenAI, Anthropic Claude and Google Gemini APIs, covering streaming, conversation state, retries, rate limits, fallbacks and basic evals.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • Structured Outputs with LLMs: JSON Schema, Pydantic and Constrained Decoding

    Getting reliable JSON from LLMs with JSON Schema, OpenAI Structured Outputs, Gemini response schemas, Pydantic/Zod, Instructor and constrained decoding, plus validation, repair loops and document data extraction.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • LLM Tool Use and Function Calling

    How to let LLMs safely call APIs, databases and internal systems via tool/function calling: tool schema design, parallel calls, MCP servers, error handling and permission boundaries.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • Prompt and Context Architecture for LLM Applications

    Context engineering for LLM applications: designing how system prompts, templates, few-shot examples, retrieved content, memory and tool definitions fill the context window, with versioning and eval-driven iteration.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • Long-Context LLM Engineering and Context Window Management

    Solving lost-in-the-middle degradation, KV cache cost and latency in large context windows through context compression, hierarchical memory, prompt caching and needle-in-a-haystack evaluation.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View
  • Model Routing and Multi-Model Architecture Design

    Engineering programme on dynamic model routing, cascades, fallbacks and LLM gateways to balance quality, cost and latency across frontier and small models without single-provider lock-in.

    LLM Application Engineering Corporate
    Live Course
    6 modules
    View

Get in touch