Model Online / On-Site
Duration
Modules 5 modules

Course Objective & Scope

Standard short-turn SFT falls short when you need to inject deep domain knowledge (legal, medical, financial, telecom), extend reasoning over long documents, teach multi-step tool-calling trajectories or prevent the model from losing its general reasoning capabilities. This advanced course covers industrial post-training techniques beyond basic SFT.

Labs cover continual pre-training (CPT) on unlabelled domain corpora, RoPE scaling for context extension, agentic trajectory fine-tuning, replay buffers and selective freezing to combat catastrophic forgetting, and weight-space model merging (SLERP, TIES, DARE) with MergeKit. Participants build a multi-stage adaptation pipeline that preserves general intelligence while mastering domain tasks.

Who Should Attend?

  • Senior AI engineers and researchers building domain-specific foundation or specialist models
  • Data science teams hitting a quality ceiling with standard LoRA/SFT workflows
  • ML platform teams customising open-weight models at an advanced level
  • Prerequisite: PyTorch, Hugging Face TRL or Axolotl, and prior SFT/LoRA experience

Key Takeaways

  • Design two-stage adaptation pipelines combining continual pre-training (CPT) with targeted SFT
  • Adapt models to longer context windows using RoPE scaling techniques (YaRN, LongRoPE)
  • Fine-tune models on multi-turn tool-calling, error-recovery and structured reasoning trajectories
  • Suppress catastrophic forgetting through data replay mixing, regularisation and weight interpolation
  • Combine multiple specialist checkpoints without retraining using MergeKit (SLERP, TIES, DARE)

Curriculum

Continual Pre-Training (CPT) and Domain Adaptation
  • When SFT alone is not enough and unlabelled continual pre-training is required
  • Unfreezing embeddings and lm_head to absorb domain terminology
  • Warmup schedules, cosine decay and mixing general-domain replay data
Long-Context Extension and Curriculum Learning
  • Positional encoding limits and RoPE base frequency scaling (ABF, YaRN)
  • Short-to-long curriculum mixes that preserve retrieval accuracy across long contexts
Agentic, Multi-Turn and Tool-Calling Fine-Tuning
  • Structuring multi-step Thought–Tool Call–Observation trajectories for training
  • Multi-turn assistant loss masking so observations are never trained as targets
  • Training on parallel function calls and recovery from tool errors
Mitigating Catastrophic Forgetting
  • Replay buffer ratios with general instruction and safety datasets
  • Signal-to-noise (Spectrum) layer selection to freeze general-knowledge layers
  • Preserving accuracy through post-training quantisation (AWQ, GPTQ, FP8)
Model Merging: SLERP, TIES, DARE and Task Arithmetic
  • Task vectors and the mathematics of weight-space interpolation
  • Using MergeKit to combine domain, language and reasoning checkpoints without GPU training
  • Evaluating merged checkpoints across both domain and general benchmarks

Other Courses in This Category

  • Supervised Fine-Tuning (SFT): Engineering and Optimisation

    Deep-dive engineering course on Supervised Fine-Tuning (SFT): completion-only loss masking, sequence packing, FlashAttention-2, NEFTune, learning rate schedules and production workflows with TRL SFTTrainer and Axolotl.

    LLM Customisation & Model Adaptation Corporate
    Live Course
    5 modules
    View
  • Parameter-Efficient Fine-Tuning (PEFT): Architectures and Multi-Adapter Serving

    Course on PEFT architectures—LoRA, DoRA, rsLoRA, VeRA, LoftQ, IA3 and Prefix Tuning—VRAM memory budgeting and serving dozens of dynamic adapters on a single base model with vLLM and SGLang.

    LLM Customisation & Model Adaptation Corporate
    Live Course
    5 modules
    View
  • Practical LLM Fine-Tuning with LoRA and QLoRA

    Hands-on training on Low-Rank Adaptation (LoRA) and 4-bit NormalFloat (NF4) QLoRA: rank/alpha tuning, target modules, double quantisation, paged optimizers and safe adapter merging.

    LLM Customisation & Model Adaptation Corporate
    Live Course
    5 modules
    View
  • Instruction Tuning: Following Complex Instructions, Schemas and Domain Constraints

    Course on designing and running instruction tuning so models reliably follow complex system prompts, strict JSON/schema constraints, corporate tone and refusal boundaries.

    LLM Customisation & Model Adaptation Corporate
    Live Course
    5 modules
    View
  • Preference Optimization: DPO, ORPO, KTO and LLM Alignment

    Advanced course on post-SFT alignment using DPO, IPO, KTO, ORPO and SimPO: building chosen/rejected preference datasets, tuning beta and KL constraints, and avoiding length bias and reward hacking.

    LLM Customisation & Model Adaptation Corporate
    Live Course
    5 modules
    View
  • Synthetic Data Generation: Data Synthesis and Distillation for LLM Training

    Hands-on course on generating high-diversity SFT and preference datasets in low-resource enterprise domains using teacher LLMs, Evol-Instruct, persona matrices, Distilabel and verification filters.

    LLM Customisation & Model Adaptation Corporate
    Live Course
    5 modules
    View

Get in touch