Model Online / On-Site
Duration
Modules 5 modules

Course Objective & Scope

Many organisations cannot send proprietary source code to cloud coding assistants for IP and regulatory reasons, yet a poorly tuned local assistant is quickly abandoned by developers due to high latency. This programme shows how to pair a small, low-latency model for inline Fill-in-the-Middle (FIM) autocomplete with a larger code model for chat, refactoring and unit test generation.

In hands-on labs participants serve Qwen2.5-Coder, DeepSeek-Coder or Codestral on vLLM or Ollama/Tabby, configure Continue.dev across VS Code and JetBrains IDEs, add internal repository indexing with local embeddings and validate developer experience against latency targets.

Who Should Attend?

  • Developer Experience and platform engineers providing internal AI tooling
  • R&D and software teams unable to use cloud coding assistants due to security policies
  • Software architects and team leads running developer productivity initiatives
  • Prerequisite: Docker, Linux and familiarity with VS Code or JetBrains IDEs

Key Takeaways

  • Select dedicated code models for FIM autocomplete versus chat and refactoring
  • Stand up a low-TTFT code completion service using vLLM or Tabby
  • Connect VS Code and JetBrains IDEs to a central server with Continue.dev
  • Configure local embedding and RAG context over internal repositories and docs
  • Define GPU capacity and privacy policies for the target number of concurrent developers

Curriculum

Coding Assistant Architecture and Model Selection
  • Differences between inline FIM completion and chat/edit/agent workloads
  • Comparing Qwen2.5-Coder, DeepSeek-Coder, StarCoder2 and Codestral
Low-Latency Serving Configuration
  • Tuning vLLM and llama.cpp for low TTFT and prefix caching on FIM requests
  • Pairing a small autocomplete model (1.5B–7B) with a larger chat model (14B–32B)
IDE Integration: Continue.dev and Tabby
  • Centralised config.yaml rollout for VS Code and JetBrains
  • Turnkey self-hosted coding assistant setup with Tabby
Internal Codebase Context (Codebase RAG)
  • AST-aware code chunking and local embedding models
  • Connecting internal documentation, libraries and MCP tools to the assistant
Security, Telemetry and Capacity Planning
  • API key management, prompt logging boundaries and licence compliance
  • GPU sizing per developer and tracking completion acceptance rates

Other Courses in This Category

  • Open-Weight LLM Selection, Licensing and Benchmarking for Enterprise Use

    Selecting open-weight LLMs for enterprise use: Llama, Qwen, Mistral, Gemma and DeepSeek families; commercial licence analysis, dense vs MoE trade-offs, Turkish and tool-calling capability, and building a benchmark matrix on company data.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • Enterprise Model Serving with NVIDIA Triton Inference Server and KServe

    Multi-model GPU serving with NVIDIA Triton Inference Server and KServe: TensorRT-LLM, ONNX and Python backends, dynamic batching, model ensembles, KServe InferenceService, Knative autoscaling and tuning with Model Analyzer.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • KV Cache Management and Continuous Batching Optimization for LLM Inference

    Solving memory and concurrency bottlenecks in LLM serving: KV cache memory math, PagedAttention, SGLang RadixAttention, prefix caching, FP8 KV cache quantization, continuous batching and chunked prefill tuning.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • LLM Inference Throughput and Latency Optimization

    Optimising TTFT, ITL and total throughput in LLM services: compute-bound vs memory-bandwidth-bound bottleneck analysis, FlashAttention/FlashInfer, CUDA Graphs, speculative decoding, parallelism tuning and SLO-driven load testing.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • Running AI and LLM Workloads on Kubernetes

    Operating GPU and LLM workloads on Kubernetes: NVIDIA GPU Operator, GPU sharing with MIG and time-slicing, queue management with Kueue/Volcano, fast model weight caching, GPU autoscaling and DCGM observability.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • Deploying Small Language Models on Edge and On-Device Hardware

    Running Small Language Models (SLMs) on edge devices, field hardware and NPUs/embedded GPUs: Llama 3.2, Qwen, Gemma and Phi (1B–4B), ONNX Runtime GenAI, ExecuTorch, TensorRT, Jetson and offline field scenarios.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View

Get in touch