Model Online / On-Site
Duration
Modules 5 modules

Course Objective & Scope

Adopting LLM capabilities in regulated sectors such as finance, defence, government and healthcare requires building a multi-tenant, auditable in-house platform rather than sending sensitive data to external APIs. This programme brings together hardware sizing, GPU cluster network topology, internal model registries, vLLM/Triton serving layers, enterprise authentication and transition to air-gapped production within a single reference architecture.

Through an organisation-adapted case study, participants produce a private AI deployment blueprint, network and storage layout, and operational readiness checklist matching their own security and capacity constraints.

Who Should Attend?

  • Platform and infrastructure architects building an in-house AI platform
  • MLOps, LLMOps and Kubernetes cluster administrators
  • Technical leads driving private AI adoption in regulated industries
  • Prerequisite: Linux, container/Kubernetes and basic LLM inference knowledge

Key Takeaways

  • Size GPU, networking and storage infrastructure for target use cases
  • Design a multi-tenant internal LLM serving and model gateway layer
  • Establish the lifecycle for model weights, container images and dependencies in air-gapped networks
  • Define enterprise IAM, quota, audit logging and network isolation policies
  • Author an end-to-end private AI deployment blueprint for the organisation

Curriculum

Enterprise Private AI Reference Architecture
  • Data residency, regulatory requirements and threat modelling
  • Compute, storage, model serving and gateway layers
Hardware, Networking and Storage Infrastructure
  • GPU server topology, NVLink, RoCE/InfiniBand and fast local disks
  • Shared storage and caching design for model weights
Serving and Multi-Tenant Platform Layer
  • vLLM and embedding/reranker services on Kubernetes
  • Internal model gateway, RBAC, team quotas and audit logging
Air-Gapped Production and Supply Chain
  • Image and model weight transfer pipeline across the DMZ
  • Signature verification, security scanning and version pinning
Operations, Monitoring and Capacity Evolution
  • GPU health monitoring, SLO tracking and failure playbooks
  • Drafting the organisation-specific private AI deployment blueprint

Other Courses in This Category

  • Air-Gapped LLM Deployment, Offline Registry Mirroring and Update Operations

    Deploying and updating LLM services in air-gapped networks: offline container, PyPI and Hugging Face mirroring, safetensors and SBOM verification, GPU driver bundling and zero-downtime patching. Built for security-focused platform teams.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • Running Local LLMs on Desktop and Workstations: llama.cpp, GGUF and LM Studio

    Running local LLMs on desktops and workstations with llama.cpp, GGUF and LM Studio: k-quant selection, CPU+GPU layer offloading, Metal/CUDA/Vulkan acceleration and local OpenAI-compatible APIs. For developers and analysts.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • Self-Hosted AI Coding Assistants with Local LLMs

    Building self-hosted AI coding assistants without sending source code outside the company: Qwen2.5-Coder and DeepSeek-Coder models, FIM autocomplete, Continue.dev/Tabby IDE plugins, vLLM serving and codebase RAG integration.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • Open-Weight LLM Selection, Licensing and Benchmarking for Enterprise Use

    Selecting open-weight LLMs for enterprise use: Llama, Qwen, Mistral, Gemma and DeepSeek families; commercial licence analysis, dense vs MoE trade-offs, Turkish and tool-calling capability, and building a benchmark matrix on company data.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • Enterprise Model Serving with NVIDIA Triton Inference Server and KServe

    Multi-model GPU serving with NVIDIA Triton Inference Server and KServe: TensorRT-LLM, ONNX and Python backends, dynamic batching, model ensembles, KServe InferenceService, Knative autoscaling and tuning with Model Analyzer.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • KV Cache Management and Continuous Batching Optimization for LLM Inference

    Solving memory and concurrency bottlenecks in LLM serving: KV cache memory math, PagedAttention, SGLang RadixAttention, prefix caching, FP8 KV cache quantization, continuous batching and chunked prefill tuning.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View

Get in touch