Course Objective & Scope
In air-gapped LLM environments the hardest problem is rarely the first install—it is securely importing and updating runtime dependencies, tokenizer files, CUDA/driver versions and new model weights without any outbound connectivity. This training covers offline mirroring for Hugging Face Hub and container images, verification chains with safetensors, SHA256 and Cosign, and strictly offline runtime configurations (HF_HUB_Offline=1).
In hands-on labs participants set up a Harbor/Nexus mirror inside an isolated network simulation, import a signed model bundle and execute a zero-downtime model and inference engine upgrade.
Who Should Attend?
- DevSecOps, systems and platform engineers operating in isolated networks
- AI infrastructure teams in defence, finance and critical infrastructure organisations
- Systems administrators managing air-gapped Kubernetes and GPU servers
- Prerequisite: Linux system administration, Docker/OCI images and basic network security
Key Takeaways
- Stand up an offline artifact repository for containers, Python packages and model weights
- Verify model files with security scans, checksums and digital signatures
- Configure vLLM, TGI and Transformers services for strictly offline execution
- Bundle NVIDIA drivers, CUDA toolkits and container runtimes for offline upgrades
- Operate a rollback-safe update procedure for models and serving engines
Curriculum
Air-Gapped AI Supply Chain
- Staging/quarantine zone, one-way transfer and approval workflow
- Pickle risks, the
safetensorsstandard and model SBOM generation
Offline Registry and Model Repository Setup
- OCI and PyPI mirroring with Harbor, Nexus or Artifactory
- Archive layout for model weights, tokenizers and chat templates
Eliminating Hidden Outbound Calls and Installation
HF_HUB_OFFLINE,TRANSFORMERS_OFFLINEand tiktoken/tokenizer cache settings- Standing up vLLM and embedding services with zero internet access
GPU Driver and Infrastructure Upgrade Bundles
- Offline apt/rpm repositories, NVIDIA driver and Container Toolkit bundling
- Compatibility matrix: kernel, driver, CUDA and vLLM image versions
Zero-Downtime Upgrade and Rollback Drill
- Blue/green and canary model cutover with offline smoke tests
- Rapid rollback to previous image and model weights on failure
Other Courses in This Category
-
Running local LLMs on desktops and workstations with llama.cpp, GGUF and LM Studio: k-quant selection, CPU+GPU layer offloading, Metal/CUDA/Vulkan acceleration and local OpenAI-compatible APIs. For developers and analysts.
- Live Course
- 5 modules
-
Building self-hosted AI coding assistants without sending source code outside the company: Qwen2.5-Coder and DeepSeek-Coder models, FIM autocomplete, Continue.dev/Tabby IDE plugins, vLLM serving and codebase RAG integration.
- Live Course
- 5 modules
-
Selecting open-weight LLMs for enterprise use: Llama, Qwen, Mistral, Gemma and DeepSeek families; commercial licence analysis, dense vs MoE trade-offs, Turkish and tool-calling capability, and building a benchmark matrix on company data.
- Live Course
- 5 modules
-
Multi-model GPU serving with NVIDIA Triton Inference Server and KServe: TensorRT-LLM, ONNX and Python backends, dynamic batching, model ensembles, KServe InferenceService, Knative autoscaling and tuning with Model Analyzer.
- Live Course
- 5 modules
-
Solving memory and concurrency bottlenecks in LLM serving: KV cache memory math, PagedAttention, SGLang RadixAttention, prefix caching, FP8 KV cache quantization, continuous batching and chunked prefill tuning.
- Live Course
- 5 modules
-
Optimising TTFT, ITL and total throughput in LLM services: compute-bound vs memory-bandwidth-bound bottleneck analysis, FlashAttention/FlashInfer, CUDA Graphs, speculative decoding, parallelism tuning and SLO-driven load testing.
- Live Course
- 5 modules
Get in touch
