Course Objective & Scope
High-accuracy ViT, segmentation or large detection models often inflate cloud GPU bills or breach strict latency SLAs. When simple model export is not enough, the architecture and weights themselves must be compressed. This programme covers teacher-student knowledge distillation, structured channel/layer pruning and Quantization-Aware Training (QAT) in depth.
FLOPs versus actual latency, memory-bound versus compute-bound kernels and layer-wise sensitivity analysis are practised hands-on with PyTorch Profiler, Torch-Pruning and NVIDIA ModelOpt. Participants build a compression pipeline that fits their own heavy models into a target latency and memory budget.
Who Should Attend?
- AI engineers optimising large computer vision and visual foundation models
- Deep learning specialists responsible for lowering inference cost and latency
- R&D teams building models for high-traffic visual APIs or resource-constrained hardware
- Prerequisite: Proficiency in training PyTorch models and familiarity with CNN/ViT architectures
Key Takeaways
- Identify true hardware bottlenecks (compute vs memory bandwidth) with PyTorch Profiler and roofline analysis
- Distil a large teacher model (e.g. DINOv2, SAM or a heavy YOLO) into a compact student model
- Achieve real wall-clock speedups with structured channel pruning and recover accuracy via fine-tuning
- Implement Quantization-Aware Training (QAT) when post-training quantization degrades accuracy
- Produce a Pareto-optimal family of models across accuracy, latency (ms), throughput (FPS) and VRAM
Curriculum
Performance Profiling and Roofline Analysis
- Why parameter counts and FLOPs alone do not predict real-world latency
- PyTorch Profiler, CUDA kernel timings and memory-bound versus compute-bound operators
Knowledge Distillation
- Logit-based (response) and intermediate-layer (feature-based) distillation
- Distilling foundation models into lightweight backbones for detection and segmentation
Structured and Unstructured Pruning
- Differences between unstructured (2:4) sparsity and structured channel/head pruning
- L1/Taylor importance scores, dependency graphs (Torch-Pruning) and iterative retraining
Quantization-Aware Training (QAT) and Mixed Precision
- Fake-quantization nodes, Straight-Through Estimator (STE) and weight/activation scaling
- Mixed-precision policies keeping sensitive layers in FP16 while quantizing the rest to INT8/INT4
Efficient Architecture Design and Compiler Optimisations
- Depthwise separable convolutions, structural re-parameterisation (RepVGG) and ViT token pruning
- torch.compile, operator fusion and reporting accuracy-latency Pareto frontiers
Other Courses in This Category
-
Computer Vision
Comprehensive Computer Vision training ranging from classical image processing techniques to PyTorch-based CNN, Vision Transformer (ViT), Swin Transformer, and ConvNeXt architectures. Hands-on coverage includes object detection with YOLO and DETR, segmentation with SAM and U-Net, video analysis, 3D vision (NeRF, Gaussian Splatting), Diffusion models, and model deployment with ONNX/TensorRT.
- Live Course
- 13 modules
-
Digital image basics, filtering and feature extraction with OpenCV, the classification/detection/segmentation task types and a first prototype with pretrained models, for technical teams scoping computer vision use cases.
- Live Course
- 5 modules
-
CNN architectures (ResNet, EfficientNet, ConvNeXt) in PyTorch, transfer learning, data augmentation and training loops, for developers and data scientists who need to train and evaluate image models on company data.
- Live Course
- 5 modules
-
Self-supervised learning (DINOv2), vision foundation models (SAM 2, Grounding DINO), synthetic data, active learning and large-scale training, for experienced computer vision engineers tackling hard production problems.
- Live Course
- 5 modules
-
Object detection and instance segmentation with YOLO, RT-DETR and Mask R-CNN: labelling, training, mAP-based evaluation and tracking integration, for teams building an end-to-end detection/segmentation prototype.
- Live Course
- 5 modules
-
Computer Vision Bootcamp
Comprehensive Computer Vision Bootcamp ranging from classical image processing techniques to PyTorch-based CNN, Vision Transformer (ViT), Swin Transformer, and ConvNeXt architectures. Hands-on projects cover object detection with YOLO and DETR, segmentation with SAM and U-Net, video analysis, 3D vision (NeRF, Gaussian Splatting), Diffusion models, and model deployment with ONNX/TensorRT.
- Live Course
- 13 modules
Get in touch
