Course Objective & Scope
An accurate deep learning model cannot ship if it exceeds memory, power or latency budgets on cameras, mobile devices, industrial PCs or high-traffic cloud APIs. This programme covers Post-Training Quantisation (PTQ) and Quantisation-Aware Training (QAT) from FP32 down to FP16/INT8/INT4, structured vs unstructured pruning, knowledge distillation and hardware-friendly architecture choices.
Exporting compressed models to ONNX and compiling them for target hardware with TensorRT, ONNX Runtime, OpenVINO and LiteRT (TFLite) are practised hands-on. Participants compress a baseline model and produce an engineering report comparing model size, p99 latency, throughput and task accuracy.
Who Should Attend?
- ML and deep learning engineers
- Embedded, mobile, IoT and computer-vision/audio application developers
- Platform teams aiming to cut cloud inference bills and API latency
- Prerequisite: experience training models in PyTorch or TensorFlow
Key Takeaways
- Apply calibrated PTQ and recover lost accuracy with Quantisation-Aware Training (QAT)
- Use structured channel and head pruning that translates into real wall-clock speedups
- Export PyTorch/Keras networks cleanly to ONNX and apply graph fusions
- Compile and run models on CPU, GPU and NPU targets with ONNX Runtime, TensorRT, OpenVINO and LiteRT
- Benchmark p50/p99 latency, memory footprint and accuracy degradation systematically
Curriculum
Inference Bottlenecks and Hardware Profiles
- Compute-bound vs memory-bandwidth-bound layers and the roofline model
- Constraints across cloud GPUs, CPUs, mobile SoCs and industrial edge devices
Quantisation: PTQ, QAT and Low-Bit Formats
- Symmetric vs asymmetric quantisation, per-channel scaling and calibration
- FP16/BF16, INT8, INT4 (AWQ/GPTQ) and layer-wise sensitivity analysis
Pruning and Knowledge Distillation
- Unstructured sparsity vs structured channel/filter pruning with fine-tuning
- Distilling heavy models into compact edge-ready networks
ONNX and the Compiler/Runtime Ecosystem
- Exporting from PyTorch to ONNX, dynamic axes and operator fusion
- ONNX Runtime, NVIDIA TensorRT, Intel OpenVINO and LiteRT (TFLite)
Edge and Production Benchmarking Lab
- Batch-1 latency (p95/p99), RAM/VRAM footprint and accuracy trade-offs
- End-to-end compression pipeline and deployment checklist
Other Courses in This Category
-
Foundations and architectures of deep generative modelling: VAEs, GANs, Normalizing Flows, Denoising Diffusion (DDPM/Latent Diffusion), Flow Matching and autoregressive models for synthetic data and multimodal generation.
- Live Course
- 5 modules
-
MDPs, Q-Learning, DQN, Policy Gradients (PPO, SAC), contextual bandits and Offline RL using Gymnasium and Stable-Baselines3 for dynamic pricing, resource allocation and sequential control problems.
- Live Course
- 5 modules
-
Deep fraud models over transaction sequences, session clickstreams and payment networks: Transformer/RNN sequence encoders, autoencoder anomaly scores, GNNs for fraud rings and hybrid GBDT+DL architectures.
- Live Course
- 5 modules
-
Deep defect and fault detection across inspection images, acoustic/vibration signals and system logs using PatchCore/PaDiM, Anomalib, YOLO/segmentation, 1D-CNNs/autoencoders and normal-only training.
- Live Course
- 5 modules
-
Multivariate, long-horizon and probabilistic time-series forecasting with N-BEATS, N-HiTS, Temporal Fusion Transformer (TFT), PatchTST, DeepAR and TCN using NeuralForecast and PyTorch Forecasting.
- Live Course
- 5 modules
-
Large-scale deep recommendation architectures: Two-Tower retrieval with ANN (FAISS/Qdrant), sequential session recommenders (SASRec/BERT4Rec), DeepFM/DCN-v2/DLRM ranking and multi-task MMoE learning.
- Live Course
- 5 modules
Get in touch
