Course Objective & Scope
Overnight churn scoring, fraud checks at payment time and anomaly detection on sensor data each need a different inference mode. The course explains the architecture of batch, real-time and streaming inference, their impact on data freshness and cost, and feature access patterns, all through examples.
Participants run the same model as a scheduled batch job with Airflow, as an online service with FastAPI and as a Kafka consumer. They finish with a decision matrix and checklist mapping the organisation's use cases to the three modes.
Who Should Attend?
- Data scientists connecting model results to business processes
- Data engineers managing data pipelines
- AI and platform engineers planning inference infrastructure
- Prerequisite: Python and basic SQL
Key Takeaways
- Select batch, real-time or streaming mode for a use case
- Build a scheduled batch scoring job with Airflow
- Design an online prediction service including feature access
- Run streaming inference on events arriving via Kafka
- Compare the cost and data-freshness trade-offs of the three modes
Curriculum
Overview of Inference Modes
- Definitions of batch, real-time and streaming
- The freshness, latency and cost axis
Batch Inference
- Bulk scoring with Airflow and Spark
- Writing results to the data warehouse and reruns
Real-Time Inference
- Online services and the latency budget
- Online feature access with a feature store
Streaming Inference
- Event processing with Kafka consumers and Apache Flink
- Windowing and ordering issues
Mode Selection and Decision Matrix
- Mapping organisational scenarios to modes
- Hybrid architectures and a migration plan
Other Courses in This Category
-
Saving models safely and portably: pickle/joblib risks, TorchScript, SavedModel, ONNX, safetensors and GGUF formats, packaging with MLflow and version compatibility. For ML engineers and data scientists.
- Live Course
- 5 modules
-
Model serving architecture patterns: embedded models, model-as-a-service, serverless, edge, multi-model and LLM gateway designs, and how platforms like KServe, Triton, Ray Serve and vLLM fit in. For architects and senior ML engineers.
- Live Course
- 5 modules
-
Balancing p95/p99 latency, throughput and unit cost in model serving: dynamic batching, quantization, caching, GPU/CPU selection and load-testing optimisation. For ML and platform engineers building scalable, cost-effective inference infrastructure.
- Live Course
- 5 modules
-
End-to-end security for inference services: mTLS, OAuth2/JWT, rate limiting, safe serialization, container hardening, defence against model extraction/adversarial attacks and data privacy. For ML, platform and AppSec engineers.
- Live Course
- 5 modules
-
Exporting TensorFlow and Keras models as SavedModel, defining SignatureDefs and serving them over REST and gRPC with TensorFlow Serving. For data scientists and ML engineers starting to deploy TensorFlow models.
- Live Course
- 5 modules
-
Multi-model configurations, dynamic batching, XLA/TensorRT and mixed-precision acceleration, GPU memory tuning, Prometheus metrics and custom servables in TensorFlow Serving. For engineers operating high-traffic TF services.
- Live Course
- 5 modules
Get in touch
