Course Objective & Scope
TensorFlow Serving is the standard tool for serving TensorFlow or Keras models through a high-performance C++ server without coupling production to training-time Python code. This course walks step by step from exporting a model in the SavedModel format to querying predictions over REST and gRPC endpoints.
Participants inspect model signatures with saved_model_cli, embed pre-processing layers into the model graph and manage multiple model versions with TensorFlow Serving running in Docker. They leave with a reusable SavedModel export and serving template ready for internal projects.
Who Should Attend?
- Data scientists building models with TensorFlow or Keras
- ML and software engineers deploying models behind a standard serving runtime
- Platform teams adding TensorFlow Serving to their MLOps pipelines
- Prerequisite: basic Python and experience training models with TensorFlow/Keras
Key Takeaways
- Export Keras and TensorFlow models in the production-ready SavedModel format
- Define SignatureDefs and inspect them using
saved_model_cli - Run TensorFlow Serving with Docker and send prediction requests via REST and gRPC
- Embed pre-processing layers inside the model to prevent training-serving skew
- Roll out new model versions seamlessly using the versioned directory layout
Curriculum
SavedModel Format and Internal Structure
- Directory layout:
saved_model.pb, variables and assets - Using Keras
exportandtf.saved_model.save
SignatureDefs and Pre-Processing Layers
- Custom serving functions with
@tf.functionandinput_signature - Inspecting input/output tensors and running local tests with
saved_model_cli - Embedding Keras preprocessing layers into the model graph
Running TensorFlow Serving with Docker
- Launching the official Docker image and configuring ports
- Querying model status and metadata endpoints
Requesting Predictions via REST and gRPC Clients
instancesandinputsJSON formats in the REST API- Writing a Python gRPC client with
tensorflow-serving-api
Version Management and Model Warmup
- Versioned directories and basic version policy settings (latest, specific)
- Creating warmup records to eliminate cold-start latency on first requests
Other Courses in This Category
-
Multi-model configurations, dynamic batching, XLA/TensorRT and mixed-precision acceleration, GPU memory tuning, Prometheus metrics and custom servables in TensorFlow Serving. For engineers operating high-traffic TF services.
- Live Course
- 5 modules
-
Operating TensorFlow Serving at scale on Docker and Kubernetes: pulling models from object storage, GPU scheduling, readiness/liveness probes, HPA/KEDA autoscaling and KServe integration. For MLOps and platform engineers.
- Live Course
- 5 modules
-
Deploying PyTorch models with TorchServe: writing custom Handlers, packaging .mar files with torch-model-archiver, using the Inference and Management APIs, and managing workers and versions. For data scientists and ML engineers using PyTorch.
- Live Course
- 5 modules
-
Multi-model DAG workflows in TorchServe, acceleration with torch.compile/ONNX/TensorRT, multi-GPU and large-model parallel serving, custom Prometheus metrics and production security. For senior ML and MLOps engineers.
- Live Course
- 5 modules
-
Dynamic batching and high-volume bulk scoring in TorchServe: calibrating batch_size and max_batch_delay, writing vectorised Handlers, padding variable-length inputs and integrating queues. For ML and data engineers.
- Live Course
- 5 modules
-
Converting PyTorch, TensorFlow and scikit-learn models to ONNX, validating graphs, accelerating inference with ONNX Runtime and Execution Providers (CPU, CUDA, TensorRT) and quantization. For ML and deployment engineers.
- Live Course
- 5 modules
Get in touch
