Model Online / On-Site
Duration
Modules 5 modules

Course Objective & Scope

How a model is called directly affects latency, client compatibility and maintenance cost. This course covers REST API design principles (resource model, HTTP methods, status codes, OpenAPI) side by side with the building blocks of gRPC (Protocol Buffers, unary and streaming RPCs, HTTP/2).

Participants build the same prediction service as REST with FastAPI and as gRPC with grpcio, then compare payload size and response time. They leave with an API design checklist and working sample services for both protocols.

Who Should Attend?

  • Backend and AI developers building model services
  • Data scientists exposing models to other teams
  • Platform engineers designing service-to-service communication
  • Prerequisite: basic programming experience in Python

Key Takeaways

  • Design REST endpoints and an OpenAPI schema for a prediction service
  • Write message and service definitions with Protocol Buffers
  • Implement unary and server-streaming gRPC calls
  • Compare REST and gRPC on latency, payload and client support
  • Select and justify the right protocol for a given scenario

Curriculum

HTTP and REST Design Principles
  • Resources, methods and status codes
  • Versioning and error response standards
A Prediction Service Contract with OpenAPI
  • Request/response schemas and validation
  • Automatic documentation with FastAPI
Protocol Buffers and gRPC Basics
  • The .proto file, message and service definitions
  • Code generation and communication over HTTP/2
Streaming RPCs and Deadline Management
  • Unary, server and bidirectional streaming
  • Deadlines, retries and error codes
Comparative Lab: REST vs gRPC
  • Serving the same model with both protocols
  • Measuring latency and payload, selection criteria

Other Courses in This Category

  • Synchronous vs Asynchronous Inference

    Synchronous and asynchronous inference patterns: request-response, queue-based processing (Celery, Redis, Kafka), job polling, webhooks and async Python. For engineers designing long-running model calls that scale.

    Model Serving & Deployment Corporate
    Live Course
    5 modules
    View
  • Batch, Real-Time and Streaming Inference Fundamentals

    A comparison of the three core inference modes: batch scoring with Spark and Airflow, REST-based real-time services and streaming inference with Kafka/Flink. For data and ML teams choosing the right mode for a business need.

    Model Serving & Deployment Corporate
    Live Course
    5 modules
    View
  • Model Serialization and Model File Formats

    Saving models safely and portably: pickle/joblib risks, TorchScript, SavedModel, ONNX, safetensors and GGUF formats, packaging with MLflow and version compatibility. For ML engineers and data scientists.

    Model Serving & Deployment Corporate
    Live Course
    5 modules
    View
  • Model Serving Architectures

    Model serving architecture patterns: embedded models, model-as-a-service, serverless, edge, multi-model and LLM gateway designs, and how platforms like KServe, Triton, Ray Serve and vLLM fit in. For architects and senior ML engineers.

    Model Serving & Deployment Corporate
    Live Course
    5 modules
    View
  • Latency, Throughput and Cost Trade-offs in Model Serving

    Balancing p95/p99 latency, throughput and unit cost in model serving: dynamic batching, quantization, caching, GPU/CPU selection and load-testing optimisation. For ML and platform engineers building scalable, cost-effective inference infrastructure.

    Model Serving & Deployment Corporate
    Live Course
    5 modules
    View
  • Security for Model Serving and Inference APIs

    End-to-end security for inference services: mTLS, OAuth2/JWT, rate limiting, safe serialization, container hardening, defence against model extraction/adversarial attacks and data privacy. For ML, platform and AppSec engineers.

    Model Serving & Deployment Corporate
    Live Course
    5 modules
    View

Get in touch