Model Online / On-Site
Duration
Modules 5 modules

Course Objective & Scope

Token consumption and idle GPU capacity that look negligible during a pilot can quickly blow through budgets once an application rolls out across the enterprise. This course teaches how to calculate the unit economics of LLM workloads (true cost per request and per million tokens), find the break-even point between commercial APIs and self-hosted GPUs (factoring in utilisation), and cut token and compute spend without sacrificing quality.

Using trace data from a representative enterprise application, participants perform a cost breakdown and build an optimisation plan combining prompt compression, concise structured outputs, model tiering and higher GPU utilisation.

Who Should Attend?

  • AI, platform and FinOps engineers managing LLM infrastructure and API budgets
  • Software architects and technical product managers building high-volume AI products
  • Technology leaders responsible for unit cost and ROI modelling
  • Prerequisite: basic LLM API/serving concepts and comfort with cost/capacity spreadsheets

Key Takeaways

  • Calculate input, cached input, reasoning and output token costs per workflow
  • Model the true break-even point between self-hosted GPUs and managed APIs
  • Trim token bloat across system prompts, RAG context and tool schemas
  • Separate synchronous traffic from async/batch jobs to monetise idle GPU hours
  • Design a cost-per-successful-task dashboard segmented by feature, team and customer

Curriculum

LLM Unit Economics and Cost Anatomy
  • Input, output and reasoning token weights: why output tokens cost far more
  • Measuring cost per resolved task rather than raw cost per token
Self-Hosted GPU vs Commercial API Break-Even Math
  • Deriving cost per 1M tokens from hourly GPU cost and achieved tokens/s
  • How off-peak and weekend under-utilisation changes true unit cost
Token Budget Reduction Techniques
  • Pruning long system prompts, chat history and RAG chunk counts (reranking + top-k)
  • Controlling output tokens with compact JSON schemas, short keys and max_tokens limits
Architectural Cost Optimization
  • Model cascading: routing routine requests to small models and hard ones to frontier models
  • Shifting non-urgent workloads to Batch APIs or off-peak GPU queues
AI FinOps, Budget Quotas and Reporting
  • Tagging token spend by user, department and feature for internal chargeback
  • Runaway agent loop alerts and a cost/performance comparison report

Other Courses in This Category

  • LLM Model Serving and Deployment Fundamentals

    Shows how an open-weight LLM is served as an OpenAI-compatible API on a GPU server: inference engines, VRAM math, core serving settings and a first benchmark. An entry-level programme for developer and platform teams.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • LLM Serving with vLLM: Production Inference Deployment

    High-throughput LLM serving with vLLM: PagedAttention, continuous batching, tensor parallelism, prefix caching, multi-LoRA serving and Prometheus metrics. Teams leave with a production-ready vLLM deployment and benchmark report.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • Hugging Face Text Generation Inference (TGI) for LLM Serving

    Serving LLMs with Hugging Face Text Generation Inference (TGI): Docker setup, sharding, quantization, Messages API, structured output with guidance and Inference Endpoints. A hands-on programme for teams in the Hugging Face ecosystem.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • Running Local LLMs with Ollama

    Running open-weight models on laptops, workstations or servers with Ollama: model library, Modelfile, REST/OpenAI-compatible API, Open WebUI, embeddings and a shared team setup. For fast prototyping and confidential-data scenarios.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • LLM Serving Stack Mastery: FastAPI, vLLM, KServe and Triton

    Combines a FastAPI model API layer, vLLM for LLM inference, KServe for model management on Kubernetes and Triton for multi-framework serving in one architecture. Participants build an end-to-end, scalable model serving stack.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    6 modules
    View
  • LLM Quantization: INT8, INT4, FP8, FP16 and BF16 in Practice

    FP16/BF16, FP8, INT8 and INT4 precision options for LLMs; post-training quantization with GPTQ, AWQ, GGUF and bitsandbytes, measuring quality loss with evals and verifying VRAM and throughput gains through benchmarks.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View

Get in touch