Model Online / On-Site
Duration
Modules 5 modules

Course Objective & Scope

When data must not leave the organisation, Ollama is the most practical way to try LLMs locally within minutes. The programme works hands-on through pulling and running models, GGUF quantization levels, defining system prompts and parameters with a Modelfile, the REST and OpenAI-compatible APIs, and the Python/JavaScript SDKs.

Participants set up a team chat interface with Open WebUI, build a small RAG prototype with embedding models and configure secure use of Ollama on a shared server. The course also clarifies where Ollama is enough and where moving to a production engine such as vLLM is needed.

Who Should Attend?

  • Software developers wanting to try LLMs locally
  • Data scientists and analysts working with confidential data
  • Technical leads setting up a rapid prototyping environment for their team
  • Prerequisite: basic command-line skills

Key Takeaways

  • Install Ollama on Windows, macOS, Linux or Docker
  • Pick a model and quantization level that fits the hardware
  • Create organisation-specific model variants with a Modelfile
  • Connect the Ollama API to applications and Open WebUI
  • Assess Ollama's limits for production use

Curriculum

Installation and Model Library
  • Installation per platform and GPU support
  • Model selection, tags and GGUF quantization levels
Modelfile and Customisation
  • Defining system prompts, parameters and templates
  • Importing GGUF files and adapters
Integration via API and SDK
  • REST and OpenAI-compatible endpoints
  • Python/JavaScript SDK, tool calling and structured output
Open WebUI and Local RAG
  • Setting up a chat interface for the team
  • Document Q&A prototype with embedding models
Shared Use and Limits
  • Server setup, access control and concurrency settings
  • When to move to vLLM/TGI?

Other Courses in This Category

  • LLM Serving Stack Mastery: FastAPI, vLLM, KServe and Triton

    Combines a FastAPI model API layer, vLLM for LLM inference, KServe for model management on Kubernetes and Triton for multi-framework serving in one architecture. Participants build an end-to-end, scalable model serving stack.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    6 modules
    View
  • LLM Quantization: INT8, INT4, FP8, FP16 and BF16 in Practice

    FP16/BF16, FP8, INT8 and INT4 precision options for LLMs; post-training quantization with GPTQ, AWQ, GGUF and bitsandbytes, measuring quality loss with evals and verifying VRAM and throughput gains through benchmarks.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • Dynamic Quantization for Neural Networks and LLMs

    Accelerating models with dynamic quantization without calibration data: PyTorch and ONNX Runtime dynamic quantization, FP8 dynamic activation scaling, comparison with static methods and measurement on CPU and GPU.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • Quantization-Aware Training, QLoRA Fine-Tuning and Quantized Serving

    Covers quantization end to end from training to serving: 4-bit fine-tuning with QLoRA, quantization-aware training (QAT), merging adapters, re-quantizing and serving in vLLM. For teams adapting models to company data on a single GPU.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • GPU Capacity Planning and Hardware Sizing for LLM Inference

    GPU capacity planning for LLM workloads: deriving VRAM, GPU count and server needs from model size, context length, concurrent users and SLO targets, plus a cost comparison of H100/H200/B200, L40S and cloud options.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View
  • In-House LLM Platform Architecture: From Hardware Sizing to Air-Gapped Production

    End-to-end in-house LLM platform design for data residency and regulated environments: GPU sizing, Kubernetes cluster, vLLM/Triton serving, model gateway, security and air-gapped production architecture. Built for platform and infrastructure teams.

    Local & On-Premise LLM Platforms Corporate
    Live Course
    5 modules
    View

Get in touch