Model Online / On-Site
Duration
Modules 4 modules

Course Objective

To provide the technical architecture needed to transparently monitor the decision processes of production autonomous and multi-step agent systems (observability), bound runtime risks with guardrail policies, and enforce enterprise compliance (governance) standards.

Participants learn how to analyze agent trajectories, detect quality regressions, and prevent privilege escalation using Google Cloud and open-standard (OpenTelemetry) tooling.

Target Audience and Prerequisites

  • Target Audience: AI platform engineers, SRE/DevSecOps specialists, enterprise architects, and Responsible AI governance teams.
  • Prerequisites: Experience building LLM or agent applications in the cloud, understanding of core logging/monitoring concepts, and familiarity with IAM architectures.

Key Takeaways and Focus Areas

  • Tracing multi-step agent reasoning and tool call chains with OpenTelemetry, Cloud Trace, and Cloud Logging.
  • Automating trajectory evaluation, groundedness, and response quality metrics using Vertex AI Gen AI Evaluation Service.
  • Implementing Model Armor, PII redaction, prompt injection filters, and least privilege agent identity management.

Curriculum

Agent Telemetry, Distributed Tracing, and Logging
  • Tracing prompts, reasoning steps, and tool invocations with OpenTelemetry semantic conventions
  • Analyzing token consumption, latency, and error rates on Cloud Trace, Cloud Monitoring, and BigQuery
Agent Evaluation and Trajectory Analysis
  • Single-turn response and end-to-end agent trajectory evaluation with Gen AI Evaluation Service
  • Golden datasets, LLM-as-a-judge metrics, and continuous quality regression testing
Runtime Guardrail Layers and Content Safety
  • Blocking prompt injection, jailbreaks, and harmful outputs with Model Armor and safety filters
  • Detecting and redacting PII in inputs and outputs using Sensitive Data Protection (DLP)
Enterprise Agent Governance, IAM, and Audit Trails
  • Agent identities, delegated user authorization, and least-privilege IAM boundaries
  • Audit Logs, policy compliance, cost quotas, and human-in-the-loop approval gates

Other Courses in This Category

  • Enterprise AI Lifecycle with the IBM watsonx Platform

    Learn how to build enterprise AI solutions, feed them with hybrid data architectures, and govern them for regulatory compliance using IBM watsonx.ai, watsonx.data, and watsonx.governance.

    Enterprise AI Platforms & Agent Cloud Ecosystem Corporate
    Live Course
    4 modules
    View
  • Enterprise Digital Workers and Automation with IBM watsonx Orchestrate

    Learn hands-on how to design and manage digital assistants, custom skill catalogs, and multi-step workflow automations integrated with enterprise applications using IBM watsonx Orchestrate.

    Enterprise AI Platforms & Agent Cloud Ecosystem Corporate
    Live Course
    4 modules
    View
  • Stateful Multi-Agent Architectures and Production Orchestration with LangGraph

    Build production-grade multi-agent systems with cyclic workflows, persistent state checkpointing, human-in-the-loop approval flows, and LangSmith observability using LangGraph.

    Enterprise AI Platforms & Agent Cloud Ecosystem Corporate
    Live Course
    4 modules
    View
  • Open-Source Model and Agent Ecosystem: Self-Hosted Enterprise AI Stack

    Learn how to serve open-source models like Llama, Mistral, and Qwen on vLLM, integrate them with open-source agent frameworks, and build a self-hosted enterprise AI stack.

    Enterprise AI Platforms & Agent Cloud Ecosystem Corporate
    Live Course
    4 modules
    View
  • Enterprise AI Development with Microsoft Azure AI Foundry

    Model selection, enterprise RAG and agent development, content safety filters, automated evaluation, OpenTelemetry tracing and secure deployment on Microsoft Azure AI Foundry.

    Enterprise AI Platforms & Agent Cloud Ecosystem Corporate
    Live Course
    5 modules
    View
  • Building Multi-Agent Systems with Microsoft Agent Framework

    Enterprise single- and multi-agent orchestration, MCP tool integration, memory and stateful process workflows using Microsoft Agent Framework (unifying Semantic Kernel and AutoGen).

    Enterprise AI Platforms & Agent Cloud Ecosystem Corporate
    Live Course
    5 modules
    View

Get in touch