Course Objective & Scope
As automation and LLM workflows scale, built-in execution logs quickly become insufficient to answer why a customer got a wrong response, which step took 40 seconds or why token spend tripled overnight. This course teaches how to make workflows and AI nodes observable, measurable and auditable in production.
Participants implement structured JSON logging, cross-system correlation_id propagation, LLM tracing (Langfuse, Arize Phoenix, OpenTelemetry), PII redaction and operational dashboards with actionable alerts.
Who Should Attend?
- Technical teams operating automation, integration and AI workflow platforms
- Software developers, data/MLOps engineers and DevOps/SRE specialists
- Operations leads tracking workflow SLAs, reliability and AI unit costs
- Enterprise IT, finance and insurance teams with strict auditability requirements
Key Takeaways
- Trace business transactions end to end across sub-flows and APIs with a
correlation_id - Capture prompt version, model, input/output tokens, latency and cost for every LLM call
- Redact personal data (PII) and secrets before payloads hit log storage
- Build dashboards tracking success rates, p95 latency, fallback rates and unit cost per run
- Configure low-noise alerts for error spikes, latency regressions and runaway token spend
Curriculum
The Three Pillars for Workflows: Logs, Metrics and Traces
- Why built-in execution history falls short and when to centralise logs
- Designing a standard JSON log schema and propagating
correlation_id
LLM and Agent Tracing (LLMOps Observability)
- Recording prompts, completions, tool calls and latency with Langfuse, Phoenix and OpenTelemetry
- Linking prompt versions and evaluation scores to execution traces
Cost, Latency and Business KPI Dashboards
- Tracking unit cost per run, p50/p95 latency, retry/fallback rates and human-override rates
- Attributing token and API spend by department and use case
Privacy in Logs: PII Redaction and Retention Policies
- Scrubbing national IDs, emails, phone numbers and credentials before persistence
- Role-based log access and KVKK/GDPR-compliant retention windows
Anomaly Detection, Alerting and Incident Triage
- Alerting on error spikes, silent trigger failures (dead-man's switch) and budget anomalies
- Hands-on lab: tracing a production incident to its root cause
Other Courses in This Category
-
Secure enterprise automation and LLM/agent workflows with least-privilege access, secret vaults, indirect prompt-injection defences, tool-calling guardrails and governance controls.
- Live Course
- 5 modules
-
Automate repetitive business tasks using triggers, conditions, data transformations and actions, building your first end-to-end workflow safely with examples in n8n, Power Automate and Make/Zapier.
- Live Course
- 5 modules
-
Map AS-IS processes, eliminate bottlenecks, standardise exception rules, design TO-BE workflows and build a prioritised automation backlog with ROI and feasibility scoring before automating.
- Live Course
- 5 modules
-
Combine deterministic workflow automation with LLM steps for classification, extraction and summarisation to process unstructured emails, PDFs and free text reliably.
- Live Course
- 5 modules
-
Design multi-step, context-aware enterprise workflows using LLM APIs, prompt chaining, routers, evaluator-optimizer loops and tool calling patterns.
- Live Course
- 5 modules
-
Extract structured data from invoices, contracts, delivery notes and scanned PDFs using OCR and vision-language models, validate it against business rules and push it into ERP or finance systems.
- Live Course
- 5 modules
Get in touch
