Model Online / On-Site
Duration
Modules 5 modules

Course Objective & Scope

A workflow that looks flawless in a demo can double-charge a customer or silently drop records in production when an API times out, an LLM returns malformed JSON or step 4 of 5 fails. This course teaches the engineering patterns required to make automation and AI workflows resilient in production.

Participants practice separating transient network/quota errors from permanent business errors, enforcing idempotency across retries, repairing malformed LLM outputs, falling back to secondary models and replaying failed payloads from a Dead-Letter Queue (DLQ).

Who Should Attend?

  • Automation builders and architects running business-critical production workflows
  • Integration, platform and SRE/DevOps engineers
  • Technical teams building multi-step LLM and agent pipelines
  • Prerequisite: hands-on experience building API-driven workflows

Key Takeaways

  • Classify failures into transient infrastructure errors, permanent payload errors, LLM schema errors and business exceptions
  • Configure exponential backoff with jitter, rate-limit throttling and circuit breakers
  • Guarantee exactly-once business effects across retries using idempotency keys and state tables
  • Recover from malformed LLM outputs or provider outages with repair prompts and model fallbacks
  • Design compensating actions (Saga pattern) and Dead-Letter Queue (DLQ) replay workflows

Curriculum

Failure Taxonomy: Deciding What to Retry
  • Transient failures (HTTP 429, 502, 503, timeouts) vs permanent errors (400, 401, 422 business rules)
  • How blind retries trigger retry storms and cascading outages
Retry Policies, Backoff and Circuit Breakers
  • Exponential backoff with jitter and honouring Retry-After headers
  • Circuit breakers and queue buffering when downstream APIs degrade
Idempotency and Handling Partial Failures (Saga Pattern)
  • Preventing duplicate emails, invoices or records when a flow re-runs
  • Compensating transactions when step N of a multi-system flow fails
LLM-Specific Failures: Schema Repair and Model Fallbacks
  • Handling malformed JSON, truncated responses and safety-filter refusals
  • Self-healing validation retries and failing over to a secondary model/provider
Dead-Letter Queues (DLQ), Alerting and Replay Operations
  • Persisting unrecoverable payloads with full context into a DLQ
  • Safe single-item and bulk replay workflows after fixing the root cause

Other Courses in This Category

Get in touch