Model Online / On-Site
Duration
Modules 5 modules

Course Objective & Scope

AI policies only provide real assurance when translated into testable internal controls and verifiable audit trails. This course teaches internal control, risk and internal audit professionals how to audit both predictive machine learning and generative AI or agent systems from end to end.

Participants build an AI Risk and Control Matrix (RCM) spanning data integrity, access control, model and prompt change management, evaluation and guardrail testing, third-party oversight and human-in-the-loop evidence. In the workshop, teams construct an AI Internal Audit Work Programme and a Prepared-by-Client (PBC) evidence request list.

Who Should Attend?

  • Internal audit, IT audit and independent assurance professionals
  • Internal control, operational risk and compliance (2nd Line) specialists
  • AI product owners, AI CoE leads and MLOps/LLMOps managers preparing for audits
  • Information security and quality management system auditors

Key Takeaways

  • Build an AI Risk and Control Matrix (RCM) with preventive and detective controls across the lifecycle
  • Audit RAG permission leaks, unmanaged prompt edits, guardrail bypasses and nominal human oversight
  • Design tests of control design effectiveness and operating effectiveness
  • Specify concrete audit evidence requests (model cards, eval logs, override records, vendor DPAs)
  • Write actionable AI audit findings with clear root causes and remediation plans

Curriculum

Defining the AI Audit Universe and Risk-Based Audit Planning
  • Scoping the AI audit universe across registered systems, SaaS AI features and Shadow AI
  • Entity-level governance audits versus system-specific technical/operational audits
  • Prioritising AI engagements in the annual internal audit plan
The Layered AI Risk and Control Matrix (RCM)
  • Data and RAG layer: data provenance, PII filtering and document-level access controls (ACLs)
  • Model and prompt layer: version control, change approvals and golden-set regression evals
  • Output and workflow layer: active guardrails, human-oversight logs and rate/budget limits
Control Testing Techniques and Evidence (PBC) Collection
  • Testing design effectiveness versus sampling for operating effectiveness
  • How non-coding auditors read eval scorecards, red-team reports and trace logs
  • Common audit findings: unapproved prompt changes in production, stale RAG sources and rubber-stamping
Auditing Third-Party Models and Cloud AI Services
  • Verifying vendor contracts, SOC 2 / ISO 42001 attestations and zero-data-retention configurations
  • Inspecting API key hygiene, RBAC permissions and usage/spend anomalies
Workshop: AI Audit Work Programme and Finding Remediation
  • Building a complete audit checklist and PBC list for a sample RAG or scoring system
  • Rating audit findings and validating management remediation plans (CAPA)

Other Courses in This Category

Get in touch