Course Objective & Scope
A minor system prompt edit, model version upgrade or new RAG data source in an LLM application can silently reintroduce previously fixed vulnerabilities. Instead of one-off manual checks, this course teaches how to build automated, measurable and repeatable security evaluation suites (security evals) using promptfoo, garak and UK AISI Inspect AI.
Participants combine deterministic assertions with LLM-as-a-judge scorers to set up a security regression pipeline in GitHub Actions or GitLab CI and define pre-release security gates for their own applications.
Who Should Attend?
- Software, AI and LLMOps engineers building LLM applications
- Application security (AppSec) and DevSecOps teams
- Quality assurance (QA) and test automation engineers
- Prerequisite: familiarity with CI/CD workflows and basic Python or YAML/TypeScript
Key Takeaways
- Curate an enterprise adversarial test dataset covering OWASP LLM and Agentic risk categories
- Benchmark multiple models and prompt variants for security posture using promptfoo
- Write deterministic assertions and rubric-based LLM-as-a-judge evaluators
- Integrate automated security gates and a cost-aware tiered testing strategy into CI/CD
- Track Attack Success Rate (ASR) and False Refusal Rate (FRR) across releases
Curriculum
Security Testing for Probabilistic Systems
- Moving from unit tests to security evals: reproducibility, temperature and sampling
- Balancing Attack Success Rate (ASR) against over-refusal / False Refusal Rate (FRR)
Designing Adversarial Test Datasets
- Curating golden and adversarial datasets from HarmBench, CyberSecEval and internal incidents
- Test cases for prompt injection, data leakage, unauthorised tool calls and policy violations
Evaluation Automation with promptfoo and Inspect AI
- YAML/Python test definitions, variable templates and multi-turn scenarios
- Combining deterministic regex/JSON schema assertions with LLM-as-a-judge rubrics
CI/CD Integration and Regression Gates
- Fast smoke evals on pull requests and comprehensive nightly garak scans
- Pass/fail thresholds, build blocking policies and cost/runtime optimisation
Continuous Monitoring and Security Scorecards
- Catching security drift across model upgrades and prompt revisions
- Feeding novel attack payloads captured in production back into the regression suite
Other Courses in This Category
-
Teaches how to design enterprise LLM, RAG and agent platforms on Zero Trust principles using a central AI gateway, network and tenant isolation, KMS key management, secure MCP integration and auditable telemetry.
- Live Course
- 5 modules
-
Comprehensive training on PII masking, reversible pseudonymisation, anonymisation and data residency architectures to adopt LLMs without exposing personal or confidential corporate data.
- Live Course
- 5 modules
-
Hands-on training on building a low-latency PII detection, masking and de-pseudonymisation gateway in front of LLM APIs using Microsoft Presidio, GLiNER and Turkish rule/NER recognisers.
- Live Course
- 5 modules
-
Teaches how to replace production customer data in testing, analytics and LLM evaluation environments using k-anonymity, differential privacy and SDV/LLM-based synthetic data generation.
- Live Course
- 5 modules
-
Teaches how to scrub personal data from LLM and agent telemetry in OpenTelemetry, Langfuse and SIEM logs, and engineer retention policies, access controls and KVKK compliance into observability pipelines.
- Live Course
- 5 modules
-
Covers Agent Goal Hijack—the top OWASP Agentic AI risk—showing how attackers divert autonomous planning loops, persistent memory and multi-step execution, and how to defend against it with intent verification.
- Live Course
- 5 modules
Get in touch
