Model Online / On-Site
Duration
Modules 5 modules

Course Objective & Scope

In many organisations, each new AI initiative spins up shadow pipelines and isolated vector databases alongside the core data platform, quickly creating security gaps, conflicting answers and runaway infrastructure costs. This advanced programme covers the architectural principles needed to unify analytics and AI workloads on a coherent foundation.

Batch and streaming ingestion, open lakehouse storage, online/offline feature stores, hybrid vector search architectures, semantic layers for LLMs, ACL propagation and cost/latency trade-offs are examined in depth. Participants assess their current platform and produce a target AI-ready reference architecture and migration roadmap.

Who Should Attend?

  • Data and solution architects designing enterprise data and AI infrastructure
  • Technical leads of data platform, MLOps and AI platform teams
  • Engineering managers responsible for technology selection and architectural standards
  • Prerequisite: experience with data warehouses/lakehouses, distributed systems and cloud architecture

Key Takeaways

  • Right-size batch, micro-batch and streaming (Kafka/Flink) layers around AI latency requirements
  • Design storage architectures that minimise duplication across tabular tables, feature stores and vector indexes
  • Position a semantic layer so Text-to-SQL and analytical agents consume governed metrics
  • Architect security patterns that propagate source RBAC/ABAC permissions all the way to RAG and agent queries
  • Deliver an enterprise AI-ready data checklist, semantic layer blueprint and phased architecture roadmap

Curriculum

AI-Ready Reference Architecture and Design Principles
  • Data access patterns across BI, predictive ML, RAG and agentic workloads
  • Unified platform topologies that eliminate shadow data pipelines
Unified Storage: Lakehouse, Feature Store and Vector Layer
  • Multimodal data management on open table formats (Iceberg/Delta)
  • Choosing between dedicated vector databases, integrated indexes and online/offline feature stores
Real-Time Context and Streaming Pipelines
  • Live context and feature updates with CDC (Debezium), Kafka and Flink
  • Trade-offs between freshness SLAs and infrastructure cost
Semantic Layer, Knowledge Graphs and AI Access Patterns
  • Standardising business entities and metrics through a semantic layer
  • MCP and API gateway patterns for safe agent data access
Security, ACL Propagation and Multi-Tenant Governance
  • Row- and document-level RBAC/ABAC enforcement from source to vector index
  • Data sovereignty, KVKK/EU AI Act auditability and data FinOps

Other Courses in This Category

  • Automated Data Quality Testing for Analytics and AI

    Writing schema, business-rule, referential integrity and statistical distribution tests with dbt tests, Soda Core and Great Expectations to catch bad data in CI/CD and staging.

    Data Science & AI-Ready Data Engineering Corporate
    Live Course
    5 modules
    View
  • Data Contracts: Producer-Consumer Agreements and Schema Governance

    Practical training on securing data exchange between software producers and data/AI consumers using OpenDataContractStandard (ODCS), Schema Registry and automated CI/CD checks.

    Data Science & AI-Ready Data Engineering Corporate
    Live Course
    5 modules
    View
  • Data Lineage and Enterprise Data Catalogs

    Building an enterprise data catalog with OpenMetadata and DataHub, managing business glossaries, automating metadata ingestion and tracking column-level lineage with OpenLineage.

    Data Science & AI-Ready Data Engineering Corporate
    Live Course
    5 modules
    View
  • Data Observability: Monitoring, Drift Detection and Incident Management

    Detecting freshness, volume, schema and distribution anomalies across data pipelines, preventing alert fatigue, running root-cause analysis and managing data incidents for data and AI platform teams.

    Data Science & AI-Ready Data Engineering Corporate
    Live Course
    5 modules
    View
  • Multimodal Data Ingestion: Document, Audio and Image Parsing

    High-accuracy extraction of structured text and metadata from complex PDF tables, scanned forms, audio recordings and images using OCR, Docling, Whisper and vision-language models.

    Data Science & AI-Ready Data Engineering Corporate
    Live Course
    5 modules
    View
  • dbt and Analytics Engineering Transformation Workflows

    Modular SQL transformations with dbt Core/Cloud, staging-intermediate-mart layers, Jinja macros, incremental models, snapshots, testing and CI/CD workflows for data and analytics engineers.

    Data Science & AI-Ready Data Engineering Corporate
    Live Course
    5 modules
    View

Get in touch