Model Online / On-Site
Duration
Modules 5 modules

Course Objective & Scope

When microservice or operational database owners change a schema for their own application needs, downstream analytics and AI pipelines consuming those tables via CDC or event streams break. This programme teaches how to implement data contracts not as bureaucratic paperwork, but as versioned, automatically verified engineering interfaces—just like API contracts.

Using the OpenDataContractStandard (ODCS) and Data Contract CLI, managing schema evolution with JSON Schema/Avro/Protobuf, integrating Schema Registry and running Git-based CI/CD checks are covered hands-on. Participants produce a production-ready data contract example and a change-management workflow for one of their organisation's interfaces.

Who Should Attend?

  • Data platform, data engineering and streaming infrastructure teams
  • Software/backend developers and architects producing events and analytical data
  • Data governance specialists and data product owners
  • Prerequisite: familiarity with JSON/YAML, Git workflows and basic SQL or APIs

Key Takeaways

  • Author ODCS-compliant contracts covering schema, semantic rules, PII classification, freshness SLAs and ownership
  • Design safe schema evolution using backward and forward compatibility rules
  • Validate application schema changes automatically against contracts during pull requests
  • Generate dbt models, DDL statements and catalog metadata directly from contract definitions
  • Establish a frictionless versioning and deprecation workflow between producer and consumer teams

Curriculum

Why Data Contracts? The CDC Trap and Producer-Consumer Disconnect
  • Why replicating internal application tables directly via CDC is fragile
  • Treating datasets as versioned APIs and data product interfaces
Anatomy of a Contract and OpenDataContractStandard (ODCS)
  • Defining schema, field types, business semantics and quality constraints in YAML
  • SLAs, latency targets, retention policies and security/PII tags
Schema Evolution and Compatibility Rules
  • Serialization with JSON Schema, Avro and Protobuf plus Schema Registry
  • Backward, forward and full compatibility; major and minor versioning
Data Contract CLI and CI/CD Automation
  • Detecting breaking changes and running contract checks on Git pull requests
  • Generating SQL DDL, dbt staging models and catalog entries from a contract
Organisational Adoption and Incident Handling
  • Low-friction contract-first and code-first workflows for software teams
  • Routing contract violations, managing exceptions and deprecation timelines

Other Courses in This Category

  • Data Lineage and Enterprise Data Catalogs

    Building an enterprise data catalog with OpenMetadata and DataHub, managing business glossaries, automating metadata ingestion and tracking column-level lineage with OpenLineage.

    Data Science & AI-Ready Data Engineering Corporate
    Live Course
    5 modules
    View
  • Data Observability: Monitoring, Drift Detection and Incident Management

    Detecting freshness, volume, schema and distribution anomalies across data pipelines, preventing alert fatigue, running root-cause analysis and managing data incidents for data and AI platform teams.

    Data Science & AI-Ready Data Engineering Corporate
    Live Course
    5 modules
    View
  • Multimodal Data Ingestion: Document, Audio and Image Parsing

    High-accuracy extraction of structured text and metadata from complex PDF tables, scanned forms, audio recordings and images using OCR, Docling, Whisper and vision-language models.

    Data Science & AI-Ready Data Engineering Corporate
    Live Course
    5 modules
    View
  • dbt and Analytics Engineering Transformation Workflows

    Modular SQL transformations with dbt Core/Cloud, staging-intermediate-mart layers, Jinja macros, incremental models, snapshots, testing and CI/CD workflows for data and analytics engineers.

    Data Science & AI-Ready Data Engineering Corporate
    Live Course
    5 modules
    View
  • Data and AI Pipeline Orchestration with Apache Airflow

    Authoring production DAGs with Apache Airflow: TaskFlow API, dynamic task mapping, dataset/asset-aware scheduling, isolated execution on Kubernetes, error handling and AI/RAG pipeline orchestration.

    Data Science & AI-Ready Data Engineering Corporate
    Live Course
    5 modules
    View
  • Dirty Data, Clean CV: Real-World ML, Interviews & Career Dirty Data, Clean CV

    Advanced workshop on applying regression to messy real-world data, presenting model metrics to non-technical stakeholders, acing ML technical interviews, and building an impactful Data Science CV and GitHub portfolio.

    Data Science & AI-Ready Data Engineering Individual Workshop
    Live Course
    4 modules
    View

Get in touch