Course Objective & Scope
Managing warehouse transformations through sprawling stored procedures and copy-pasted SQL scripts turns every metric change into weeks of debugging and impact analysis. This training brings version control, modularity, automated testing and living documentation to the transformation layer using dbt (data build tool).
Project structure, ref and source functions, Jinja templating and custom macros, incremental strategies for large tables (merge, insert_overwrite, microbatch), dbt snapshots for SCD Type 2, model contracts and Slim CI are covered hands-on. Participants refactor one of their organisation's transformation workflows into a production-ready dbt project.
Who Should Attend?
- Data engineers building SQL transformations on data warehouses and lakehouses
- Analytics engineers and BI developers translating business logic into modular data models
- Data scientists standardising analytical and ML feature tables
- Prerequisite: strong SQL skills and basic familiarity with Git
Key Takeaways
- Structure a maintainable dbt project across staging, intermediate and marts layers
- Choose between view, table, ephemeral and incremental materializations based on volume and query patterns
- Eliminate repetitive SQL using Jinja macros and community packages
- Track historical changes (SCD Type 2) and point-in-time states with dbt snapshots
- Enforce pre-deployment quality using data tests, unit tests, model contracts and Slim CI
Curriculum
Analytics Engineering and dbt Project Architecture
- Responsibilities of sources, staging, intermediate and marts layers
- Building the DAG (ref/source), naming conventions and folder structure
Materialization Strategies and Incremental Models
- View, table, ephemeral and incremental (append, merge, insert_overwrite, microbatch)
- Managing full-refresh runs and schema changes on large event tables
Jinja, Macros, Hooks and Package Management
- Jinja control flow, target-aware variables and writing custom macros
- Using dbt_utils, pre/post-hooks and declarative permission grants
Snapshots, Testing and Model Contracts
- Capturing SCD Type 2 history with dbt snapshots
- Generic/singular data tests, unit tests and schema-enforced model contracts
Documentation, dbt Mesh and CI/CD Workflows
- Automated catalog and lineage generation; declaring BI/AI consumers with exposures
- State-aware Slim CI (state:modified+) and multi-team dbt Mesh patterns
Other Courses in This Category
-
Authoring production DAGs with Apache Airflow: TaskFlow API, dynamic task mapping, dataset/asset-aware scheduling, isolated execution on Kubernetes, error handling and AI/RAG pipeline orchestration.
- Live Course
- 5 modules
-
Dirty Data, Clean CV
Advanced workshop on applying regression to messy real-world data, presenting model metrics to non-technical stakeholders, acing ML technical interviews, and building an impactful Data Science CV and GitHub portfolio.
- Live Course
- 4 modules
-
Warehouse reliability on object storage: Apache Iceberg table internals, metadata tree, hidden partitioning, time travel, REST Catalog, multi-engine querying (Trino, Spark, DuckDB) and table maintenance.
- Live Course
- 5 modules
-
Centralising enterprise metrics and dimensions with dbt Semantic Layer (MetricFlow) and Cube so BI tools, Text-to-SQL systems and analytical AI agents return consistent numbers.
- Live Course
- 5 modules
-
Metadata engineering to boost retrieval precision and security in RAG and agent systems: schema design, automated extraction, pre-filtering, temporal validity and ACL permission filtering.
- Live Course
- 5 modules
-
Web Scraping
Hands-on Web Scraping training covering scalable data extraction from static and dynamic web sources using BeautifulSoup, Scrapy, and Playwright, bypassing anti-bot defenses, LLM-powered data extraction, and data validation with Pandas.
- Live Course
- 13 modules
Get in touch
