Course Objective & Scope
On structured tabular data, gradient boosted decision trees (GBDT) remain the industry standard for speed, accuracy and flexibility, yet calling them with default parameters taps only a fraction of their capability. This course dives into the internals of XGBoost, LightGBM and CatBoost (depth-wise, leaf-wise and oblivious trees, GOSS, EFB, ordered boosting), native categorical and missing-value handling, regularisation and early stopping.
Going beyond standard classification and regression, participants practise learning-to-rank with LambdaMART, quantile regression intervals, monotonic and interaction constraints, custom first/second-order objectives and low-latency serving via Treelite/ONNX on an enterprise case.
Who Should Attend?
- Data scientists and ML engineers working with tabular data
- Specialists building risk, pricing, recommendation and marketing models
- Teams looking to push performance and speed of existing GBDT models
- Prerequisite: Python, scikit-learn and basic decision-tree concepts
Key Takeaways
- Select between XGBoost, LightGBM and CatBoost based on data size and categorical density
- Tune num_leaves, min_child_samples, subsampling and L1/L2 regularisation systematically
- Enforce domain rules using monotonic and feature-interaction constraints
- Implement quantile, Poisson/Tweedie and custom gradient/hessian objectives
- Generate TreeSHAP explanations and package models with ONNX or Treelite for fast serving
Curriculum
GBDT Internals and Library Comparison
- Second-order Taylor expansion, histogram binning, GOSS and EFB
- XGBoost (depth-wise), LightGBM (leaf-wise) and CatBoost (oblivious trees)
Data Representation and Hyperparameter Strategy
- Native categoricals, missing-value routing and scale invariance
- Search-space design with Optuna, early stopping and overfitting control
Advanced Objective Functions
- Tweedie, Poisson, Cox survival and quantile losses
- Custom gradient and hessian functions aligned with business metrics
Constraints, Ranking and Explainability
- Controllable models with monotonic and interaction constraints
- Learning to rank with LambdaMART and TreeSHAP interaction values
Performance, Memory and Serving Optimisation
- DMatrix/Dataset memory management on large datasets and GPU training
- Low-latency inference with ONNX Runtime and Treelite
Other Courses in This Category
-
Automated model selection, hyperparameter search and multi-layer stacking with AutoGluon, FLAML and Optuna, covering compute budget management, leakage checks and moving from rapid baselines to production.
- Live Course
- 5 modules
-
Glass-box models (EBM, GAM), TreeSHAP/KernelSHAP, ALE, counterfactual explanations (DiCE), uncertainty quantification and regulatory compliance for producing auditable, actionable explanations in high-stakes decisions.
- Live Course
- 5 modules
-
Moving from correlation to causation with DAGs, propensity scores, Double Machine Learning (EconML/DoWhy), heterogeneous treatment effects (CATE) and Difference-in-Differences to measure price, campaign and policy impact.
- Live Course
- 5 modules
-
Collaborative model training across institutions or edge devices without centralising raw data: Flower framework, FedAvg/FedProx, non-IID data, differential privacy (Opacus) and secure aggregation.
- Live Course
- 5 modules
-
Intro to Machine Learning - Regression Models
Hands-on workshop covering Linear, Ridge, Lasso, and Polynomial regression models, feature engineering, MSE/RMSE/MAE/R² evaluation metrics, and regularization techniques.
- Live Course
- 3 modules
-
Incremental learning on streaming data with River, prequential evaluation, concept drift detection (ADWIN, Page-Hinkley) and contextual bandits for continuous adaptation in real-time systems.
- Live Course
- 5 modules
Get in touch
