Course Objective & Scope
For teams that can already build basic models, the next step is to raise performance and reliability systematically. The programme covers boosting and stacking strategies, Bayesian hyperparameter search with Optuna, resampling and cost-sensitive learning for imbalanced classes, probability calibration and loss functions designed around the business objective.
Work is based on a case taken from or modelled on the organisation; every technique is measured against a baseline, experiments are tracked in MLflow, and the group produces a reproducible experiment report.
Who Should Attend?
- Data scientists building models for production
- ML engineers and analytics team leads
- Quantitative analysts and risk modelling specialists
- Prerequisite: experience training and evaluating models with scikit-learn
Key Takeaways
- Build ensemble and stacking setups with a correct validation scheme
- Design efficient hyperparameter searches with Optuna
- Choose thresholds and class weights that reflect business cost on imbalanced data
- Calibrate model probabilities with Platt scaling and isotonic regression
- Track experiments in MLflow and report them comparably
Curriculum
Ensemble Methods and Stacking
- Bagging, boosting and blending compared
- Leak-free stacking with out-of-fold predictions
Hyperparameter Optimisation
- Grid, random and Bayesian search; Optuna pruning
- Honest performance estimates with nested cross-validation
Imbalanced Data and Cost-Sensitive Learning
- SMOTE, class weights and focal loss
- Setting decision thresholds from business cost
Calibration and Custom Loss Functions
- Reliability diagrams, Brier score, isotonic regression
- Quantile and asymmetric losses aligned with business goals
Experiment Management and Reproducibility
- Tracking experiments, parameters and artefacts with MLflow
- Improvement report against the baseline on the case
Other Courses in This Category
-
Takes analysts beyond classic reporting with segmentation, driver analysis, anomaly flagging and simple predictive models in Python, scikit-learn and SHAP, producing analyses that tie insight to business decisions.
- Live Course
- 5 modules
-
Categorical encoding, time and window-based features, target encoding, interaction and text features, feature selection and feature stores: a systematic approach to the step that most improves model performance on tabular data.
- Live Course
- 5 modules
-
Choosing the right metrics, evaluation tied to business KPIs, error analysis, fairness checks and model explanation with SHAP, LIME and PDP, so models can be presented to stakeholders reliably and understandably.
- Live Course
- 5 modules
-
Independent model validation: data and conceptual soundness checks, backtesting, stability (PSI), challenger models, stress tests and findings reporting, producing a validation report aligned with a model risk management framework.
- Live Course
- 5 modules
-
Time series forecasting with ARIMA, ETS, Prophet and LightGBM-based approaches, working hands-on with seasonality, exogenous variables, hierarchical series, backtesting and prediction intervals to build reliable forecasts.
- Live Course
- 5 modules
-
Detecting unusual behaviour in unlabelled data with statistical methods, Isolation Forest, LOF, autoencoders and time series anomaly techniques, including threshold setting, alert fatigue and evaluation strategies.
- Live Course
- 5 modules
Get in touch
