Course Objective & Scope
When today's decision alters tomorrow's state and rewards—as in dynamic pricing, inventory replenishment, bidding, energy management and industrial control—supervised learning falls short. This programme covers Markov Decision Processes (MDPs), Bellman equations, value-based methods (Q-Learning, Double/Dueling DQN), policy-gradient and actor-critic algorithms (PPO, SAC), alongside pragmatic Contextual Bandits.
Special attention is given to enterprise constraints where reckless online exploration is unacceptable: simulator construction, Offline RL from logged interactions (CQL, IQL) and Off-Policy Evaluation (OPE). Using Gymnasium and Stable-Baselines3, participants build a custom decision environment and policy evaluation report.
Who Should Attend?
- Senior data scientists, ML engineers and control/automation engineers
- Dynamic pricing, recommendation/bidding and resource allocation teams
- Supply chain, energy grid and simulation-based optimisation specialists
- Prerequisite: Python, probability and basic deep learning (PyTorch)
Key Takeaways
- Frame a sequential business problem as a Contextual Bandit or MDP with well-behaved states, actions and rewards
- Implement a custom Gymnasium environment that enforces operational constraints
- Train DQN, PPO and SAC agents with Stable-Baselines3 while avoiding reward-hacking pitfalls
- Use Offline RL and Off-Policy Evaluation (IPS, Doubly Robust) to learn safely from historical logs
- Benchmark learned policies against heuristic rules with hard safety guardrails
Curriculum
MDPs, Bellman Equations and Contextual Bandits
- States, actions, rewards, discounting and exploration (LinUCB, Thompson Sampling)
- Deciding when a Contextual Bandit suffices vs full multi-step RL
Value-Based Deep RL: The DQN Family
- From tabular Q-Learning to Deep Q-Networks: replay buffers and target networks
- Double DQN, Dueling DQN and discrete action spaces
Policy Gradients and Actor-Critic: PPO and SAC
- REINFORCE, Generalized Advantage Estimation (GAE) and PPO
- Continuous control and allocation with Soft Actor-Critic (SAC)
Simulator Design and Reward Engineering
- Building custom enterprise environments with the Gymnasium API
- Sparse rewards, preventing reward hacking and action masking for hard constraints
Offline RL and Off-Policy Evaluation (OPE)
- Out-of-distribution action extrapolation on static logs and CQL/IQL
- Pre-deployment policy evaluation using IPS and Doubly Robust estimators
Other Courses in This Category
-
Deep fraud models over transaction sequences, session clickstreams and payment networks: Transformer/RNN sequence encoders, autoencoder anomaly scores, GNNs for fraud rings and hybrid GBDT+DL architectures.
- Live Course
- 5 modules
-
Deep defect and fault detection across inspection images, acoustic/vibration signals and system logs using PatchCore/PaDiM, Anomalib, YOLO/segmentation, 1D-CNNs/autoencoders and normal-only training.
- Live Course
- 5 modules
-
Multivariate, long-horizon and probabilistic time-series forecasting with N-BEATS, N-HiTS, Temporal Fusion Transformer (TFT), PatchTST, DeepAR and TCN using NeuralForecast and PyTorch Forecasting.
- Live Course
- 5 modules
-
Large-scale deep recommendation architectures: Two-Tower retrieval with ANN (FAISS/Qdrant), sequential session recommenders (SASRec/BERT4Rec), DeepFM/DCN-v2/DLRM ranking and multi-task MMoE learning.
- Live Course
- 5 modules
-
Text preprocessing, Turkish morphology, regex, tokenisation, TF-IDF, word/sentence embeddings and building first text classification, NER and sentiment pipelines with spaCy, scikit-learn and Hugging Face.
- Live Course
- 5 modules
-
Advanced NLP under scarce labels: domain-adaptive pretraining (MLM), DeBERTa/RoBERTa fine-tuning, LoRA/PEFT, few-shot learning with SetFit, weak supervision and label cleaning (Cleanlab), long documents and cross-lingual transfer.
- Live Course
- 5 modules
Get in touch
