Course Objective & Scope
In enterprise NLP, the hardest obstacle is rarely the model architecture—it is domain-heavy jargon across legal contracts, clinical notes, technical logs or financial filings combined with a shortage of clean human labels. This advanced programme covers continual Domain-Adaptive Pretraining (DAPT/MLM) on internal corpora, advanced fine-tuning of DeBERTa-v3 and Turkish/multilingual encoders (layer-wise learning-rate decay, R-Drop, multi-sample dropout), few-shot learning with SetFit and parameter-efficient LoRA adapters.
Participants also practise cleaning noisy annotations with Cleanlab, distilling LLM teacher labels into fast compact encoders, and modelling long documents beyond 512 tokens via hierarchical pooling and ModernBERT/long-context encoders.
Who Should Attend?
- Senior data scientists and ML engineers building production NLP systems
- Teams processing specialised legal, financial, clinical, insurance and operational text
- Engineers replacing costly LLM calls with fast, high-accuracy task-specific encoders
- Prerequisite: Python, PyTorch and prior NLP / Hugging Face experience
Key Takeaways
- Run Domain-Adaptive Pretraining (MLM) on unlabelled internal text to master company jargon
- Train accurate classifiers from only dozens of examples using SetFit and contrastive fine-tuning
- Filter noisy human or synthetic LLM labels with Cleanlab and distil them into a compact encoder
- Process multi-page contracts and reports using sliding-window aggregation and long-context encoders (ModernBERT)
- Optimise thresholds and losses for multi-label, long-tail and cross-lingual transfer scenarios
Curriculum
Domain Adaptation and Continual Pretraining (DAPT/TAPT)
- Vocabulary expansion and Masked Language Modeling (MLM) on internal corpora
- Selecting between Turkish-specific and multilingual (XLM-R, mDeBERTa) backbones
Advanced Fine-Tuning and Stability Recipes
- Layer-wise learning-rate decay (LLRD), R-Drop and adversarial (FGM) regularisation
- Parameter-efficient fine-tuning with LoRA/DoRA and multi-task heads
Few-Shot NLP with SetFit and Active Learning
- Contrastive sentence-pair fine-tuning with SetFit on 8–64 labels per class
- Active learning loops using uncertainty and diversity sampling
Weak Supervision, LLM Distillation and Cleanlab
- Generating synthetic training data with an LLM teacher and distilling into an encoder
- Detecting and fixing mislabelled records with Cleanlab (Confident Learning)
Long Documents and Multi-Label / Long-Tail Classification
- Chunk-pooling, hierarchical encoders and long-context ModernBERT (8k+ tokens)
- Asymmetric/Focal losses and per-class calibrated decision thresholds
Other Courses in This Category
-
End-to-end neural NLP covering subword tokenisation, BiLSTM-CRF and Transformer (BERT, DeBERTa, T5) architectures, with hands-on sequence tagging, seq2seq summarisation and question answering in PyTorch and Hugging Face.
- Live Course
- 5 modules
-
Running NLP models in production at low latency and cost: ONNX/Optimum quantisation, Triton/FastAPI serving, dynamic batching, PII redaction (Presidio), behavioural testing (CheckList) and text drift monitoring.
- Live Course
- 5 modules
-
Turning unstructured documents into structured tables and JSON: hierarchical and multi-label classification, GLiNER and Transformer NER, relation and event extraction, layout-aware PDF parsing and schema-validated extraction.
- Live Course
- 5 modules
-
Enterprise search beyond keyword matching: Sentence-Transformers (bi-encoders), multilingual embeddings, BM25/SPLADE + dense hybrid retrieval, Reciprocal Rank Fusion, Cross-Encoder reranking, Qdrant/pgvector and NDCG@k evaluation.
- Live Course
- 5 modules
-
Deep Learning with PyTorch
Comprehensive deep learning training with PyTorch 2.x covering tensor operations, autograd, CNNs, Transformers, LLM fine-tuning (LoRA/QLoRA), diffusion models, GNNs, and production deployment.
- Live Course
- 12 modules
-
Neural networks, forward pass, backpropagation, activation and loss functions, AdamW, dropout and batch normalisation, giving engineers the skills to build and train their first deep learning models in PyTorch.
- Live Course
- 5 modules
Get in touch
