Advanced Large Language Model Fine-Tuning Training Course

Course overview

Organisations rarely need a generic chatbot; they need language models that reliably perform a defined task using their terminology, policies, document types and quality thresholds. Fine-tuning can improve classification, extraction, drafting and domain-specific assistance, but poorly prepared data, unsuitable training objectives and weak evaluation can produce memorisation, regressions, unsafe outputs and expensive models that cannot be justified in production. This course equips practitioners to make defensible fine-tuning decisions rather than treating training as a trial-and-error exercise.

Participants work through the end-to-end fine-tuning workflow for transformer-based large language models: selecting a base model, framing an instruction-following task, preparing supervised datasets, tokenising and packing examples, and choosing between full fine-tuning and parameter-efficient methods such as LoRA and QLoRA. They configure training with Hugging Face Transformers and PEFT, monitor experiments in Weights & Biases, diagnose overfitting, and evaluate model quality with task metrics, benchmark sets, error taxonomies and human review rubrics. The course also addresses data governance, licensing, privacy, bias, red-teaming and release controls.

Delivery combines instructor demonstrations with guided notebook labs, model-evaluation workshops and peer review of training decisions. Participants build and assess a fine-tuned model for a realistic enterprise use case, such as support-ticket routing, policy-grounded response drafting or structured information extraction. They leave with a documented fine-tuning experiment pack: dataset specification, training configuration, evaluation report, error-analysis log and production recommendation that can be adapted for a workplace pilot.

The course is designed for ML engineers, data scientists and AI technical leads who already understand Python and core machine-learning concepts and now need to develop, evaluate or govern custom LLM adaptations.

Course objectives

By the end of this course, participants will be able to:

• Design a supervised fine-tuning dataset specification with instruction, input, output, provenance and quality fields
• Select an appropriate base model using task fit, context length, licensing, hardware and deployment constraints
• Prepare and validate instruction datasets through deduplication, formatting checks, PII review and train-validation-test splitting
• Configure LoRA and QLoRA adapters with Hugging Face Transformers and PEFT
• Tune training hyperparameters using learning curves, loss monitoring, checkpoint comparison and experiment tracking
• Evaluate fine-tuned models with task-specific metrics, held-out test sets, human scoring rubrics and error taxonomies
• Conduct safety and regression testing for harmful outputs, data memorisation, prompt sensitivity and capability loss
• Produce a fine-tuning experiment report with deployment recommendations, risks, costs and acceptance criteria

Benefits of attending

Target competencies

Who should attend

• Machine Learning Engineers — who need to train, evaluate and package task-specific LLM adaptations
• Data Scientists — who need rigorous dataset, experiment and evaluation methods for generative AI use cases
• AI Engineers — who must move from prompt-only prototypes to measurable model customisation
• MLOps Engineers — who support reproducible training runs, model artefacts and release controls
• AI Technical Leads — who must choose between fine-tuning, retrieval-augmented generation and larger base models
• Data and AI Governance Specialists — who assess training-data provenance, privacy, safety and model-release evidence

Requirements and prerequisites

Participants should be comfortable writing and running Python notebooks, working with pandas data frames and reading basic PyTorch training code. Prior knowledge of supervised learning, train-validation-test splits, loss functions, gradient descent, overfitting and standard evaluation metrics is assumed. Familiarity with transformer concepts such as tokens, embeddings, attention and context windows is helpful; participants should also have used an LLM API or open-source model at least once. No prior experience of LoRA, QLoRA, distributed training, GPU administration or building a model from scratch is required. The course provides guided environments and starter code.

Training methodology

Each day alternates focused instructor-led technical sessions with guided notebook labs using Hugging Face model workflows. Participants inspect real instruction datasets, configure adapter-based training runs, compare checkpoints and interpret learning curves rather than only following prebuilt examples. Small groups review evaluation failures and decide whether the cause is data quality, task design, model capacity or safety risk. Case discussions examine fine-tuning decisions in enterprise settings. On the final day, each participant converts their work into a documented pilot plan with technical assumptions, evaluation gates and release recommendations.

Course outline

Day 1: Fine-tuning strategy and task design

• Fine-tuning versus prompting and retrieval-augmented generation
• Base-model selection by capability, licence and context window
• Instruction-following task decomposition and success criteria
• Supervised fine-tuning data schemas and message templates
• Tokenisation, sequence length and example packing
• Training-data provenance, consent and licensing checks
• GPU memory estimation and training-cost modelling

Workshop: Participants define a target LLM use case and produce a fine-tuning decision canvas with task boundaries, success measures, data sources and cost assumptions.

Day 2: Dataset engineering for supervised fine-tuning

• Instruction-response pair construction and formatting conventions
• Data cleaning with pandas validation rules
• Deduplication and near-duplicate detection
• PII identification, redaction and sensitive-data handling
• Train-validation-test split design for generative tasks
• Dataset versioning and data cards
• Quality sampling and human annotation guidelines

Workshop: Participants transform a raw enterprise-style text dataset into a validated instruction dataset and produce a data card with split rationale and quality checks.

Day 3: Adapter training and experiment control

• Hugging Face Transformers training architecture
• PEFT adapter configuration with LoRA
• QLoRA quantisation and memory-efficient training
• Hyperparameters for batch size, learning rate and epochs
• Gradient accumulation, checkpointing and mixed precision
• Weights & Biases experiment tracking and artefact logging
• Learning-curve interpretation and overfitting diagnosis

Workshop: Participants configure and run a LoRA or QLoRA training experiment, then produce a tracked comparison of two hyperparameter configurations.

Day 4: Evaluation, safety and failure analysis

• Held-out evaluation set construction
• Task metrics for extraction, classification and response generation
• LLM-as-judge limitations and calibration methods
• Human evaluation rubrics and inter-rater review
• Error taxonomies for hallucination, omission and formatting failures
• Memorisation, privacy leakage and prompt-sensitivity testing
• Regression testing against the base model

Workshop: Participants evaluate their fine-tuned model against a held-out test set and produce an error-analysis log with prioritised remediation actions.

Day 5: Production decisions and fine-tuning governance

• Model acceptance thresholds and release gates
• Adapter merging, model packaging and inference trade-offs
• Latency, throughput and cost benchmarking
• Model cards and fine-tuning experiment documentation
• Red-teaming scenarios and escalation criteria
• Monitoring signals for post-release quality drift
• Pilot design, rollback planning and stakeholder approval

Workshop: Participants complete a fine-tuning experiment pack and present a production-pilot recommendation covering evidence, risks, operating costs and release criteria.

Tools & standards covered

PyTorch, Hugging Face Transformers, PEFT, Weights & Biases

A typical training day

Live online deliveries follow the same structure in the East Africa Time zone, with shorter screen blocks and longer breaks.

What the fee includes

How you can take this course

Certification

Participants who complete the full five days receive the Skillset Development Certificate of Completion, stating the course title, course code, dates and delivery format — suitable for professional-development records and employer reimbursement.

Frequently asked questions