08 - Fine-Tuning & PEFT Lab

What You Will Learn

• The Hugging Face ecosystem: AutoModelForCausalLM/AutoTokenizer, config objects, chat templates, and the datasets library
• How to load, format, and mask instruction datasets for supervised fine-tuning
• Hands-on LoRA and QLoRA configuration - rank, alpha, dropout, target modules, 4-bit bitsandbytes quantization
• Merging LoRA adapters into a base model vs serving them separately at inference time
• Reading a real loss curve to catch overfitting, too-high learning rate, or too-small datasets
• Building a held-out eval set and benchmarking a fine-tuned model against its base - quality, latency, VRAM, and cost per 1k requests
• When fine-tuning is the wrong tool and prompting or RAG should win instead
• Interview-ready answers on the full hands-on fine-tuning workflow

Chapter Map

#	File	Topic	Difficulty
1	HuggingFace Ecosystem	AutoModelForCausalLM/AutoTokenizer, tokenization, chat templates, datasets, the Hub	Beginner
2	LoRA & QLoRA Hands-On	LoRA config (rank/alpha/dropout/target modules), QLoRA (NF4, double quant), merging vs serving adapters	Intermediate
3	Instruction Data & Training Runs	Formatting instruction data, prompt/completion masking, reading loss curves, a hand-written PEFT training loop	Intermediate
4	Benchmarking Base vs Tuned	Held-out eval sets, quality delta, measured latency/tokens-per-sec, VRAM and cost-per-1k-requests	Advanced
5	Q&A Review Bank	15 Q&A pairs across all topics in this module	All levels

Recommended Learning Paths

Path A: First Fine-Tune, Start to Finish

1. HuggingFace Ecosystem - learn the tools you'll be using
2. LoRA & QLoRA Hands-On - understand what you're configuring and why
3. Instruction Data & Training Runs - format your data and run the training loop
4. QLoRA Fine-Tune Code Lab - do it end-to-end on a real small model
5. Benchmarking Base vs Tuned - prove the fine-tune actually helped

Path B: Interview Preparation (Accelerated)

1. LoRA & QLoRA Hands-On - config parameters and merge-vs-serve questions are very common
2. Instruction Data & Training Runs - loss masking and loss-curve reading come up constantly
3. Benchmarking Base vs Tuned - "how do you know it worked" is a favorite follow-up question
4. Q&A Review Bank - drill all 15 questions

Path C: Production Engineering (Advanced)

1. LoRA & QLoRA Hands-On - merging adapters vs adapter-swapping at serve time
2. Benchmarking Base vs Tuned - cost-per-1k-requests and when fine-tuning is the wrong call
3. QLoRA Fine-Tune Code Lab - train_qlora.py + benchmark.py as a reusable template

Resources

• Q&A Review Bank - 15 Q&A pairs in this module
• Cross-topic Interview Questions
• QLoRA Fine-Tune Code Lab - hands-on PEFT + QLoRA fine-tune with a base-vs-tuned benchmark

Key Cross-References

• LoRA math (ΔW = BA, rank/alpha derivation) and the DPO/RLHF taxonomy → 01-LLM-Models: Fine-Tuning - this module builds hands-on tooling on top of those foundations, it does not re-derive the math
• VRAM estimation and quantization formats (NF4, INT8, AWQ) → 01-LLM-Models: GPU & Hardware
• The raw-PyTorch training loop this module's hand-written PEFT loop extends → PyTorch Fundamentals: Training Loop From Scratch
• Checkpointing and mixed precision used during training runs → PyTorch Fundamentals: Checkpointing & Mixed Precision

Next Topic

09 - Serving & Inference (coming soon)