Fine-tuning a Pretrained Model

Don't start from zero — stand on a giant's shoulders

Pretraining a model (fundamentals Module 15) costs millions and months. But that model already learned grammar, facts, and reasoning. Fine-tuning takes that pretrained model and continues training it — briefly — on your specific data, so it picks up your style, domain, or task.

Compare the cost: scratch vs fine-tune

Slide the target quality and watch the two approaches' cost/time. Fine-tuning reaches good results for a tiny fraction of the effort — because it starts from something that already works.

The fine-tuning workflow (it's the training loop again)

Good news: fine-tuning is the Module 3 training loop — just started from pretrained weights instead of random ones, with a small learning rate (you don't want to erase what it knows).

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("gpt2")   # pretrained! not random
tok   = AutoTokenizer.from_pretrained("gpt2")

optimizer = torch.optim.AdamW(model.parameters(), lr=5e-5)  # small lr!

for epoch in range(3):              # just a few passes
    for batch in your_data_loader:   # YOUR data (Module 4)
        loss = model(**batch).loss    # forward + loss
        loss.backward()               # autograd (Module 2)
        optimizer.step()              # update
        optimizer.zero_grad()

See fine-tuning change the output

Same prompt, same model — but after fine-tuning on a specific style. Pick a target style and watch the "before" (generic GPT-2) become the "after" (your specialized model).

How to format your fine-tuning data

The #1 practical detail: your data must be in the right shape for the task. Different goals need different formats. Click each to see the structure.

Freeze layers — train only what you need

A common trick: freeze most of the pretrained layers (keep them fixed) and only train the top few. This is faster, uses less memory, and reduces overfitting on small data. Click layers to freeze/unfreeze them.

# freeze everything, then unfreeze just the last block
for p in model.parameters():
    p.requires_grad = False          # ❄️ frozen
for p in model.transformer.h[-1].parameters():
    p.requires_grad = True           # 🔥 train only this

Watch a fine-tune run (starts low!)

Because the model already knows language, fine-tuning loss starts low and drops fast — very different from the high starting loss of training from scratch. Run it and see.

Checkpoints — save your progress

Training can crash, or you might overfit (Module 3). So you save checkpoints — snapshots of the model's weights — periodically. Then you can resume, or roll back to the best one. Click a checkpoint to "load" it and see its validation score.

# save a checkpoint
model.save_pretrained("./my-model-epoch3")     # weights + config
tok.save_pretrained("./my-model-epoch3")

# load it back later (or the best one)
model = AutoModelForCausalLM.from_pretrained("./my-model-epoch3")

Quick challenge

What you learned

Next up → Module 8: LoRA & PEFT — even fine-tuning all of a big model is expensive. LoRA fine-tunes just a tiny set of "adapter" weights (often <1% of the model), and quantization shrinks models to fit one GPU. This is how fine-tuning is actually done today.