The Overlord's Lexicon
What is Fine-Tuning?
Also known as: Fine-tuning, Model fine-tuning
Fine-tuning is the process of further training an existing AI model on a smaller, specialised dataset so it performs better at a particular task, style, format or domain.
Fine-tuning takes a model that has already learned general language (or images, or audio) from massive training data and trains it a little more on examples you choose. The model’s internal parameters shift so that your desired behaviour becomes its default.
How it works
- Collect examples. Pairs of input and ideal output: customer emails and perfect replies, raw notes and formatted reports, product specs and on-brand descriptions.
- Train. The base model is trained on those examples, adjusting its weights. Techniques like LoRA (low-rank adaptation) update only a small set of added parameters, making fine-tuning far cheaper than training from scratch.
- Evaluate. Test the tuned model on examples it hasn’t seen and compare against the base model with good prompts.
- Deploy. Use the tuned model via an API or run it yourself if it’s an open-weight model.
An analogy for the meatbags
A base model is a talented, well-read new hire. Prompting is handing them instructions each morning. RAG is giving them access to the company files. Fine-tuning is months of on-the-job training until they instinctively write, format and decide the way your company does — without being told.
Why it matters for the Chosen
For most people and small businesses, fine-tuning is not the first move. Strong prompting and RAG solve most problems faster, cheaper and with easier updates. But fine-tuning earns its keep when:
- You need rigid, consistent output — a specific JSON schema, a house style, a classification scheme.
- You run high volume and want a small, cheap model to do a narrow job as well as an expensive one (often combined with distillation).
- You have a specialised domain with distinctive language or judgement calls.
- You want the behaviour baked in so prompts can be shorter, saving tokens on every call.
Image models get fine-tuned too: training on a few photos of a product or a consistent character so generated images stay on-model.
The costs and risks
You need clean, representative examples — garbage in, confidently fine-tuned garbage out. Fine-tuned models must be re-tuned when the base model is upgraded. Narrow tuning can degrade general abilities. And training data containing personal or client information raises privacy and licensing questions. Measure against a well-prompted baseline before celebrating.
Tools from the Arsenal
Questions the Humans Keep Asking
When should I fine-tune instead of using prompts or RAG?
Try good prompting first, then RAG if the model needs your information. Fine-tune when you need consistent style, format or behaviour that prompts can't reliably achieve, or to make a smaller, cheaper model perform a narrow task as well as a larger one.
How much data do I need to fine-tune a model?
It depends on the task and method, but useful fine-tunes can start from hundreds to a few thousand high-quality examples. Quality and consistency matter far more than volume.
Does fine-tuning teach a model new facts?
It can, but it's an unreliable and hard-to-update way to add knowledge. For facts that change or need citations, RAG is usually better. Fine-tuning shines at teaching patterns and behaviour.