Fine-tuning
Fine-tuning is the practice of taking an already-trained model and training it further on your own examples, so it behaves more like you want. It adjusts how the model responds rather than what it knows, and that distinction decides when it is the right tool.
Fine-tuning sits alongside prompting and retrieval as one of three ways to adapt a general model to a specific job. Prompting instructs at the moment of the request, retrieval supplies relevant content, and fine-tuning changes the model itself.
How does fine-tuning an LLM work?
You assemble examples of the behaviour you want, typically pairs of an input and the response you would have wanted, then continue training the model on that set. The weights shift towards your examples while the model's general capability stays largely intact. What fine-tuning is genuinely good at:
Consistent format: output that has to match a structure every time
Tone and style: a house voice that a prompt keeps drifting away from
Narrow classification: a specialised labelling task with plenty of examples
Shorter prompts: behaviour baked in rather than restated on every request
What it does not fix is knowledge that changes. A model fine-tuned on last quarter's fee schedule will state last quarter's fees confidently, and updating that means another training run.
When is prompting or RAG the better answer?
Most of the time, for most teams. The decision turns on whether you are changing what the model knows or how it behaves.
Facts that change: use retrieval. Content updates take minutes, and a training run does not
Account or customer data: use tools. No amount of fine-tuning gives a model access to a live record
Instructions and rules: start with prompting, because it is immediate and reversible
Style and format prompting cannot hold: fine-tuning earns its place here
There is a control dimension too. A prompt can be read, reviewed, and changed by a person, while fine-tuned behaviour is encoded in weights nobody can inspect. In a regulated setting that opacity carries a cost.
Frequently asked questions
Does fine-tuning stop hallucination?
Not reliably. Fine-tuning on examples of confident answers can teach confidence rather than accuracy, which sometimes leaves a model more assured about things it still doesn't know. Grounding is the lever that actually addresses hallucination.
How much data does fine-tuning need?
Less than teams expect for style and format, and far more than they expect for anything resembling knowledge. Hundreds of good examples can shift tone reliably. Teaching genuine domain knowledge takes a great deal more, and retrieval usually gets there faster.
Is a fine-tuned model harder to audit?
Yes, in one specific way. Prompted behaviour can be read and changed by a person, while fine-tuned behaviour lives in weights nobody inspects directly. You can still test the outputs, and you cannot point a reviewer at the rule that produced them. Gradient Labs' guide to evaluating AI agents in financial services sets out how to test an agent's behaviour before production and keep measuring it afterwards, which is the evidence a reviewer wants when the rule itself cannot be read.