Models
When is fine-tuning an LLM actually worth it?
Prompting and retrieval go a long way. Here is how to tell when they don’t.
Fine-tuning is often the first thing teams ask about, and usually not the first thing they should do.
Try these first
- Better prompts and examples: clear instructions plus a few good samples fix many quality problems.
- Retrieval: if the model lacks your facts, give it your documents instead of teaching it.
- A stronger model: sometimes the gap is simply capability.
Fine-tuning helps when
- You need a consistent tone, format or structured output across thousands of requests.
- Your domain uses specialised terminology or classification schemes.
- You want a smaller, cheaper or faster model to match a larger one on a narrow task.
- You have hundreds to thousands of high-quality examples of the right output.
Measure it
Build an evaluation set from real cases first. Compare the base model, the prompted model and the tuned model side by side, and ship the tuned one only if it clearly wins on your metrics and your costs.
We’ll say plainly when fine-tuning isn’t needed. Often a well-designed prompt and retrieval layer is enough.