Skip to main content
AI & Agents

Fine-Tuning

Fine-tuning adjusts a model's weights by training it on examples of desired behaviour, changing how it responds rather than what it knows. It is used to enforce a consistent output format, tone or task pattern — not to teach facts, which cannot then be updated without retraining.

The decision between fine-tuning and retrieval reduces to one question: is the gap knowledge or behaviour? If the model lacks information, retrieval supplies it and can be updated by editing a document. If the model has the information but responds in the wrong shape, fine-tuning is the correct tool. Using fine-tuning to memorise facts produces a system where correcting a price means retraining a model.

For behaviour and format tasks, far less data is needed than teams expect — a few hundred high-quality, internally consistent examples routinely outperform thousands of mediocre ones, because contradictory examples teach the model to be inconsistent. In practice the dataset curation is the project; the training run is the easy part.

FAQ

Fine-Tuning
FAQ.

Common questions about fine-tuning.

Ask Us Anything

Less than most teams assume for behaviour and format tasks — often a few hundred high-quality examples beat thousands of mediocre ones. Consistency matters more than volume, because contradictory examples teach inconsistency. Expect the curation of that dataset to be the bulk of the work, and expect prompting alone to solve the problem more often than not.

Always. Prompting costs nothing, changes in seconds, and resolves more problems than either fine-tuning or retrieval. The sensible escalation is prompting, then retrieval, then fine-tuning, stopping as soon as quality is acceptable. Teams that skip to fine-tuning frequently end up with an expensive system a better-structured prompt would have matched.

Related terms

RAG (Retrieval-Augmented Generation)

RAG is a technique where an AI system retrieves relevant passages from your own documents at query time and generates an answer grounded in them, with citations.

Embedding

An embedding is a list of numbers representing the meaning of a piece of content, produced by a model so that semantically similar items sit close together in the resulting space.

Browse the full glossary →