The insight is that the weight change a fine-tune learns is approximately low-rank: it can be expressed as the product of two small matrices rather than a full-size update. With a rank of 8 to 64, the adapters add a fraction of a percent of the base model's parameters. At serving time the adapter can be merged into the base weights, adding zero inference latency, or served alongside them.
Two consequences matter operationally. QLoRA quantises the frozen base to 4-bit during training, which put 70-billion-parameter-class models within reach of a single high-memory GPU. And multi-LoRA serving — supported by vLLM among others — lets one base model host many tenants' adapters at once, so per-customer fine-tunes become a routing decision rather than a fleet of deployments.
LoRA changes behaviour, not knowledge, which defines when to use it. Format, tone, structure and domain style respond well to a few hundred well-chosen examples; data quality dominates every other variable. Teaching the model new facts with LoRA is the expensive way to do what retrieval does better, and pushing for a genuinely new capability is what full fine-tuning is for. The right default is retrieval for knowledge, LoRA for behaviour, full training as the rare exception.
The practical recipe is consistent across projects. Curate a few hundred examples that demonstrate the target behaviour — real inputs paired with ideal outputs, cleaned aggressively — then train at a modest rank and evaluate against both a held-out set and the base model. The two signature failures are overfitting, where the model parrots training phrasing, and capability bleed, where the adaptation quietly degrades general performance. Both are caught by the same evaluation suite that gates prompt changes, which is why a fine-tune without evals is an assertion, not an improvement.
Adapters are software artefacts and deserve the same discipline: versioned, stored, reviewed and rolled back like code. Because an adapter is small and training is cheap, the iteration cycle is short — a dataset fix and a retrain is an afternoon rather than a procurement decision — and that changes how experiments should be scoped. The bottleneck is almost never the training run; it is the quality of the examples and the rigour of the evaluation that follows.
Codazz builds this in production — LLM Integration.