Glossary
Fine-Tuning
Meaning and details
Fine-tuning takes a model that already knows language and teaches it your specifics. You gather hundreds to thousands of input-output examples of the behavior you want, the format, the tone, the edge-case judgments, and...
Fine-Tuning meaning
Continuing a pretrained model's training on your own examples so its behavior specializes to your task or voice.
What is Fine-Tuning?
Fine-tuning takes a model that already knows language and teaches it your specifics. You gather hundreds to thousands of input-output examples of the behavior you want, the format, the tone, the edge-case judgments, and continue training on them. The result is a custom checkpoint that behaves consistently on your task, often a smaller and cheaper model than you would otherwise need.
Why it matters to an indie builder: it is the middle rung of the specialization ladder, and knowing where each rung starts saves real money. Rung one is prompting: cheap, instant, reversible. Rung two is retrieval, giving the model your documents at request time. Rung three is fine-tuning, warranted when prompting plus retrieval still cannot hold the behavior, class imbalance, strict output schemas, a distinctive voice, or when you need a small model to match a large one on a narrow task and cut the per-token bill.
A concrete example: a legal-drafting assistant whose JSON output must classify 140 clause types. Prompting a frontier model gets 93 percent accuracy on rare clause families, because examples of them do not fit in context. Fine-tuning a smaller open model on 4,000 labeled clauses reaches 97 percent while cutting inference cost per draft by 80 percent. The tuning job itself cost an afternoon of compute, less than a week of the frontier model's bill at that error rate.
Common mistakes: reaching for it first, before prompting and retrieval have been exhausted (start cheap; most behavior is teachable in context), tiny or dirty datasets (500 well-labeled examples beat 10,000 noisy ones, and evaluation sets must be held out before training or you are grading your own homework), catastrophic forgetting, where the tuned model degrades on general tasks (keep general regression examples in the mix), and no objective eval: define the metric and the test set first, or you cannot prove the tune helped. The fine-tuning trait and llm-ops blocks on IndieBoot cover the platforms that manage the process.