RAG or fine tuning, and why facts and style are two different problems

The difference, and which one you probably want

Retrieval augmented generation grounds a model in documents you supply at the moment of the question. Fine tuning changes the model's own weights by training it on examples. For business facts, prices, policies and anything that changes, use retrieval, because it is cheaper, updateable and auditable. Fine tuning is for style, format and consistent labelling, which are patterns rather than facts.

The practical consequence is about correction. When retrieval gives a wrong answer, you can find the exact passage that produced it and edit the passage. When a fine tuned model gives a wrong answer, there is nothing to correct except the training run, and the model has no idea which of the things it absorbed have since stopped being true.

Most business questions are fact questions wearing a style costume, which is why the answer is retrieval far more often than the question implies.

What retrieval is doing, in plain terms

A language model does not know your prices, your policies or your product range. Retrieval fixes that without retraining anything. When a question arrives, a search step finds the passages in your own material most likely to contain the answer, and those passages are placed in front of the model alongside the question. The model then answers from what it was handed.

Three separate things are happening, and they fail separately. Your content is prepared and indexed. A question becomes a search and passages come back. The model writes an answer from those passages. When the answer is wrong, the cause is usually the middle step, and the usual response is to rewrite the instructions, which changes nothing at all.

That is why most of the work in a retrieval project sits in the documents rather than in the model choice. A knowledge base written for people who already know the context will not retrieve well, because each passage is judged on its own.

• Facts change and retrieval follows them: edit the source and the next answer is current.
• Every answer has a traceable origin, which makes a wrong answer fixable rather than mysterious.
• Access rules can be applied before the search, so different readers see different material.
• Adding new material is an upload rather than a training cycle.
• The failure mode is visible: you can read what the system retrieved and see why it went wrong.

When fine tuning genuinely earns its cost

Fine tuning is not useless, it is misapplied. It teaches a model to behave a certain way rather than to know a certain thing, and there are jobs where behaviour is the whole problem.

The clearest case is output that must take the same shape every time, at volume, with no room for variation. Classifying incoming messages into a fixed set of categories, producing records in a strict structure, or writing in a house style that a long instruction never quite reproduces are all pattern problems. When you find yourself writing an ever-growing instruction full of examples, and it still drifts, training on those examples is the correct move.

There is a cost dimension too. A fine tuned smaller model can do a narrow, high-volume job with a shorter instruction attached, which matters when the same task runs constantly. That is an efficiency argument rather than a quality one, and it only applies once the volume is real.

• Consistent output structure at high volume, where variation causes downstream breakage.
• A house style or tone that instructions describe badly and examples describe well.
• Classification into a fixed set of labels that does not change often.
• A narrow, repetitive task where a smaller trained model is cheaper to run than a large one with a long instruction.
• Behaviour that must persist even when the instruction is short, because the instruction budget is needed for other things.

Comparing them on what actually differs

The rows that decide real projects are the middle three. Capability is rarely the deciding factor.

Criterion	Retrieval (RAG)	Fine tuning
What it is good for	Facts, policies, prices, anything that changes	Style, format, labels, consistent behaviour
Updating it	Edit the document, effective on the next question	Prepare data and train again
Fixing a wrong answer	Find the passage that caused it and correct it	Little to correct except the training set
Showing your working	The source can be shown with the answer	No source exists to show
Access control	Applied before the search, per reader	Baked in for everyone who uses the model
Effort concentrated in	Writing and structuring the documents	Assembling and cleaning the training examples

When neither is what you need

Live values belong to neither. Order status, current stock, a delivery slot and an account balance are lookups into a system rather than searches over documents, and dressing them up as either technique produces an assistant that is confidently wrong about the one thing the customer asked. Connect to the system that holds the value.

Arithmetic and reporting belong to neither as well. If the question is how many of something happened last month, the answer comes from a query, and a language model asked to count from retrieved passages will produce a plausible number rather than the right one.

There is also the case where the business has not decided the answer. Plenty of assistant projects stall at the point where three people give three versions of the refund policy. Neither retrieval nor training rescues that. Writing down what is true is the project, and it is worth doing even if you never build the assistant.

How to choose, and what to do first

Ask what kind of wrong the system would be. If the failure you fear is stating something out of date or inventing a policy, that is a facts problem and retrieval is the answer.

If the failure you fear is an answer that is correct but formatted inconsistently or written in the wrong register, that is a pattern problem and fine tuning is worth costing. Many production systems use retrieval for the facts and a fine tuned model for the shape of the reply, and treating them as rivals is the error.

Whatever you choose, start with the documents. Write the question as the heading and answer it directly underneath. Name things explicitly instead of writing this and the above. Keep one canonical answer per question rather than four near copies in four places. That work improves retrieval, improves any training set you might build later, and improves the humans who read the same documents.

What each one costs to keep, and the third answer the comparison hides

No price is published here, and the two options move on entirely different things. Retrieval's number is decided by the document set: how many, how contradictory, how often they change and who keeps them current. Fine tuning's is decided by the examples: how many you have, how consistent they are, and how often the behaviour has to be retrained when it drifts. The audit takes the first week and a first retrieval assistant over a narrow document set is live inside a fortnight. Fine tuning is not a fortnight, and a supplier saying otherwise has not counted your examples.

Both need the same stop. Where retrieval finds nothing relevant, the correct behaviour is to hand it to a person rather than let the model answer from general knowledge, because that answer is indistinguishable from a good one until somebody checks it against the policy. Anything about a price, a refund or a complaint waits for human approval. A fine tuned model is worse here rather than better, because the confident tone is precisely the thing that was trained in.

Ownership differs sharply between them and it is worth knowing before choosing. A retrieval setup is your documents, your index and your prompts, in accounts under your own logins, so you own the system and can change an answer by editing a file. A fine tuned model is a set of weights held wherever it was trained, on a provider's terms, and changing an answer means retraining. The reversible option is worth a great deal in a business whose policies move.

The third answer the comparison hides is that most of the work is neither. Live values are lookups, and the cheapest project in this entire subject is your own team writing one canonical version of each policy in-house, which improves the human answers immediately and costs nothing. Moiz Khan owns automation architecture at Wobble and decides which of the three a question actually is. The published work behind that is systems in operation rather than model experiments: Quillon's delivery line as a single audited automation of 34 AI nodes, and THE MAGBOOK, a ghostwriting house serving more than 500 authors, moved onto an AI caller, AI CRM and AI creative generation end to end. Wobble works from Karachi, bills month to month, across 25 engagements in six countries.

Common questions

What is the difference between RAG and fine tuning?

Retrieval augmented generation supplies the model with passages from your own documents at the moment of the question, so the answer is grounded in material you control. Fine tuning changes the model itself by training it on examples. Retrieval is for knowledge that changes; fine tuning is for behaviour that should stay the same.

Which is cheaper for a business assistant?

Retrieval, in almost every case, because updating it means editing a document rather than preparing data and training again. Fine tuning also has a hidden ongoing cost: every meaningful change to what the model should do means another training cycle and another evaluation.

Can I use both together?

Yes, and mature systems often do. Retrieval supplies the facts and a fine tuned model supplies the shape, tone and consistency of the reply. Treating them as alternatives is the common mistake, because they solve problems that do not overlap.

Why does my retrieval chatbot give confident wrong answers?

Usually because the search returned a plausible but incorrect document: last year's pricing, a policy for another region, or a draft nobody approved. The model uses what it was given, so the defences belong around the retrieval step. Remove superseded documents from the index, filter on metadata before searching, and show the source with each answer.

Does fine tuning stop a model inventing things?

No, and it can make it worse. A fine tuned model produces the style of your training examples with more confidence, including when it has nothing factual to say. Grounding answers in retrieved passages and allowing the system to say that nothing relevant was found does more for reliability than any amount of training.

How much material do I need before retrieval is worth it?

Less than most people assume, but it has to be written for retrieval. A few dozen well-structured pages where each passage stands alone will outperform hundreds of pages of internal documents full of unexplained references. Fix the writing before adding volume.