What RAG means, explained without the jargon

The three words, one at a time

The acronym hides a great deal. Taken apart, it describes the entire mechanism.

Read in that order, the name states the sequence: search first, hand over the results, then write. Most of what goes wrong in these systems goes wrong inside the first word, which is the one nobody in a sales conversation spends any time on.

• Retrieval. A search runs across your own material and returns the passages most likely to contain the answer. This happens before the AI writes anything at all.
• Augmented. Those passages are placed in front of the model alongside the question, so it answers with your document open rather than from memory.
• Generation. The model writes the reply, in sentences, using what it was handed.

Why it exists: the model does not know your business

A language model has read an enormous amount of text and none of it was your price list. It does not know your delivery areas, which product you discontinued in March, or what your warranty actually covers. Asked anyway, it produces a fluent answer shaped like the truth, because producing fluent answers is the thing it does.

There are two ways to fix that. Teach the model, by training it further on your material, or hand it the material at the moment the question arrives. The second is retrieval, and for facts it wins for a reason that has nothing to do with sophistication: facts change, and information kept outside the model can be edited by a person on a Tuesday afternoon.

That is the whole business case. Change the price list and the next answer uses the new price. No retraining, no waiting, and no wondering which of the things it learned last year have quietly stopped being true.

What happens between the question and the answer

Six things happen, in order, and the whole sequence takes about a second. Knowing the order is what lets you ask a supplier a question they have to answer properly.

Two of those steps hold the money and the risk. Step three decides whether the right passage came back at all, and step five decides whether the model used it properly. They fail separately, they look identical to a customer, and they need completely different repairs, which is why they have to be judged apart.

• A customer asks something in their own words, which may share no vocabulary with your documents
• The question becomes a search, one that matches on meaning rather than on exact wording
• The search returns a handful of passages, ranked by how relevant they appear to be
• Those passages, the question, and your instructions about tone and scope all go to the model
• The model writes an answer from the passages, and should be told to say so when they do not contain one
• The answer appears, ideally with its source beside it, so a reader can check it and a reviewer can spot a pattern

What you are actually paying for

Buyers assume they are paying for a model. Almost none of the effort goes there.

The model itself is largely a settled choice. What separates a system that works from one that embarrasses you is whether somebody did the content work, and the content work is the part clients most want to skip, because it is dull and it looks as though it ought to be free.

• Getting your material into one place, which usually means first finding it in four
• Deciding which version of each document is current, and removing the rest rather than leaving them beside it
• Rewriting passages so each stands alone, because the search returns pieces rather than whole documents
• Tagging content so a search can be filtered by market, product line, language or effective date
• Building a question set from real traffic, with the correct answer recorded for each one
• Deciding who maintains all of the above on the day the project ends

The vocabulary you will hear in a sales call

None of these are difficult ideas. They are difficult words, delivered quickly, usually to somebody who feels they should already know them.

If a supplier uses one of these and cannot explain it in a sentence you already understand, that is information about the supplier rather than about you.

• Chunk. A piece of a document. Search works on pieces rather than whole files, which is why a passage that only makes sense with the paragraph above it retrieves badly.
• Embedding. A numeric representation of a piece of text. It is how a search matches on meaning instead of on matching words.
• Vector database. Where those numbers live so they can be searched quickly.
• Index. The prepared, searchable copy of your content. A document that is not in it cannot be found, however important the document is.
• Grounding. Answering from supplied material rather than from memory, which is the entire point of the exercise.
• Hallucination. A confident answer with nothing behind it. Retrieval reduces these and does not remove them.
• Context window. How much text the model can be handed at once, which is why the search returns a handful of passages rather than everything you own.

Questions retrieval cannot answer, however good the setup

Anything whose answer changes during the day. Where is my order, is that slot free, how much is left on the account. Searching documents finds text somebody wrote down, while an order status lives in a system and moves. That needs a direct lookup, which is a different build with different failure modes.

Anything that was never written down. The system retrieves what exists. If the answer to your most common customer question lives only in one person's head, this is a documentation project with a chatbot attached, and the honest sequence is to write the answers first. That work pays for itself whether or not anything gets built on top of it.

Anything your documents disagree about. Retrieval finds both versions, and the model presents one of them as fact with no visible sign of uncertainty. Deciding which is correct is a judgement the business has to make, and no configuration makes it for you.

What it costs, how long the document work takes, and who answers the rest

No price appears on this site, and on retrieval the number is moved by the documents rather than by the model. How many there are, what state they are in, how many of them contradict each other, how often they change, and who keeps them current afterwards. Almost none of the effort goes into the model, which is why a quote built around model choice is a quote about the cheapest part of the work.

The audit takes the first week and most of it is spent reading your documents rather than writing anything, because a retrieval system built over a set containing three versions of the refund policy will answer confidently from whichever one it found. A first assistant over a clean, narrow document set is live inside a fortnight. Widening it happens afterwards, one subject at a time, since every new subject is a fresh chance to reintroduce a contradiction.

The last section names what retrieval cannot answer, and the design answer to that is a stop rather than a better search. Anything whose answer changes during the day goes to a lookup or to a person. Anything the retrieval step finds nothing for hands it to a person rather than being answered from the model's general knowledge, which is the failure that looks exactly like a correct answer. Any price, refund or complaint waits for human approval.

The half of this you should do in-house is the half that pays. One canonical answer per question, superseded versions removed rather than left sitting beside the current one, and a named owner for each subject. Your own team can do that this quarter and it improves the answers your people give whether or not anything is built on top of them. Moiz Khan owns automation architecture at Wobble and decides what an assistant may answer without a person. The work published behind that includes Center for Sight in New York, booking largely hands-free, and THE MAGBOOK, a ghostwriting house serving more than 500 authors now running an AI caller and AI CRM end to end.

Common questions

What does RAG stand for?

Retrieval augmented generation. A search retrieves relevant passages from your own material, those passages augment the question by being placed in front of the model, and the model generates the answer from them. The name is the sequence, read left to right.

What is RAG in plain business terms?

RAG is the difference between an assistant answering from memory and one answering with your policy document open in front of it. Your material stays outside the model where a person can edit it, and the system looks things up at the moment the question arrives rather than relying on what it once absorbed.

Is RAG the same as training a model on our data?

No, and the difference matters most when facts change. Training bakes information into the model, so correcting it means another training run and you cannot tell which old facts are still in there. Retrieval keeps the information in documents you control, so editing the document changes the next answer immediately.

Does RAG stop the AI making things up?

It reduces it and does not eliminate it. The more damaging failure is different anyway: the search hands over a passage that looks right and is not, such as a superseded price list or a policy written for another region, and the model uses it because using what it was given is the whole design. Showing the source beside each answer is what makes that visible.

What do we need to prepare before building a RAG system?

One place holding your material, a decision about which version of each document is current, passages written so each one makes sense on its own, tags for market or product line or date, and a list of real questions with the right answers recorded. That preparation is most of the project, and it is the part suppliers rarely price loudly.

Can RAG answer questions about live data like order status?

Not through retrieval. Searching documents finds what somebody wrote, and an order status changes hourly inside a system. Answering it requires a direct lookup into that system. Most workable deployments combine the two, using retrieval for policy and product questions and lookups for anything current.