What is RAG? Retrieval augmented generation without the jargon

We build assistants this way for paying clients. The AI does the repeated running of the work. People decide what it may say and check what it said.

The three words, one at a time

• Retrieval. A search runs across your own material and returns the passages most likely to contain the answer. This happens before the AI writes anything at all.
• Augmented. Those passages are placed in front of the model alongside the question, so it answers with your document open rather than from memory.
• Generation. The model writes the reply, in sentences, using what it was handed.

Read in that order, the name states the sequence: search first, hand over the results, then write. Most of what goes wrong in these systems goes wrong inside the first word, which is the one nobody in a sales conversation spends any time on.

What is RAG in gen AI?

In generative AI, RAG is a design pattern that gives a language model a reference library at the moment it answers. The model still writes the sentences, but a search step chooses the facts it writes from. This keeps answers tied to documents you can edit, and lets a reader check where each answer came from.

Picture two new members of staff asked the same customer question. One answers from whatever they half remember. The other walks to the filing cabinet, pulls the right policy, reads it and then answers. RAG is the second person. The walk to the cabinet is the retrieval, and the written reply is the generation.

Why it exists: the model does not know your business

A language model has read an enormous amount of text and none of it was your price list. It does not know your delivery areas, which product you discontinued in March, or what your warranty actually covers. Asked anyway, it produces a fluent answer shaped like the truth, because producing fluent answers is the thing it does.

There are two ways to fix that. Teach the model, by training it further on your material, or hand it the material at the moment the question arrives. The second is retrieval, and for facts it wins for a reason that has nothing to do with sophistication: facts change, and information kept outside the model can be edited by a person on a Tuesday afternoon.

That is the whole business case. Change the price list and the next answer uses the new price. No retraining, no waiting, and no wondering which of the things it learned last year have quietly stopped being true.

Is ChatGPT a RAG LLM?

No, not by itself. ChatGPT is a product built around a large language model (LLM), and a bare model answers from training alone. It behaves like a RAG system only when retrieval is wrapped around it, for instance when you attach your own files or let it search the web before replying. The model is the writer. RAG is the research step in front of it.

That distinction matters when you buy. A general chat tool has not read your documents unless somebody supplies them, and a business assistant needs that supply to be permanent, curated and kept current.

What happens between the question and the answer

A fixed sequence runs every time, and the whole of it takes moments. Knowing the order is what lets you ask a supplier a question they have to answer properly.

• A customer asks something in their own words, which may share no vocabulary with your documents
• The question becomes a search, one that matches on meaning rather than on exact wording
• The search returns a handful of passages, ranked by how relevant they appear to be
• Those passages, the question, and your instructions about tone and scope all go to the model
• The model writes an answer from the passages, and should be told to say so when they do not contain one
• The answer appears, ideally with its source beside it, so a reader can check it and a reviewer can spot a pattern

Two of those steps hold the money and the risk. The search step decides whether the right passage came back at all, and the writing step decides whether the model used it properly. They fail separately, they look identical to a customer, and they need completely different repairs, which is why they have to be judged apart.

What are the components of a RAG system?

A RAG system has six working parts: a document store, a chunking step, an embedding model with an index, a retriever, prompt assembly and a language model. Retraining the model is not one of them. The table shows what each part does and where it most often goes wrong.

Component	What it does	Where it usually goes wrong
Document store	Holds the source files the answers must come from	Old versions sit beside current ones
Chunking	Splits files into passages small enough to retrieve	Cuts a rule off from its exception
Embedding model and index	Turns passages into a searchable form	Misses exact product codes and names
Retriever	Finds the passages closest to the question	Returns near matches that are not answers
Prompt assembly	Places the passages and the question in front of the model	Passages crowd out the instructions
Language model	Writes the reply from the passages supplied	Fills a gap with a confident guess

Five of the six parts sit around the model rather than in it. That ratio is a fair picture of where the effort goes.

What you are actually paying for

Buyers assume they are paying for a model. Almost none of the effort goes there.

The model itself is largely a settled choice. What separates a system that works from one that embarrasses you is whether somebody did the content work, and the content work is the part clients most want to skip, because it is dull and it looks as though it ought to be free.

• Getting your material into one place, which usually means first finding it in several
• Deciding which version of each document is current, and removing the rest rather than leaving them beside it
• Rewriting passages so each stands alone, because the search returns pieces rather than whole documents
• Tagging content so a search can be filtered by market, product line, language or effective date
• Building a question set from real traffic, with the correct answer recorded for each one
• Deciding who maintains all of the above on the day the project ends

How do you improve RAG retrieval?

Most poor answers are retrieval faults rather than model faults. If the right passage never reaches the model, no model can answer correctly. Fix the sources first, then the splitting, then the search, and measure after each change.

• Clean the sources. Remove superseded files and duplicates.
• Chunk by meaning, keeping a rule with its exceptions, rather than cutting at a fixed length.
• Combine keyword search with meaning-based search, so exact names and codes are still found.
• Add a reranking step that reorders the retrieved passages before the model sees them.
• Instruct the model to say it does not know when the passages do not contain the answer.

Then test. Keep a fixed set of real questions with known correct answers, run it after every change, and keep only the changes that raise the score. That is how we run this work: evidence first, one change at a time, with the result recorded.

Two cases on our work page show the kind of live, customer-facing assistant work we run. An eye care business in the USA, in New York, logged 82 conversations through AI booking. A publishing business in the USA, a ghostwriting house serving 500+ authors, now runs an AI caller and AI CRM end to end. Neither is a measure of document-retrieval accuracy. They show that we operate assistants in daily use, with people deciding the limits.

The vocabulary you will hear in a sales call

If a supplier uses one of these and cannot explain it in a sentence you already understand, that is information about the supplier rather than about you. The AI glossary carries the wider list.

• Chunk. A piece of a document. Search works on pieces rather than whole files, which is why a passage that only makes sense with the paragraph above it retrieves badly.
• Embedding. A numeric representation of a piece of text. It is how a search matches on meaning instead of on matching words.
• Vector database. Where those numbers live so they can be searched quickly.
• Index. The prepared, searchable copy of your content. A document that is not in it cannot be found, however important the document is.
• Grounding. Answering from supplied material rather than from memory, which is the entire point of the exercise.
• Hallucination. A confident answer with nothing behind it. Retrieval reduces these and does not remove them.
• Context window. How much text the model can be handed at once, which is why the search returns a handful of passages rather than everything you own.

What are the different types of RAG?

There is no fixed, agreed list of types. People describe variations by how the search works: plain meaning-based search, keyword search blended with it, search with a reranking step, or an agent that decides what to look up. Those are choices inside one design.

Retrieval augmented generation is the full name worth searching for. A supplier who offers you a catalogue of exotic types is usually selling complexity before they have looked at your documents.

Questions retrieval cannot answer, however good the setup

Anything whose answer changes during the day. Where is my order, is that slot free, how much is left on the account. Searching documents finds text somebody wrote down, while an order status lives in a system and moves. That needs a direct lookup, which is a different build with different failure modes.

Anything that was never written down. The system retrieves what exists. If the answer to your most common customer question lives only in one person's head, this is a documentation project with a chatbot attached, and the honest sequence is to write the answers first. That work pays for itself whether or not anything gets built on top of it.

Anything your documents disagree about. Retrieval finds both versions, and the model presents one of them as fact with no visible sign of uncertainty. Deciding which is correct is a judgement the business has to make, and no configuration makes it for you.

Should you use RAG or fine-tuning?

Use RAG when the knowledge changes or must be traceable to a document. Use fine-tuning when you need to change the model's style or behaviour rather than its facts. Many businesses need retrieval and nothing more. The full comparison, with the costs of each, is on RAG vs fine-tuning, and the other explainers in this series sit on the Wobble blog.

What it costs, how long the document work takes, and who answers the rest

For reference, our 2026 price list puts a business website of up to 8 pages, with AI chat included, at $2,000–4,500. That is a website price, not a RAG price. A document assistant has no listed price, because the quote is moved by the documents rather than by the model: how many there are, what state they are in, how many contradict each other, how often they change, and who keeps them current afterwards. A quote built around model choice is a quote about the cheapest part of the work.

How we bill depends on the build. Some projects are a one-time build, some are a monthly service, and some are both. You own what we build. Monthly services stop in any month, with no lock-in. If a new idea comes up mid-project, it becomes the next phase: we quote it first and never add it quietly.

The audit comes first and most of it is spent reading your documents rather than writing anything, because a retrieval system built over a set containing several versions of the refund policy will answer confidently from whichever one it found. It produces three things: a list of which documents are current, a list of the ones that contradict each other, and a list of the common questions that have no written answer. The first assistant then covers one subject only, such as returns, using just the cleaned documents for it, and hands everything outside that scope to a person. Widening it happens afterwards, one subject at a time, since every new subject is a fresh chance to reintroduce a contradiction.

The earlier section on what retrieval cannot answer has a design answer, and it is a stop rather than a better search. Anything whose answer changes during the day goes to a lookup or to a person. Anything the retrieval step finds nothing for is handed to a person rather than answered from the model's general knowledge, which is the failure that looks exactly like a correct answer. Any price, refund or complaint waits for human approval. We decide those limits with you before the build, and our RAG chatbots page shows how they are set.

The part you can do in-house is the preparation list under what you are actually paying for, and your own team can start it this quarter.

Common questions

What does RAG stand for?

Retrieval augmented generation. A search retrieves relevant passages from your own material, those passages augment the question by being placed in front of the model, and the model generates the answer from them. The name is the sequence, read left to right.

What is RAG in plain business terms?

RAG is the difference between an assistant answering from memory and one answering with your policy document open in front of it. Your material stays outside the model where a person can edit it, and the system looks things up at the moment the question arrives rather than relying on what it once absorbed.

Is RAG the same as training a model on our data?

No, and the difference matters most when facts change. Training bakes information into the model, so correcting it means another training run and you cannot tell which old facts are still in there. Retrieval keeps the information in documents you control, so editing the document changes the next answer immediately.

Does RAG stop the AI making things up?

It reduces it and does not eliminate it. The more damaging failure is different anyway: the search hands over a passage that looks right and is not, such as a superseded price list or a policy written for another region, and the model uses it because using what it was given is the whole design. Showing the source beside each answer is what makes that visible.

What do we need to prepare before building a RAG system?

Your content, first: one place for your material, one current version of each document, and a set of real questions with the right answers recorded. The full list is under what you are actually paying for, and it is most of the project.

Can RAG answer questions about live data like order status?

Not through retrieval. Searching documents finds what somebody wrote, and an order status changes hourly inside a system. Answering it requires a direct lookup into that system. Most workable deployments combine the two, using retrieval for policy and product questions and lookups for anything current.

Where does RAG not fit?

It will not rescue messy or contradictory documents, and will repeat them faster. It is a poor fit where the answer needs calculation across many records, which belongs in a database query. It also adds a search system to maintain, and it needs a named person to update the files whenever a price, returns rule or procedure changes. If your sources are thin, write them before you build anything.