RAG, explained for people who run businesses

If you have been quoted for an AI project recently, you have probably seen the letters RAG. It stands for retrieval-augmented generation, which explains nothing useful. Here is what it actually means and why it decides whether your AI is worth having.

The problem it solves

A language model on its own knows a great deal about the world and nothing whatsoever about your business. Ask it your delivery timeline and it will produce a confident, fluent, entirely invented answer. That is not a bug you can prompt your way out of — the model has no source for the fact.

RAG fixes this by changing the order of operations. Before answering, the system searches your own material — catalogues, policies, manuals, past quotations — pulls out the passages that relate to the question, and instructs the model to answer using those. The model stops recalling and starts reading.

The one-sentence version

RAG means the AI looks up your documents before it answers, and answers from what it found. That is the whole idea. Everything else is engineering.

Why it matters commercially

• Answers can be checked. A RAG system can cite the document it used, so a person can verify it. An ungrounded system gives you fluency with no audit trail.
• Updating is editing, not retraining. Change the price list and the answers change. You are not paying to retrain a model.
• Access can be controlled. Staff can retrieve from internal documents; customers only from public ones. That separation is enforceable.

What makes it hard in practice

The retrieval is the difficult half, and it is where most implementations disappoint.

• Contradictory documents. If three files give three prices, retrieval will find all three and the answer becomes a coin toss. Deciding what is authoritative is unavoidable human work.
• Documents that were written for humans. A price list where meaning lives in the layout of a spreadsheet is hard to retrieve from usefully. Preparation matters.
• Questions that don't match the wording. A customer asks “how long for delivery to Salem?”; the document says “dispatch timelines by region”. Bridging that gap is the real engineering.
• Knowing when it found nothing. The system must be willing to say it does not know. Systems that always answer are the dangerous ones.

Do you need it?

If the answers your customers or staff need already exist in written material — and the problem is that nobody can find them fast enough — then yes, this is exactly the tool. If your answers live only in people's heads, the first project is writing them down, not buying AI. We would rather tell you that than sell you a system with nothing to retrieve from.

More on how we build these: RAG knowledge assistants, or check whether your data is ready in about two minutes.