The Overlord's Lexicon
What is Retrieval-Augmented Generation?
Also known as: RAG
Retrieval-augmented generation (RAG) is a technique where an AI system first searches a set of documents or data for information relevant to a question, then gives that information to a language model so its answer is grounded in those sources.
Retrieval-augmented generation is how you make an AI answer questions about your stuff — your company handbook, product docs, contracts, support tickets — instead of whatever it half-remembers from training. The term was popularised by a 2020 research paper from Facebook AI Research (now Meta AI) and collaborators.
How it works
- Prepare. Your documents are split into chunks. Each chunk is usually converted into an embedding — a list of numbers capturing its meaning — and stored, often in a vector database.
- Retrieve. When a user asks a question, the system finds the chunks most relevant to it (by meaning, keywords, or both).
- Augment. Those chunks are inserted into the prompt alongside the question.
- Generate. The language model writes an answer based on the provided text, ideally citing which chunks it used.
Because only relevant chunks are sent, RAG works even when your knowledge base is far larger than the model’s context window.
An analogy for the meatbags
A model without RAG is a student answering from memory: impressive, occasionally wrong, unable to know anything after its last lesson. A model with RAG is the same student allowed to search the library before answering — and to point at the page it used.
Why it matters for the Chosen
RAG is behind a huge share of practical business AI:
- Internal knowledge assistants that answer staff questions from policies and wikis.
- Customer-support bots that answer from help-centre articles and order data.
- Research and legal tools that search case files or contracts.
- AI search engines like Perplexity, which retrieve web pages before answering.
For freelancers and agencies, “chat with your documents” systems are a common, sellable project: businesses have plenty of documents and little patience for searching them. The value is in clean data preparation, good retrieval, and honest citations — not just plugging a PDF into a chatbot.
Where RAG goes wrong
Bad chunking splits answers across pieces; retrieval returns similar-sounding but wrong passages; outdated documents produce outdated answers; and the model may still embellish beyond its sources. RAG reduces hallucinations — it doesn’t abolish them. Test with real questions, show sources, and keep the knowledge base current.
Tools from the Arsenal
Perplexity
ProductivityAI answer engine that searches the web and replies with cited sources.
Notion
ProductivityAll-in-one workspace for docs, wikis and databases, with Notion AI agents and meeting notes built in.
Claude
AI WritingAnthropic's assistant: the strongest long-form writer and careful reasoner in the consumer AI pack.
Relevance AI
Automation & AgentsBuild an AI workforce of specialist agents for sales and customer operations.
Questions the Humans Keep Asking
What is the difference between RAG and fine-tuning?
RAG gives the model relevant information at question time, so it's easy to update and can cite sources. Fine-tuning changes the model's weights through extra training, which is better for teaching a style, format or specialised behaviour than for keeping facts current. Many systems use both.
Does RAG stop hallucinations?
It reduces them significantly for questions your documents cover, but doesn't eliminate them. The model can still misread sources, or retrieval can miss the right document. Showing citations lets users check.
Do I need a vector database for RAG?
Not always. Many RAG systems use embeddings stored in a vector database for semantic search, but retrieval can also use keyword search, a regular database, web search or a mix (hybrid search).