RAG (Retrieval-Augmented Generation)
Improving accuracy by injecting external knowledge through retrieval
RAG (Retrieval-Augmented Generation) was proposed by Meta (Facebook) in 2020 and is the most practical method for solving LLM hallucination and knowledge limitations. The core idea is to "find relevant documents first, then generate answers referencing those documents." Documents are split into chunks, vectorized with an embedding model, and stored in a vector DB (Pinecone, Chroma, etc.). When a question arrives, similarity search finds relevant chunks that are appended to the prompt. This enables utilizing latest information and specialized knowledge without retraining.
Key Concepts
Collect documents and split into chunks (500-1000 token units)
Vectorize each chunk with an embedding model โ store in vector DB
Vectorize the user question with the same embedding model
Search for top-k relevant chunks via vector similarity (cosine similarity)
Append retrieved chunks as "reference material" to the prompt
LLM generates answers based on reference material (can cite sources)
Pros
- ✓ Knowledge updates possible without retraining
- ✓ Reduced hallucination
- ✓ Improved credibility through source citation
- ✓ Cost-effective compared to fine-tuning
- ✓ Can maintain data security (internal DB)
Cons
- ✗ Retrieval quality determines overall performance
- ✗ Embedding/vector DB management costs
- ✗ Complex chunk splitting strategies
- ✗ Reference material limited by context window
- ✗ Retrieval-generation pipeline latency