RAG Design Patterns

Naive vector database lookups are often insufficient for enterprise-grade applications. Production systems utilize specialized RAG Design Patterns to evaluate retrieval quality, dynamically route queries based on complexity, balance search granularity, and leverage caching mechanisms to control costs.

1. 🔀 Corrective RAG (CRAG)

Corrective RAG (CRAG) introduces a self-correction mechanism to validate retrieved contexts before feeding them to the synthesizer. It uses a lightweight retrieval evaluator (often a fine-tuned binary classifier or a smaller LLM) to score the relevance of retrieved document chunks relative to the user query.

If the evaluator determines the retrieved context is insufficient or irrelevant, the system triggers a fallback route to query external search APIs (e.g. Tavily, Google Search) to retrieve fresh public web context.

CRAG Execution Flow

Key Advantages

• Safety Net: Prevents the LLM from synthesizing answers based on noisy or missing internal documents.
• Hybrid Knowledge: Automatically bridges private data with public web data when internal resources fall short.

2. 🔃 Self-RAG (Self-Reflective RAG)

Self-RAG is an active retrieval framework where the language model generates both answers and specialized reflection tokens to critique its own generations. Rather than operating as a one-way pipeline, Self-RAG executes an iterative generator-evaluator loop.

The model evaluates the output across three distinct dimensions represented by critique tokens:

1. Is Retrieval Needed? ([Retrieval]): Determines if the query requires external facts.
2. Is Context Relevant? ([Relevant]): Rates if the retrieved passages support the query.
3. Is Response Grounded? ([Grounded]): Assesses if the generated response is strictly supported by the retrieved context, eliminating hallucinations.
4. Is Response Useful? ([Utility]): Ranks the helpfulness and factual correctness of the answer.

Self-RAG Iterative Refinement Loop

3. 🧠 Adaptive RAG

Adaptive RAG is a query-routing design pattern that dynamically adapts the retrieval strategy based on the complexity of the user query. Simple queries do not require multi-stage vector search pipelines, while complex, multi-hop queries cannot be answered by a single vector search turn.

The system uses a query classifier (often a fast classifier model like a fine-tuned DistilBERT or a lightweight LLM running with structured schema output) to classify queries into three complexity tiers:

1. No Retrieval: Simple factual or conversation queries (e.g., “Hello” or “Translate this phrase”). Routed directly to the LLM.
2. Single-Step RAG: Standard queries requiring specific internal documentation (e.g., “What is our company’s remote work policy?”). Routed to a fast Hybrid Search pipeline.
3. Multi-Step Agentic RAG: Complex queries requiring analysis across multiple databases or entity connections (e.g., “Compare our Q3 sales figures in Texas to Q4 figures in California and summarize the trends”). Routed to an agentic loop that executes sub-queries and aggregates contexts.

4. 🗂️ Parent-Child Hierarchical Retrieval

Vector similarity searches perform best when documents are broken down into small, semantically cohesive snippets (e.g., 100–250 tokens). However, feeding extremely small snippets into the LLM context window strips away critical surrounding context, headers, and structural document metadata, leading to fragmented responses.

Parent-Child Retrieval addresses this by separating the text chunks used for vector indexing from the text blocks injected into the prompt:

• Child Chunks: Small, granular text segments (e.g., 128 tokens) parsed with high overlap and vectorized.
• Parent Chunks: Larger, cohesive text sections (e.g., 1024 tokens) containing the children’s text and all surrounding context.

At query time, the system performs vector search against the child embeddings. When a child match is identified, the system uses parent reference pointers to retrieve the corresponding parent chunk, injecting the larger parent text into the LLM context.

Parent-Child Store Retrieval Implementation

Below is a Python implementation demonstrating how to build a parent-child mapping vector retrieval interface:

from typing import Dict, List, Tuple
import numpy as np
 
class ParentChildVectorStore:
    def __init__(self):
        # Maps child_id -> (embedding_vector, parent_id)
        self.child_index: Dict[str, Tuple[np.ndarray, str]] = {}
        # Maps parent_id -> parent_document_text
        self.parent_store: Dict[str, str] = {}
 
    def insert_document(self, parent_id: str, parent_text: str, children: List[Tuple[str, np.ndarray]]):
        """
        Stores the raw parent text and indexes child embeddings.
        
        parent_id: Unique identifier for the parent block.
        parent_text: The complete surrounding context.
        children: List of tuples containing (child_id, child_embedding_vector).
        """
        self.parent_store[parent_id] = parent_text
        for child_id, embedding in children:
            self.child_index[child_id] = (embedding, parent_id)
 
    def retrieve_context(self, query_vector: np.ndarray, top_k_children: int = 3) -> List[str]:
        """
        Matches query against child embeddings and returns the parent contexts.
        """
        scores = []
        for child_id, (embedding, parent_id) in self.child_index.items():
            # Calculate Cosine Similarity
            dot_product = np.dot(query_vector, embedding)
            norm_q = np.linalg.norm(query_vector)
            norm_e = np.linalg.norm(embedding)
            similarity = dot_product / (norm_q * norm_e) if (norm_q > 0 and norm_e > 0) else 0.0
            scores.append((similarity, parent_id))
        
        # Sort child matches by similarity score descending
        scores.sort(key=lambda x: x[0], reverse=True)
        
        # Collect unique parent documents from top child matches
        retrieved_parents = []
        seen_parents = set()
        
        for _, parent_id in scores:
            if parent_id not in seen_parents:
                seen_parents.add(parent_id)
                retrieved_parents.append(self.parent_store[parent_id])
                # Bounded by desired context chunk count
                if len(retrieved_parents) >= top_k_children:
                    break
                    
        return retrieved_parents

5. 💾 Context Caching

Large RAG applications often process long, static system contexts, persistent API documentation, or heavy historical chat logs. Re-sending and re-tokenizing this static context on every user turn incurs significant processing latency and API cost.

Context Caching allows LLM providers (e.g., Anthropic Prompt Caching, Google Gemini Context Caching) to cache prefix tokens (e.g. system instructions, background documents) in memory between API calls.

Operational Guidelines

• Minimum Size Bounds: Providers typically enforce a minimum token limit to trigger caching (e.g. 1024 tokens for Anthropic, 32k tokens for Gemini).
• Stable Prefixes: Caching is sequential. All cached content must sit at the absolute beginning of the prompt. Dynamic components (like the new user query or current timestamp) must be placed after the cached block, or the cache will invalidate.
• Time-to-Live (TTL): Caches remain active for short windows (e.g., 5 to 10 minutes) and auto-expire to free provider memory.

🔗 Related Sections

• Anatomy of RAG Systems — Ingestion structures, hybrid search, and pgvector HNSW configurations.
• Paradigms of RAG — Comparison of Naive RAG, Advanced RAG, Agentic RAG, and GraphRAG.
• Open Source RAG Tools — Framework comparisons and hybrid retrieval implementation scripts.