RAG for Codebase Understanding

RAG for Codebase Understanding enables engineers to answer complex, natural-language questions about a codebase by combining retrieval of relevant code snippets with large language model generation. It matters when engineers need to quickly grasp unfamiliar code, onboard new team members, debug cross-cutting issues, or refactor legacy systems—especially in monorepos, polyglot environments, or rapidly evolving codebases. This guide walks through the end-to-end workflow, from document ingestion to query execution, with specific, executable steps and common pitfalls.

Ingesting Codebase Documents

To build a retrieval-augmented knowledge base, source code must be transformed into structured, searchable documents. Each document represents a code unit: a file, function, class, or module, with metadata and content suitable for embedding and retrieval.

Use git ls-files to identify all source files, then extract content with find and xargs:

find src -name "*.py" -o -name "*.ts" -o -name "*.go" -o -name "*.rs" | \
  xargs -I {} sh -c 'echo "{}"; cat {}' > codebase-docs.txt

Each document should be a JSON object with path, language, content, and metadata fields. The metadata includes file_type, function_names, class_names, tags, and source.

For example, a TypeScript file src/api/users.ts might become:

{
  "path": "src/api/users.ts",
  "language": "typescript",
  "content": "export class UserService {\n  async getUser(id: number): Promise<User> {\n    const res = await fetch(`/api/users/${id}`);\n    return res.json();\n  }\n}\n\nexport interface User {\n  id: number;\n  name: string;\n  email: string;\n}",
  "metadata": {
    "file_type": "module",
    "function_names": ["getUser"],
    "class_names": ["UserService"],
    "tags": ["api", "user", "service"],
    "source": "github.com/org/project"
  }
}

Use jq to transform a directory of source files into this format:

find src -type f -name "*.ts" | \
  xargs -I {} sh -c '
    echo "{\n  \"path\": \"{}\",\n  \"language\": \"typescript\",\n  \"content\": \"$(cat {})\",\n  \"metadata\": {\n    \"file_type\": \"module\",\n    \"function_names\": [],\n    \"class_names\": [],\n    \"tags\": [\"module\"],\n    \"source\": \"github.com/org/project\"\n  }\n}"' | \
  jq -s . > docs.json

Embedding Documents with a Pre-trained Model

Once documents are structured, apply embeddings using a model like sentence-transformers/all-MiniLM-L6-v2 via transformers or onnxruntime.

Generate embeddings with python and torch:

from transformers import AutoTokenizer, AutoModel
import torch
import json

model_name = "sentence-transformers/all-MiniLM-L6-v2"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

def embed_document(doc):
    inputs = tokenizer(doc["content"], return_tensors="pt", truncation=True, padding=True, max_length=512)
    with torch.no_grad():
        outputs = model(**inputs)
    return outputs.last_hidden_state.mean(dim=1).squeeze().tolist()

with open("docs.json", "r") as f:
    documents = json.load(f)

embeddings = []
for doc in documents:
    embedding = embed_document(doc)
    doc["embedding"] = embedding
    embeddings.append(doc)

with open("documents-with-embeddings.json", "w") as f:
    json.dump(embeddings, f, indent=2)

Embedding is expensive and must be batched. Use torch.utils.data.DataLoader for efficient batching:

from torch.utils.data import DataLoader

dataset = [doc["content"] for doc in documents]
dataloader = DataLoader(dataset, batch_size=64, shuffle=False)

embeddings = []
for batch in dataloader:
    inputs = tokenizer(batch, return_tensors="pt", padding=True, truncation=True, max_length=512)
    with torch.no_grad():
        outputs = model(**inputs)
    batch_embeddings = outputs.last_hidden_state.mean(dim=1).tolist()
    embeddings.extend(batch_embeddings)

Storing and Querying with a Vector Database

Persist embeddings in a vector database. Use pgvector in PostgreSQL or weaviate with docker-compose.

Example docker-compose.yml for Weaviate:

version: '3.8'
services:
  weaviate:
    image: semitechnologies/weaviate:latest
    ports:
      - "8080:8080"
    environment:
      - QUERY_DEFAULTS_LIMIT=25
      - DEFAULT_VECTORS_CONFIG
      - AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED=true
      - PERSISTENCE_DATA_PATH=/var/lib/weaviate
    volumes:
      - weaviate_data:/var/lib/weaviate
    networks:
      - weaviate-net

volumes:
  weaviate_data:

networks:
  weaviate-net:

Use curl to define a schema in Weaviate:

curl -X POST http://localhost:8080/v1/schemas \
  -H "Content-Type: application/json" \
  -d '{
    "classes": [
      {
        "class": "CodeDocument",
        "vectorizer": "text2vec-transformers",
        "properties": [
          { "name": "path", "dataType": ["string"] },
          { "name": "language", "dataType": ["string"] },
          { "name": "content", "dataType": ["text"] },
          { "name": "tags", "dataType": ["string[]"] },
          { "name": "source", "dataType": ["string"] }
        ]
      }
    ]
  }'

Import documents using the Weaviate REST API:

curl -X POST http://localhost:8080/v1/objects \
  -H "Content-Type: application/json" \
  -d '{
    "class": "CodeDocument",
    "properties": {
      "path": "src/api/users.ts",
      "language": "typescript",
      "content": "export class UserService { ... }",
      "tags": ["api", "user", "service"],
      "source": "github.com/org/project"
    },
    "vector": [
      0.12, -0.34, 0.55, ...
    ]
  }'

For bulk import, use jq to convert documents-with-embeddings.json to Weaviate import format:

jq -c 'map({class: "CodeDocument", properties: .})' documents-with-embeddings.json > import.json

Then stream it with curl:

curl -X POST http://localhost:8080/v1/objects/batch \
  -H "Content-Type: application/json" \
  -d @import.json

Querying the Codebase

To answer a natural language question, retrieve relevant code snippets and pass them to a language model.

Given the question *"How is user authentication handled in the API layer?"*, perform a nearest-neighbor search in the vector database:

curl -X POST http://localhost:8080/v1/query \
  -H "Content-Type: application/json" \
  -d '{
    "nearVector": {
      "vector": [0.12, -0.34, 0.55, ...],
      "certainty": 0.7
    },
    "limit": 10,
    "className": "CodeDocument",
    "returnProperties": [
      "path",
      "language",
      "content",
      "tags",
      "source"
    ]
  }'

The nearVector query returns top-k documents. Each document includes path, content, and metadata. Use jq to extract and filter relevant snippets:

curl -X POST http://localhost:8080/v1/query \
  -H "Content-Type: application/json" \
  -d '{
    "nearVector": {
      "vector": [0.12, -0.34, 0.55, ...],
      "certainty": 0.7
    },
    "limit": 10,
    "className": "CodeDocument",
    "returnProperties": [
      "path",
      "content"
    ]
  }' | \
  jq -r '.objects[] | "\(.path)\n\n\(.content)\n\n---"' > relevant-snippets.txt

Augmenting the LLM Prompt

Combine retrieved snippets into a prompt for a language model like gpt-3.5-turbo via curl to OpenAI's API:

curl -X POST https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-..." \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {
        "role": "system",
        "content": "You are a senior engineer. Answer the user's question using the provided codebase context."
      },
      {
        "role": "user",
        "content": "How is user authentication handled in the API layer?\n\nRelevant code:\n\nsrc/api/auth.ts\n\nexport class AuthMiddleware {\n  async authenticate(req, res, next) {\n    const token = req.headers.authorization?.split(' ')[1];\n    if (!token) return res.status(401).send('Unauthorized');\n    const payload = jwt.verify(token, process.env.JWT_SECRET);\n    req.user = payload;\n    next();\n  }\n}\n\nsrc/api/users.ts\n\nexport class UserService {\n  async getUser(id: number): Promise<User> {\n    const res = await fetch(`/api/users/${id}`);\n    return res.json();\n  }\n}\n\nexport interface User {\n  id: number;\n  name: string;\n  email: string;\n}"
      }
    ],
    "temperature": 0.4,
    "max_tokens": 512
  }'

The prompt includes the full question and the concatenated code snippets. The model generates a natural language answer with references to specific files and functions.

Common Failure Modes and Sharp Edges

Embedding mismatch: The model used to generate embeddings during ingestion must match the model used at query time. A mismatch in tokenizer or embedding dimension causes poor retrieval. Use tensor.size() to verify dimensions.

Query vector not normalized: Most vector databases expect normalized vectors. If embeddings are not L2-normalized, nearVector queries return suboptimal results. Normalize embeddings before storing:

from sklearn.preprocessing import normalize
embeddings = normalize(embeddings, axis=1)

Metadata filtering ignored: Weaviate allows filtering by metadata (tags, language) during retrieval, but queries often omit the filters section. Include it explicitly:

"filters": {
  "operator": "And",
  "operands": [
    { "path": ["language"], "operator": "Equal", "valueString": "typescript" },
    { "path": ["tags"], "operator": "ContainsAll", "valueStringArray": ["api", "service"] }
  ]
}

Snippet overlap: Multiple documents from the same file or function can repeat content. Use path and content deduplication in the query pipeline. In Weaviate, use grouping to merge duplicate documents.

Poor retrieval when tags are missing: A file with no tags in metadata may be missed in a query filtering by tags. Always ensure metadata is extracted consistently across the ingestion pipeline.

Missing certainty in nearVector: The certainty field in nearVector is often misunderstood. A value of 0.7 means “return documents with cosine similarity ≥ 0.7 to the query vector.” Use certainty to control recall vs. precision.

LLM prompt overflow: When too many snippets are retrieved, the prompt exceeds the model’s context window. Use max_tokens and temperature to balance clarity and length. Pre-trim large code blocks.

Embedding model choice matters: all-MiniLM-L6-v2 performs well across languages, but microsoft/codebert-base or facebook/roberta-base may outperform on code-heavy questions. Evaluate using a small set of real engineering questions.

Iterative Improvement

To refine the system, track query performance. Log the original question, retrieved snippets, and generated answer. Use psql to store logs:

CREATE TABLE rag_queries (
  id SERIAL PRIMARY KEY,
  question TEXT NOT NULL,
  retrieved_snippets JSONB NOT NULL,
  generated_answer TEXT NOT NULL,
  query_time_ms INTEGER,
  embedding_model TEXT,
  query_vector FLOAT[],
  created_at TIMESTAMPTZ DEFAULT NOW()
);

Insert results from a query:

INSERT INTO rag_queries (question, retrieved_snippets, generated_answer, query_time_ms, embedding_model, query_vector)
VALUES (
  'How is user authentication handled in the API layer?',
  '[{"path": "src/api/auth.ts", "content": "..."}, ...]',
  'User authentication is handled via JWT middleware in the API layer...',
  214,
  'sentence-transformers/all-MiniLM-L6-v2',
  '{0.12, -0.34, 0.55, ...}'
);

Use this log to tune embedding models, retrieval parameters, and prompt templates.

This page was rewritten on 11 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.

Join the Newsletter

Get the latest updates and deep insights shipped directly to your inbox.