RAG - lesson 3 | HackIndia Skills

Key point

One idea

A vector index is a search table for meaning. You embed each document chunk once, store it with its source details, then embed the user question and fetch the closest chunks.

What you are building

In lesson 2, you prepared clean chunks. Now you will create a local vector index for those chunks.

Use Chroma for the local vector store. It is free, runs on your laptop, and saves the index to a folder you can rebuild from your repo.

Your to-do

Install packages

Do each one yourself, then tap it to tick it off. The ticks are only a checklist for you: they are not marked or scored.

Create a new Python virtual environment if you do not have one.
Install Chroma and an embedding library.
Keep your package list in requirements.txt.
Do not commit large generated index folders unless your project needs them.

0 of 4 done

pip install chromadb sentence-transformers
pip freeze > requirements.txt

Example

Index your chunks

This example assumes you have a JSONL file from lesson 2. Each line has chunk text and metadata.

Use a small Sentence Transformers model so the project works without paid APIs. You can swap it later.

# index_chunks.py
import json
import chromadb
from chromadb.utils import embedding_functions

CHUNKS_FILE = "data/chunks.jsonl"
DB_DIR = "chroma_db"
COLLECTION = "doc_chunks"

embedding_fn = embedding_functions.SentenceTransformerEmbeddingFunction(
    model_name="all-MiniLM-L6-v2"
)

client = chromadb.PersistentClient(path=DB_DIR)
collection = client.get_or_create_collection(
    name=COLLECTION,
    embedding_function=embedding_fn
)

ids = []
documents = []
metadatas = []

with open(CHUNKS_FILE, "r", encoding="utf-8") as f:
    for i, line in enumerate(f):
        item = json.loads(line)
        ids.append(item.get("id", f"chunk-{i}"))
        documents.append(item["text"])
        metadatas.append({
            "source": item.get("source", "unknown"),
            "page": item.get("page", ""),
            "chunk_id": item.get("id", f"chunk-{i}")
        })

collection.add(ids=ids, documents=documents, metadatas=metadatas)
print(f"Indexed {len(ids)} chunks into {DB_DIR}")

Example

Search the index

Retrieval is the test. If your index cannot fetch useful chunks, the answer step will fail later.

Start with top 3 results. Read them. Check if they actually help answer the question.

# search.py
import chromadb
from chromadb.utils import embedding_functions

DB_DIR = "chroma_db"
COLLECTION = "doc_chunks"

embedding_fn = embedding_functions.SentenceTransformerEmbeddingFunction(
    model_name="all-MiniLM-L6-v2"
)

client = chromadb.PersistentClient(path=DB_DIR)
collection = client.get_collection(
    name=COLLECTION,
    embedding_function=embedding_fn
)

question = "What is the refund policy for late cancellation?"

results = collection.query(
    query_texts=[question],
    n_results=3
)

for rank, (doc, meta, dist) in enumerate(zip(
    results["documents"][0],
    results["metadatas"][0],
    results["distances"][0]
), start=1):
    print(f"\nResult {rank} | distance={dist:.4f}")
    print(f"Source: {meta['source']} page {meta.get('page', '')}")
    print(doc[:700])

Common mistake

Do not store only vectors. Store the original chunk text and metadata too.

Your app needs the text for the prompt in lesson 4. It needs metadata for citations in lesson 5, such as file name, page, section, or URL.

Tip

Use AI carefully

You can ask ChatGPT or Gemini to review your retrieval results. Paste the question and top chunks, then ask: “Which chunks are relevant, which are noise, and what query wording would retrieve better evidence?”

Check the advice yourself. AI may suggest changing the question, but users will ask messy questions. Your index should handle normal wording.