RAG - lesson 4 | HackIndia Skills

Key point

The core rule

A RAG answer is only as good as the context you put into the prompt. Treat retrieved chunks as evidence, not background decoration.

Your prompt must clearly separate instructions, evidence, and the user question. Then ask the model to answer only from the evidence.

What the prompt needs

After lesson 3, you have search results from a vector index. Each result should include chunk text and metadata such as file name, page, section, or chunk id.

In this lesson, your job is to pack the top chunks into a clean prompt. Do not paste the whole document. Use the few most relevant chunks that fit your model limit.

Your to-do

Build the prompt

Do each one yourself, then tap it to tick it off. The ticks are only a checklist for you: they are not marked or scored.

Collect the top 3 to 6 chunks returned by your retriever.
Label each chunk with a stable id like S1, S2, S3.
Include useful metadata such as file name and page number.
Tell the model to use only the provided evidence.
Tell the model what to do when the answer is not in the evidence.

0 of 5 done

Example

Prompt template

Use a template like this before calling your LLM. The important parts are the evidence labels and the fallback rule.

This example is plain Python. Connect it to whatever model call your app uses.

def build_prompt(question, retrieved_chunks):
    evidence_blocks = []

    for i, chunk in enumerate(retrieved_chunks, start=1):
        source_id = f"S{i}"
        file_name = chunk["metadata"].get("file", "unknown file")
        page = chunk["metadata"].get("page", "unknown page")
        text = chunk["text"].strip()

        evidence_blocks.append(
            f"[{source_id}] File: {file_name}, Page: {page}\n{text}"
        )

    evidence = "\n\n".join(evidence_blocks)

    return f"""
You are a document question-answering assistant.
Answer the user's question using only the evidence below.
If the evidence does not contain the answer, say: "I could not find this in the provided documents."
Do not use outside knowledge.
Keep the answer short and specific.

Evidence:
{evidence}

Question:
{question}

Answer:
""".strip()

Tip

Keep evidence small

More chunks do not always mean better answers. Too much context can bury the useful line and increase cost.

Start with top 4 chunks. If answers miss details, improve chunking or retrieval before increasing the number of chunks.

Common mistake

Do not trust wording

A model may still sound confident when the evidence is weak. Strong wording is not proof of grounding.

Make the prompt force uncertainty. The fallback sentence is part of your app, not a nice extra.