One habit
Treat a prompt like a function. It takes input, returns output, and should pass repeatable checks before you trust it in a demo.
You do not need a huge test suite. For a hackathon project, 8 to 15 real cases are enough to catch many bad prompt changes.
What to test
Test the task your user actually needs. If your app helps students turn messy notes into a study plan, test messy notes from real college life, not only clean samples.
Each test case needs input, the expected behaviour, and a simple pass check. The pass check can be exact, like valid JSON. It can also be human-rated, like “mentions deadline and splits work by day”.
Build a test set
Do each one yourself, then tap it to tick it off. The ticks are only a checklist for you: they are not marked or scored.
0 of 5 done
Small test file
A plain JSON file is enough. Start simple before adding a test framework.
This example tests a prompt that converts a student request into a support ticket summary.
[
{
"id": "case_01_late_fee",
"input": "I paid hostel fees by UPI yesterday but portal still says pending. Last date is today.",
"must_include": ["hostel fees", "UPI payment", "portal pending", "deadline today"],
"must_not_include": ["refund confirmed", "payment failed"],
"format": "json"
},
{
"id": "case_02_hinglish",
"input": "Mera ID card lost ho gaya, kal lab exam hai. Entry milegi kya?",
"must_include": ["lost ID card", "lab exam", "entry request"],
"must_not_include": ["police complaint required"],
"format": "json"
}
]Use AI carefully
You can ask ChatGPT, Gemini, or Claude to draft test cases. Give it your app goal and ask for realistic Indian college examples, edge cases, and failure cases.
Do not accept the list blindly. Check that the cases match your real users, do not include private data, and do not test features your app does not support.
Common mistake
Do not test only one happy path in the UI. A prompt that works once during practice can fail when the input is shorter, noisier, or in mixed language.
Also avoid changing the prompt during the demo without rerunning tests. Small wording changes can break structure, tone, or safety rules.
Simple test runner
This script calls your own function and checks basic rules. Replace runPrompt with your app’s model call.
Store the output logs. They help you explain your testing in the final submission.
import fs from "fs";
const cases = JSON.parse(fs.readFileSync("prompt-tests.json", "utf8"));
async function runPrompt(input) {
// Replace this with your actual model call.
return JSON.stringify({
summary: "Student reports hostel fees paid by UPI but portal is pending. Deadline is today.",
category: "fees_support"
});
}
for (const testCase of cases) {
const output = await runPrompt(testCase.input);
let passed = true;
for (const phrase of testCase.must_include) {
if (!output.toLowerCase().includes(phrase.toLowerCase())) passed = false;
}
for (const phrase of testCase.must_not_include) {
if (output.toLowerCase().includes(phrase.toLowerCase())) passed = false;
}
if (testCase.format === "json") {
try {
JSON.parse(output);
} catch {
passed = false;
}
}
console.log(`${testCase.id}: ${passed ? "PASS" : "FAIL"}`);
console.log(output);
}Do this now
Do each one yourself, then tap it to tick it off. The ticks are only a checklist for you: they are not marked or scored.
0 of 5 done