Why this matters
In a hackathon, you may test two button labels, onboarding screens, prompts, or recommendation rules. An A/B test helps you compare them without just trusting vibes.
You are not trying to prove a big truth. You are trying to make a small, honest claim like: “In our sample, version B got more sign-ups than A.”
Single idea
Change one thing, choose one metric, and split people randomly. If you change many things or pick the metric after seeing results, your test becomes a story, not evidence.
Set up the test
Do each one yourself, then tap it to tick it off. The ticks are only a checklist for you: they are not marked or scored.
0 of 5 done
Tiny split script
For a web project, assign a visitor once and save it in localStorage. This is enough for a demo, but not for serious production analytics.
Log the group and outcome to your backend, Google Sheet, Supabase table, or a CSV export. Check that both groups got similar numbers of users.
// Browser JS: assign A or B once per visitor
let variant = localStorage.getItem("variant");
if (!variant) {
variant = Math.random() < 0.5 ? "A" : "B";
localStorage.setItem("variant", variant);
}
// Example use
const button = document.querySelector("#submitBtn");
button.textContent = variant === "A" ? "Submit" : "Get result";
// When the user completes the action, log:
// { variant: "A", completed: true }Do not peek too much
Beginners often check results after every few users and stop when B looks ahead. That makes random luck look like a win.
For a hackathon, decide a simple stop rule first. For example: “We will compare after 40 total users or at demo time, whichever comes first.”
Read the result
Use counts first. Percentages without counts can mislead, especially with small samples.
Here is a simple result you could put in your evidence section.
A: 9 completions out of 25 users = 36%
B: 14 completions out of 27 users = 52%
Difference: B is 16 percentage points higher
Plain-language insight: In this sample, "Get result" had more completions than "Submit".
Limitation: The sample is small and mostly came from hackathon friends, so it may not represent real users.Use AI carefully
You can ask ChatGPT, Gemini, or Claude to help phrase the insight. Give it the counts, not just the percentages.
Check that it mentions sample size, does not say “proved”, and includes a limitation. Replace confident claims with “in this sample” or “suggests”.
Prompt:
Rewrite this A/B test result in plain language for a hackathon README.
Counts: A had 9 completions out of 25 users. B had 14 completions out of 27 users.
Do not claim causation beyond this test. Mention one limitation.Do this now
Do each one yourself, then tap it to tick it off. The ticks are only a checklist for you: they are not marked or scored.
0 of 5 done