LLM APIs - lesson 3 | HackIndia Skills

Key point

One idea

By default, the API returns the full generated output in one HTTP response. Streaming lets your app display the beginning while the rest is still being generated.

Use the Responses API from your server route, set streaming on there, then forward small text chunks to the browser.

Why stream

Streaming is useful when the answer may take a few seconds. A resume reviewer, trip planner, study helper, or bug explainer feels better when text starts appearing quickly.

Do not stream by calling OpenAI from React, Flutter, or browser JavaScript. Your OpenAI key must stay in the backend, usually loaded from OPENAI_API_KEY.

Your to-do

Data path

Do each one yourself, then tap it to tick it off. The ticks are only a checklist for you: they are not marked or scored.

Send the user’s prompt from the UI to your own backend route.
Call the OpenAI Responses API from that route.
Enable streaming on the backend request.
Forward text chunks from your route to the UI using an HTTP stream.
Append each chunk to the page as it arrives.

0 of 5 done

Example

Server route

This example shows the shape you want in a Node-style server route. Keep your real product rules in instructions. Keep the user’s text in input.

Older tutorials may show Chat Completions first. For this track, use Responses API for new text features.

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

export async function POST(req) {
  const { prompt } = await req.json();

  const stream = await client.responses.create({
    model: "gpt-4o-mini",
    instructions: "You are a concise study assistant. Explain in simple steps.",
    input: prompt,
    stream: true,
  });

  const encoder = new TextEncoder();

  return new Response(
    new ReadableStream({
      async start(controller) {
        try {
          for await (const event of stream) {
            if (event.type === "response.output_text.delta") {
              controller.enqueue(encoder.encode(event.delta));
            }
          }
          controller.close();
        } catch (error) {
          controller.error(error);
        }
      },
    }),
    {
      headers: {
        "Content-Type": "text/plain; charset=utf-8",
      },
    }
  );
}

Example

UI reader

The browser calls only your route. It reads the response body chunk by chunk and appends decoded text to state.

This keeps the API key out of the client and still gives a live typing effect.

async function askAI(prompt) {
  setAnswer("");

  const res = await fetch("/api/ai/stream", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ prompt }),
  });

  if (!res.ok || !res.body) {
    setAnswer("Something went wrong. Try again.");
    return;
  }

  const reader = res.body.getReader();
  const decoder = new TextDecoder();

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    setAnswer((prev) => prev + decoder.decode(value, { stream: true }));
  }
}

Common mistake

Do not wait for the whole answer on the server, then send it to the UI. That is a normal response, not streaming.

Also do not parse streamed text as final JSON. If your app needs reliable JSON, use Structured Outputs in the next lesson instead of a free-form stream.

Tip

Make it demoable

Add a visible loading state before the first chunk arrives. Then replace it with streamed text once chunks start coming in.

For a hackathon demo, use a feature where partial text is useful: explanation, feedback, summary, or chat reply. Keep the prompt short so your demo works on college Wi-Fi.