LLM APIs - lesson 5 | HackIndia Skills

Key point

One idea

Your AI route is not done when it works once. It is done when it fails safely, retries only when useful, and tells the user what happened.

The main failures to plan for are rate limits, overloaded service, too much input, and too much output.

Know the errors

A 429 means you sent too many requests or tokens in a short time, or you hit a project, organization, credit, spend, or usage limit. Do not spam retries.

A 503 server_is_overloaded means the requested model temporarily lacks capacity. Wait, retry a few times, then show a useful message.

Your to-do

Set safe limits

Do each one yourself, then tap it to tick it off. The ticks are only a checklist for you: they are not marked or scored.

Set max_output_tokens on every Responses API call.
Reject very large user input before it reaches OpenAI.
Keep your developer instructions short and stable.
Show a clear message when the request is too large.
Log enough detail on the server to debug, not in the browser.

0 of 5 done

Common mistake

Beginners retry every failed request the same way. That can make 429 errors worse and burn credits faster.

Retry only temporary failures like 429 and 503. Follow Retry-After when the API sends it. Otherwise use exponential backoff with jitter and stop after a small number of tries.

Example

Retry wrapper

This example is for a server route. It keeps the key on the backend, caps output, and turns API failures into app-friendly errors.

Older tutorials may use max_tokens or Chat Completions. For the Responses API, use max_output_tokens.

import OpenAI from "openai";

const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

function sleep(ms) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}

function retryDelayMs(error, attempt) {
  const retryAfter = error?.headers?.get?.("retry-after");
  if (retryAfter) return Number(retryAfter) * 1000;

  const base = 500 * 2 ** attempt;
  const jitter = Math.floor(Math.random() * 250);
  return base + jitter;
}

export async function createAiReply(userText) {
  if (!userText || userText.length > 4000) {
    return { ok: false, status: 400, message: "Please send a shorter prompt." };
  }

  for (let attempt = 0; attempt < 3; attempt++) {
    try {
      const response = await client.responses.create({
        model: "gpt-4o-mini",
        instructions: "You help students improve hackathon project descriptions.",
        input: userText,
        max_output_tokens: 300
      });

      return { ok: true, text: response.output_text };
    } catch (error) {
      const status = error?.status;
      const temporary = status === 429 || status === 503;

      if (!temporary || attempt === 2) {
        console.error("OpenAI request failed", {
          status,
          code: error?.code,
          message: error?.message
        });

        const message = status === 429
          ? "AI is busy or this project hit a limit. Try again in a minute."
          : status === 503
            ? "AI service is temporarily overloaded. Try again shortly."
            : "AI request failed. Please try again.";

        return { ok: false, status: status || 500, message };
      }

      await sleep(retryDelayMs(error, attempt));
    }
  }
}

Tip

Good user messages

Do not show raw provider errors to users. They are often confusing and may reveal internal details.

Use plain messages: “Try a shorter prompt”, “Try again in a minute”, or “AI service is temporarily busy”. Keep the detailed error in server logs.