Introduction

Large language models are flexible by default — they can answer in any format, any length, any style. For real production use inside web apps, that flexibility becomes a problem. You need predictable, parseable output. This lesson covers four core techniques to control what an LLM produces: system prompts to set persistent behavior, few-shot examples to teach output patterns by demonstration, format enforcement to guarantee structure, and edge case handling for when the model tries to go off-script.

System Prompts vs User Prompts

System prompts are sent as a separate message with higher priority than user messages. They define the model's role, constraints, and tone for the entire conversation. User prompts are the actual task or question. Keep the system prompt focused on stable rules and the user prompt focused on the variable task.

const response = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [
    { role: "system", content: "You are a JSON API. Respond ONLY with valid JSON. Never include explanations, markdown fences, or extra text. If you cannot comply, return {\"error\": \"reason\"}." },
    { role: "user", content: "List 3 colors" }
  ],
  temperature: 0
});

Notice `temperature: 0` — when you need structured output, lock randomness to zero. Higher temperatures introduce formatting drift even with a strong system prompt.

Few-Shot Examples

Few-shot prompting means showing the model 2-4 examples of the exact input-to-output mapping you want, instead of describing it. The model learns the pattern by pattern-matching. Place examples inside the user prompt or as alternating user/assistant message pairs.

const messages = [
  { role: "system", content: "Extract the product name and price from review text." },
  { role: "user", content: "Review: The Sony WH-1000XM5 headphones cost $399." },
  { role: "assistant", content: '{"product": "Sony WH-1000XM5", "price": 399}' },
  { role: "user", content: "Review: Got the Instant Pot Duo for $99, works great." },
  { role: "assistant", content: '{"product": "Instant Pot Duo", "price": 99}' },
  { role: "user", content: "Review: My new Logitech MX Master 3S was a steal at 79 dollars." },
];

The third user message has no assistant reply — the model fills it in. Because it saw two consistent examples, it will almost always follow the same JSON shape.

Format Enforcement Strategies

When the model is stubborn and ignores your format instructions, layer these defenses in order: first, sharpen the system prompt with explicit negative examples ("Do NOT write 'Here is your JSON:'"). Second, switch to a model that supports structured output mode (JSON mode or tool/function calling) where the API guarantees valid JSON. Third, add a post-processing step in your code that validates the output with a schema and retries on failure.

import { z } from "zod";
import { ZodError } from "zod";

const ColorList = z.object({
  colors: z.array(z.string()).length(3)
});

async function getColors() {
  const raw = await callLLM("List 3 colors");
  try {
    const parsed = ColorList.parse(JSON.parse(raw));
    return parsed;
  } catch (e) {
    if (e instanceof ZodError) {
      return { colors: [] as string[] };
    }
    throw e;
  }
}

The Zod schema acts as a contract — if the LLM returns a number instead of a string, or returns 4 colors instead of 3, parse throws and your fallback kicks in. Never trust raw LLM output in production code.

Handling Edge Cases

Three edge cases come up constantly. First, the model refuses and returns an apology instead of data — handle by detecting refusal phrases and retrying with a clearer prompt. Second, the model hallucinates a field that does not exist in your schema — handle by validating against the schema, not just JSON.parse. Third, the model returns truncated output because it hit the token limit — handle by setting max_tokens explicitly and chunking long requests, or asking the model to be concise.

Summary

Controlling LLM output is not one trick — it is a stack of defenses. Start with a clear system prompt, add few-shot examples when format is ambiguous, enforce structure with schemas and validation, and always have a fallback for refusals, hallucinations, and truncation. In the next lesson we will look at function calling, which lets the model choose between tools instead of generating free-form text.