Module 03 · Lesson 5 of 12Free

Structured outputs: getting reliable JSON

Level

Intermediate

Topic

Prompting and structured outputs

Reading time

16 min

Amara’s ticket waits five hours

Deskora is a help desk app. A shop’s support team uses it to answer customer emails from one shared inbox. Each customer problem becomes a ticket: one customer’s whole conversation with the support team, message by message.

Deskora receives 120,000 new tickets a day. Tomás, an engineer at Deskora, is building a ticket router. When a ticket arrives, a model reads the first message and returns three fields: a category, an urgency and the order number. Deskora’s code then puts the ticket in the right queue, with the order details attached.

Tomás’s system prompt ends with “Reply in JSON with the keys category, urgency and order_number.” On Monday morning, Amara emails a kitchenware shop: her order KS-20417 hasn’t arrived, and it’s a gift for Saturday. One run returned this:

Here is the ticket data:
{"category": "delivery", "urgency": "high", "order_number": "KS-20417"}

The fields are right, but the first line isn’t JSON. Python’s json.loads raises an error, and the ticket falls into the Unsorted queue. Nobody looks there until the afternoon. Priya, a support agent at the shop, finally sees it five hours later.

This lesson answers one question: how does Tomás get JSON his code can read on every ticket, and what can that JSON still get wrong?

The 120,000 tickets a day is a fact used across this course. The other numbers below, such as failure rates, token counts and the share of long tickets, are estimates for this lesson’s scenario.

Asking politely still breaks the router

A model writes its reply one token at a time. At each step it scores every possible next token and picks one; Next-token prediction shows how. Your prompt changes those scores. It can make “Here is the ticket data:” very unlikely, but nothing in the process forbids it.

So a prompt-only router fails in three ways:

Suppose, as an illustration, that 1 reply in 200 fails like this. That is 600 tickets a day in the Unsorted queue. Each one is a customer like Amara waiting hours for a first reply.

Retrying a failed reply is cheap: 600 extra calls add about $0.10 a day at the cost worked out below. But a retry can fail too, and it adds a second or more of waiting. Older tricks don’t help on current models either. Prefilling the reply with an opening { returns a 400 error on all current Claude models.

The prompt can only make bad tokens unlikely. To fix the router, Tomás needs something that makes them impossible.

Constrained decoding rules out broken JSON

Structured outputs is an API feature that makes the reply match a JSON schema you send with the request. A JSON schema is a JSON document that describes another one: which keys it must have, and what type each value is ^{4JSON Schema Validation: A Vocabulary for Structural Validation of JSON (draft 2020-12)Austin Wright, Henry Andrews, Ben Hutton, Greg Dennis, JSON Schema, 2022}.

The guarantee comes from how the reply is generated. The API compiles your schema into a grammar: rules for which characters may come next at each point in the reply. It then uses constrained decoding: it only lets the model produce tokens the grammar allows ^{1Structured outputsAnthropic, Claude API documentation}.

In general, constrained decoding works like this ^{5Efficient Guided Generation for Large Language ModelsBrandon T. Willard, Rémi Louf, arXiv, 2023}:

1. The model scores every possible next token, as it always does.
2. Before one is picked, every token that would break the grammar is removed.
3. The model picks from the tokens that are left.
4. The grammar moves forward, and the next step starts.

The model can no longer write “Here is”, because the grammar says a reply must start with {. It can’t skip a required key, because the grammar won’t let the object close without it. It can’t write "urgent", because the schema allows only three urgency values. How decoding picks a token from the scores is covered in Under the hood: how decoding picks each token.

This changes where the work happens. Before, the model had to choose to follow the format, and code had to cope when it didn’t. Now the format is enforced while the reply is generated. The model only chooses the values.

Tomás’s router call in Python

On the Claude API, structured outputs is one request field: output_config.format, holding the type json_schema and your schema ^{1Structured outputsAnthropic, Claude API documentation}. Two rules apply to every object in the schema. List its keys in required, and set "additionalProperties": false so the model can’t add keys of its own.

Here is the router call. Install the SDK with pip install anthropic and put your key in ANTHROPIC_API_KEY.

import json

import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY from the environment

TICKET_SCHEMA = {
    "type": "object",
    "properties": {
        "category": {
            "type": "string",
            "enum": ["delivery", "returns", "billing", "product", "account", "other"],
        },
        "urgency": {"type": "string", "enum": ["low", "normal", "high"]},
        "order_number": {"type": ["string", "null"]},  # null when none is given
    },
    "required": ["category", "urgency", "order_number"],
    "additionalProperties": False,
}

SYSTEM = (
    "You sort new support tickets for an online shop. Choose the category and urgency. "
    "Copy the order number exactly as the customer wrote it, or use null if there is none."
)


def route_fields(first_message: str) -> dict[str, str | None]:
    response = client.messages.create(
        model="claude-haiku-5-5",
        max_tokens=2000,  # thinking counts toward this too
        system=SYSTEM,
        messages=[{"role": "user", "content": first_message}],
        output_config={"format": {"type": "json_schema", "schema": TICKET_SCHEMA}},
    )
    # A refusal or a cut-off reply may not match the schema.
    if response.stop_reason != "end_turn":
        raise RuntimeError(f"No usable fields: stop_reason={response.stop_reason}")
    text = next(block.text for block in response.content if block.type == "text")
    return json.loads(text)


if __name__ == "__main__":
    print(route_fields("Hi, my order KS-20417 still hasn't arrived. It's a gift for Saturday!"))

The JSON arrives as a string inside the text block, so json.loads is still needed. A thinking block may come before it, which is why the code picks the block by type; Calling a model API explains content blocks and stop_reason. Here is one example response, shortened; the values can differ between runs:

{
  "content": [
    {"type": "thinking", "thinking": "", "signature": "Eq8B..."},
    {"type": "text", "text": "{\"category\": \"delivery\", \"urgency\": \"high\", \"order_number\": \"KS-20417\"}"}
  ],
  "stop_reason": "end_turn"
}

The API also adds its own instructions describing the format, which count as input tokens ^{1Structured outputsAnthropic, Claude API documentation}. The first request with a new schema is slower while the grammar compiles. The compiled grammar is then cached for 24 hours after its last use ^{1Structured outputsAnthropic, Claude API documentation}.

What one routing call costs

The task is one routed ticket, which takes one call. This estimate leaves out retries and any second call. As of October 2026, Claude Haiku 5.5 costs $0.10 per million input tokens and $0.50 per million output tokens, for prompts up to 100,000 tokens ^{3PricingAnthropic, Claude API documentation}. Thinking tokens are billed as output.

usage.output_tokens_details.thinking_tokens reports the real thinking on each call. Log it, because thinking is most of this bill.

Try it yourself

This curl call routes a ticket that has no order number. It costs at most about $0.0005, even if the model uses all 1,000 output tokens. If you don’t have an API key, skip it.

curl https://api.anthropic.com/v1/messages \
  -H "content-type: application/json" \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2026-10-21" \
  -d '{"model": "claude-haiku-5-5", "max_tokens": 1000,
  "system": "Sort this support ticket. Use null for an order number the customer did not give.",
  "messages": [{"role": "user", "content": "Hi, the lid of my new pan cracked the first time I used it. Can I get a replacement?"}],
  "output_config": {"format": {"type": "json_schema", "schema": {"type": "object",
    "properties": {"category": {"type": "string", "enum": ["delivery", "returns", "billing", "product", "account", "other"]},
      "urgency": {"type": "string", "enum": ["low", "normal", "high"]},
      "order_number": {"type": ["string", "null"]}},
    "required": ["category", "urgency", "order_number"], "additionalProperties": false}}}}'

One run returned this text block, shortened; yours may differ:

{"type": "text", "text": "{\"category\": \"product\", \"urgency\": \"normal\", \"order_number\": null}"}

What to look for:

• The text field is a string that holds JSON, not a JSON object.
• order_number is null, not a guess.
• Change ["string", "null"] to "string" and run it again. Now the model must write some string, and you’ll see what it chooses.

Designing the schema the router needs

The schema is the contract between the model and the router’s code. Each field should match something the code does with it.

An enum is a list of the only values a field may take ^{4JSON Schema Validation: A Vocabulary for Structural Validation of JSON (draft 2020-12)Austin Wright, Henry Andrews, Ben Hutton, Greg Dennis, JSON Schema, 2022}. Without one, “the category” is whatever words the model picks.

The order number raises a harder question. Many customers don’t give one.

Keep the schema small. Tomás was tempted to add the customer’s mood, their language and a one-line summary. Each extra field costs output tokens and some thinking, and gives the model another value to get wrong. It also makes the grammar larger; a schema that is too complex returns a 400 error ^{1Structured outputsAnthropic, Claude API documentation}. A field earns its place only if code acts on it.

Valid JSON can still be wrong

Structured outputs guarantee the shape: valid JSON, every required key, the right types and only allowed enum values ^{1Structured outputsAnthropic, Claude API documentation}. They don’t guarantee that the values are true.

Two cases break even the shape ^{1Structured outputsAnthropic, Claude API documentation}. If the model refuses, stop_reason is refusal, and the reply may not match the schema. If the reply hits max_tokens, it stops in the middle of the object. That is why the router checks stop_reason before json.loads.

Here is how a valid reply goes wrong. Femi writes: “I returned order KS-18842 last month. Now KS-20417 is late.” One run returned "order_number": "KS-18842". The JSON is valid and the type is right. The router attaches the old order. Priya reads that it was delivered and returned, and tells Femi his refund is on its way. Femi was asking about a different parcel.

So the router’s code still checks the values it can check:

• The order number exists for this shop and this customer’s email. If it doesn’t, treat it as null.
• other is counted. A rising share means a missing queue or a prompt problem.
• Unsorted stays as a fallback. Any call that fails, or any value that fails a check, still reaches a person.

The SDK can also build the schema from a Pydantic model and check the reply for you; the next lesson covers it. To measure how often values are wrong across many tickets, see Code-based checks.

Strict tool use: the other route

Tools are functions your code offers the model, such as lookup_order; Function calling teaches them. Each tool definition has a name and an input_schema. Set "strict": true on it, and the API guarantees two things: the tool call’s input matches the schema, and its name is one of your tools ^{2Strict tool useAnthropic, Claude API documentation}. The input arrives already parsed, in a tool_use block. Choose strict tool use when the model is choosing an action to take. Choose output_config.format, as the router does, when the reply itself is the data. One request can use both ^{1Structured outputsAnthropic, Claude API documentation}.

Where this shows up

• Design: an invoice data extractor uses a larger schema, where many fields are often missing and must be null.
• Lab: turning messy emails into clean records builds an extractor on output_config.format.
• Design: an AI customer support platform routes every incoming ticket before anything else happens.

The router now always gets JSON it can read, but it trusts every value inside. Validating model output with Pydantic checks those values in code.

Common mistakes

• Reading content[0].text. The first block may be thinking. Pick the block whose type is text.
• Parsing before checking stop_reason. A refusal or a cut-off reply may not match the schema. Parse only when stop_reason is end_turn.
• A required field with no way to say “none”. The model must then invent a value. Allow null.
• Treating valid JSON as correct JSON. The schema fixes the shape. Check the values your code can check.
• Adding fields nobody uses. Each one costs tokens and is one more value to trust.

Check your understanding

Summary