Structured Output That Holds: LLM JSON in Production

LLM structured output finally grew up in 2026: Anthropic, OpenAI, and Google all now enforce JSON Schema at the sampling level, using grammar-constrained decoding rather than polite prompting. A schema that the vendor accepts will produce output that parses. The catch is the word “accepts” — each provider honors a different subset of JSON Schema, and the differences hide exactly where production pipelines break.

The stakes are practical. Extraction pipelines, agent tool calls, CRM enrichment jobs, and report generators all depend on the model returning an object your code can trust. Teams that standardized on one Zod schema and assumed it would travel across providers are discovering that Anthropic rejects numeric bounds, OpenAI rejects root-level unions, and Gemini declines to publish its nesting ceiling at all.

This playbook covers each vendor's current structured-output surface, a side-by-side support matrix of what each schema subset actually allows, how streaming partial JSON works, and how the Vercel AI SDK 6 layer — our own house stack — wraps all of it, including the deprecation path away from generateObject.

For most of the past three years, reliable JSON from an LLM was a defensive engineering exercise: prompt hard, parse leniently, strip markdown fences, retry on failure. That era is effectively over. Each of the three major providers now offers a mode in which the model is not merely asked to follow your schema — its token sampling is constrained so that schema-invalid output cannot be emitted. Anthropic names the technique directly: grammar-constrained sampling.

The three surfaces look different but do the same job. What actually differs — and what this guide is really about — is the slice of JSON Schema each vendor's constraint engine understands.

The strategic consequence is subtle but important: once validity is guaranteed at the decoding layer, the engineering effort moves up a level. Your JSON will parse. Whether it parses into something correct — the right fields populated, cross-field invariants holding, business rules respected — is still your job, and each vendor's schema-subset gaps determine how much of that job the constraint engine can carry for you.

Anthropic's forced-tool-call control is tool_choice, with four modes: auto (the default when tools are present), any (a tool must be used, no particular one), tool (forces one specific tool), and none. Schema enforcement is a separate switch — set "strict": true at the top level of the tool definition, alongside name, description, and input_schema. The documented pattern for a fully guaranteed structured response combines the two: tool_choice of type any plus a strict tool, so a tool is always called and its input always validates.

The feature is available on a wide model range — including Claude Opus 5, Sonnet 5, and Haiku 4.5 — and Anthropic also documents a dedicated non-tool structured-outputs path that reuses the same constraint pipeline, with compiled schemas cached for up to 24 hours after last use.

Two interaction caveats matter in production. With manual extended thinking enabled, tool_choice modes any and tool error out — only auto and none work; adaptive thinking (the default on Opus 5) does support forced tool use. And Claude Mythos Preview does not support forced tool use at all — any or tool return a 400 error on that model.

The subset is where teams get surprised. Objects must set additionalProperties: false. Array minItems accepts only the values 0 or 1. Numeric constraints (minimum, maximum, multipleOf) and string-length constraints (minLength, maxLength) are not supported at all. Recursive schemas and external HTTP $ref are out. What you do get: enum and const, anyOf and allOf (though the allOf-plus-$ref combination is unsupported), internal $ref and definitions, default, pattern, and ten string format values — date-time, time, date, duration, email, hostname, uri, ipv4, ipv6, and uuid.

If your Zod schema says "an array of 3 to 5 tags, each 2 to 40 characters, score between 0 and 100," none of those bounds reach Anthropic's constraint engine. The output will be a valid array of strings and a number — the ranges are your validation layer's problem.

OpenAI's Structured Outputs with strict: true currently rides on the Responses API as text: { format: { type: "json_schema", strict: true, schema } }. Its two signature requirements shape how you design schemas: every object must set additionalProperties to false, and every field must appear in required. There are no truly optional fields in strict mode — anything conceptually optional has to be typed as nullable instead. For a full walkthrough of the OpenAI side, see our OpenAI structured-outputs complete guide.

The unsupported-keyword list is the opposite shape from Anthropic's. OpenAI accepts the constraint keywords Anthropic rejects — numeric minimum/maximum/multipleOf and unrestricted array minItems/maxItems — alongside pattern, string format, and enum, which both vendors support. What it rejects is composition: root-level anyOf (no discriminated union at the top of your schema), allOf, not, dependentRequired, and if/then/else.

Strict mode also has published scale ceilings — the numbers below are hard limits, not guidance.

Here is the finding most existing content gets wrong: Gemini's current structured-output parameter is response_format, set on the Interactions API as response_format: { type: "text", mime_type: "application/json", schema }. The classic generateContent + generationConfig.responseSchema shape that fills most tutorials — and older codebases — is not what today's canonical docs page teaches. The older field may still work for back-compatibility, but new integrations should target response_format.

Gemini's documented schema subset covers the core types (including nullable via "type": ["string", "null"]), object-level properties/required/additionalProperties, array items/prefixItems/minItems/maxItems, numeric minimum/maximum, and string enum/format. Google's own SDKs also accept Pydantic models in Python or Zod schemas in JavaScript and convert them to the JSON Schema the API expects. Two notable extras: structured outputs stream as valid partial JSON (more in section 06), and combining schema-constrained output with built-in tools — Google Search grounding, URL Context, Code Execution, File Search, Function Calling — is scoped to Gemini 3-series models, with the docs example using gemini-3.1-pro-preview.

The limitation language is the honest tell. Google states plainly that “Not all JSON Schema features are supported” and that “Very large or deeply nested schemas may be rejected” — with no numeric nesting or size ceiling disclosed. Where OpenAI gives you hard numbers to design against, Gemini gives you a warning and lets you discover the edge at request time.

The table below assembles each vendor's documented schema subset side by side — the comparison most “they all support JSON Schema now” posts skip. Every cell is sourced from the vendors’ own structured-output documentation as of this post's August 14, 2026 snapshot.

Read as a whole, the matrix shows three different philosophies. Anthropic constrains structure tightly but delegates value-range enforcement entirely to your code. OpenAI enforces the most constraint keywords but bans the composition patterns — discriminated unions above all — that typed codebases lean on. Gemini sits in the middle with the least explicit contract: a reasonable subset, softly-documented edges, and no hard numbers. The practical takeaway is that the intersection of all three subsets is the only safely portable schema language: plain objects, required fields, nullable-instead-of-optional, enums, and no bounds you actually depend on.

Streaming and strict schemas coexist better than they used to, but the guarantees differ by vendor and even by SDK. Gemini's docs confirm streaming support directly: “The streamed chunks are valid partial JSON strings that can be concatenated to form the final JSON object” — consumed via stream: true and step.delta events. That means a UI can render a progressively-filling object without waiting for the final token.

On the Anthropic side, the one concrete piece of SDK guidance we can cite is scoped to the Java SDK: structured responses there must be fully accumulated (via a message accumulator) before JSON deserialization — incremental parsing is the caller's responsibility. That guidance is explicitly per-language-binding; don't assume it describes the TypeScript or Python SDKs.

In practice, most application teams shouldn't hand-roll partial JSON handling at all — the Vercel AI SDK's streamObject (and its successor pattern) hands you a typed partial-object stream, plus an elementStream for arrays, and handles chunk concatenation and partial parsing for you across providers. That is the right altitude for product code; raw delta-event handling is for infrastructure layers.

The Vercel AI SDK is the abstraction most Next.js teams — ours included; this site runs ai@^6.0.193 — use to avoid coding against three vendor surfaces directly. You hand it a Zod schema, and the SDK translates to whichever provider mechanism is underneath. Field-level .describe() calls double as inline prompting hints to the model:

import { generateObject } from "ai";
import { z } from "zod";

const invoiceSchema = z.object({
  vendor: z.string().describe("Supplier name as printed on the invoice"),
  totalEur: z.number().describe("Grand total in EUR"),
  lineItems: z.array(
    z.object({
      description: z.string(),
      amountEur: z.number(),
    })
  ),
});

const { object } = await generateObject({
  model, // any AI SDK provider model instance
  schema: invoiceSchema,
  prompt: "Extract the invoice fields from the document text below. ...",
});
// object is fully typed as z.infer<typeof invoiceSchema>

Streaming is the same shape with progressive delivery — the partial-object stream is where the SDK earns its keep:

import { streamObject } from "ai";

const { partialObjectStream } = streamObject({
  model,
  schema: invoiceSchema,
  prompt: "Extract the invoice fields...",
});

for await (const partial of partialObjectStream) {
  // typed, progressively-filled partial object — safe to render
}

Now the caveat: both functions are deprecated in AI SDK 6 — softly. They still exist and still work, but the forward path is generateText / streamText with an output parameter: Output.object({ schema }), Output.array({ element }), Output.choice({ options }), Output.json(), or Output.text(). An automated codemod (npx @ai-sdk/codemod upgrade) migrates existing call sites. The correct framing is “legacy, migrate when convenient” — not broken, not urgent. Our AI SDK 5-to-6 migration playbook covers the wider upgrade, and the AI SDK 6 deep dive goes further into the new tool-calling and streaming surfaces.

Error handling is where structured-output code most often stays naive. The object functions throw NoObjectGeneratedError when the response can't be parsed or fails schema validation, and NoOutputGeneratedError when the final step doesn't finish with a stop reason — both preserve the raw text, response, usage, and cause for debugging. Catch them specifically, log the preserved fields, and make the retry decision explicit rather than wrapping everything in a generic try/catch. Teams migrating off LangChain's output parsers will find this error surface considerably more debuggable than parser-chain failures.

Grammar constraints remove one failure class; they do not remove the need for a validation boundary. Here is how we structure the decisions on client builds.

The layer that ties this together is validate-then-retry. Even with grammar constraints on, parse every response through the same Zod schema you sent — because the constraint engines cannot enforce everything your schema means. Anthropic can't enforce your numeric ranges. OpenAI can't enforce your top-level union. None of them can enforce cross-field invariants like “end date after start date.” Zod catches those post-hoc, and a failed parse becomes a bounded retry with the validation errors appended to the prompt. Schema conformance also belongs in your evaluation suite — our guide to agent eval frameworks covers testing agent outputs systematically.

Looking forward, we expect convergence pressure on these subsets — the divergence is a migration tax on every multi-provider team, and vendors know it. But convergence is not here yet: none of the three documents a shared subset today. Until that changes, the durable posture is to treat each provider's schema subset as a versioned API surface: documented in your codebase, tested in CI, and owned by someone. This is exactly the kind of reliability engineering we build into client systems through our AI transformation engagements.