OpenAI-Compatible APIs: Build Provider-Agnostic AI Apps

OpenAI-compatible APIs became the connective tissue of the AI stack in 2026 — and Kimi K3's July 17 launch made the pattern unmissable. Moonshot's new flagship shipped OpenAI-SDK compatible on day one: pip install openai, point base_url at Moonshot's endpoint, set the model string to kimi-k3, done. No proprietary client library, no new wire format — the fifth major model line this year to launch speaking OpenAI's dialect from its first minute in public.

That convergence is genuinely good news for anyone building AI products: switching or hedging across vendors is now closer to a config change than a rewrite. But "compatible" is doing quiet, unglamorous work in that sentence. It reliably covers chat completions, tool calls, and streaming. It leaks — every time — on the newer surface area the original 2023-era spec never defined: reasoning deltas, thinking-effort parameters, sampling contracts, and structured-output dialects.

This guide maps where the compatibility actually holds and where it breaks, across five named vendors, at the parameter level. We cover the base-URL swap pattern with the stock OpenAI SDK, the newer dual-spec trend (vendors shipping OpenAI and Anthropic wire formats simultaneously), the Vercel AI SDK's compatible-provider layer, the Open Responses standardization effort, and a practical playbook for designing provider-agnostic apps around the gaps.

Moonshot announced K3 with a one-line launch post: "Kimi K3 is now live on Kimi.com, Kimi Work, Kimi Code, and the Kimi API. Open Weights by July 27, 2026." The API half of that sentence is the part that matters here. K3's official quickstart doesn't document a proprietary client — it documents the OpenAI Python SDK with a swapped base URL. That is the entire integration story, and it was true at minute zero of the launch. (Our full K3 launch breakdown covers the model itself — architecture, vendor benchmarks, pricing, and the July 27 open-weights promise.)

Zoom out and the pattern is unmistakable. DeepSeek's docs describe an OpenAI/Anthropic-compatible format as the API's native shape. Z.ai's GLM line exposes an OpenAI-compatible path as its direct API. Alibaba's DashScope serves Qwen through a dedicated compatible-mode endpoint. MiniMax follows the same playbook. Five major model lines, five independent decisions that the fastest route to developers is somebody else's API spec — the strongest kind of evidence that the OpenAI Chat Completions shape has become a de facto standard, even though no standards body ever blessed it.

Why did this convergence happen? Because the switching cost is the product. A vendor launching a frontier-class model in mid-2026 is selling into codebases that already speak Chat Completions — every agent framework, every eval harness, every internal tool. Shipping OpenAI-shaped means the eval starts today instead of after an SDK migration. For buyers, the same logic runs in reverse: every additional compatible vendor makes your existing client code more valuable and your incumbent provider less leveraged. Compatibility is a pricing strategy wearing an engineering costume.

The canonical pattern is a two-line change: swap the API key and the base URL, keep everything else. Against K3, per Moonshot's own quickstart, that looks like this:

# The base-URL swap — stock OpenAI SDK, Kimi K3 backend
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["MOONSHOT_API_KEY"],
    base_url="https://api.moonshot.ai/v1",
)

response = client.chat.completions.create(
    model="kimi-k3",
    messages=[
        {"role": "user", "content": "Summarize this contract."},
    ],
    # Note what's ABSENT: no temperature, no top_p.
    # K3 pins them (1.0 / 0.95) and asks callers to
    # omit sampling params rather than override them.
)

Within this pattern, three things work reliably across every provider in this post: standard chat completions (messages in, choices out), function/tool calling in the OpenAI tools array shape, and server-sent-event streaming. That is the durable core of the de facto standard — the 2023-era surface that every implementer copied faithfully because it was the whole spec at the time.

The catch is everything that happened to LLM APIs after 2023. Reasoning traces, thinking budgets, structured outputs, prompt caching, context-window flags — none of it existed when the Chat Completions shape froze into a convention. Every vendor bolted those features on independently, which is exactly where compatibility stops being a guarantee and starts being a per-provider audit. If you're wiring K3 into a real coding workflow, our hands-on Kimi Code setup guide walks the practical side of that audit.

K3 is the cleanest case study precisely because its compatibility story is otherwise so good. The endpoint speaks fluent Chat Completions — and then, at the parameter level, it quietly rewrites four contracts that OpenAI-SDK code tends to assume. All four are documented in Moonshot's own quickstart; none of them throw an obvious error if you get them wrong.

The pattern to internalize: these are not bugs, and they are not hostile. They are rational vendor decisions — pinned sampling because Moonshot tuned K3 for those values, always-on reasoning because that's the product's identity — expressed through a wire format that has no vocabulary for "this parameter exists but is fixed." The OpenAI shape can carry the request; it cannot carry the contract. That gap between syntax and semantics is where every "compatible" integration eventually needs provider-specific knowledge.

Most coverage treats "OpenAI-compatible" as a per-vendor yes/no. The table below is the version that actually matters when you're building a provider-agnostic client: five vendors side by side at the parameter level, each cell sourced from the vendor primary named in its Source column. To our knowledge, nobody has published this matrix as a single view before.

Read down the leaks column and a taxonomy emerges. Some leaks are parameter pinning (K3's sampling), some are shape divergence (DeepSeek's thinking object versus K3's max-only string versus OpenAI's own effort param), some are feature interlocks (DashScope's tools-versus-streaming exclusion), and some are string-encoded flags (GLM's [1m] suffix). None of these is visible in the wire format. All of them will find you in production if your abstraction assumed the OpenAI docs were the contract.

The under-covered trend inside the trend: vendors no longer pick a side. Z.ai ships an OpenAI-compatible direct API and an Anthropic-Messages-compatible surface for Claude Code simultaneously. Alibaba's DashScope mirrors it — an OpenAI-compatible mode plus a dedicated Anthropic-compatible endpoint for Qwen3.7-Max. DeepSeek claims both formats against one base URL. MiniMax reportedly follows the same pattern (we treat its exact URLs as unverified — see the table above). The GLM version of the Anthropic side is literally two environment variables:

# Point Claude Code at Z.ai's Anthropic-compatible surface
export ANTHROPIC_BASE_URL="https://api.z.ai/api/anthropic"
export ANTHROPIC_AUTH_TOKEN="<your-z-ai-key>"

# The leak, even here: GLM's 1M-token context window is
# unlocked by a bracket suffix bolted onto the model string:
#   glm-5.2[1m]
# — a vendor flag the Anthropic wire format has no field for.

Why it matters strategically: dual-spec vendors are betting that the market has standardized on exactly two client stacks — the OpenAI SDK and the Anthropic SDK (increasingly via agent tooling like Claude Code) — and that meeting both erases the last integration excuse. For buyers, it means a team standardized on either wire format can now hedge across nearly the whole open-weight field without touching client code. That is a materially stronger negotiating position than 2025, when second-sourcing usually meant maintaining two clients.

Structured outputs deserve their own section because the leak here starts at the source. OpenAI's canonical implementation requires "strict": true plus "additionalProperties": false on every schema object to guarantee schema-conformant JSON — and OpenAI's own docs concede the limits: "While Structured Outputs supports much of JSON Schema, some features are unavailable either for performance or technical reasons."

Sit with that for a second: even the reference implementation is a JSON Schema subset. Every "OpenAI-compatible" provider then picks its own subset of the subset, documents it in its own terms, and exposes it through the same-looking response_format field. Two providers can both truthfully claim structured-output support and disagree about which of your schemas will validate. This is the single most common place we see provider-agnostic apps fail silently: the request succeeds, the JSON parses, and a field your schema required simply is not enforced.

Two pieces of Vercel tooling formalize the base-URL-swap pattern into something maintainable. First, the AI SDK organizes providers as separate npm packages behind a shared language-model specification — switching models becomes a config change — and ships a dedicated @ai-sdk/openai-compatible package for everything without a first-party package. That factory covers LM Studio and NVIDIA NIM in the docs, and by extension the whole compatible field this post covers:

// AI SDK: any OpenAI-compatible endpoint as a provider
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { generateText } from "ai";

const kimi = createOpenAICompatible({
  name: "kimi",
  apiKey: process.env.MOONSHOT_API_KEY,
  baseURL: "https://api.moonshot.ai/v1",
  // Opt in explicitly — the SDK never auto-detects this:
  supportsStructuredOutputs: true,
});

const { text } = await generateText({
  model: kimi("kimi-k3"),
  prompt: "Draft the migration checklist.",
});

Second, Vercel's AI Gateway inverts the pattern: instead of your client visiting each provider's compatible endpoint, one OpenAI-compatible endpoint fronts all of them. Point an unmodified openai client at the Gateway and address any of its 40+ connected providers with prefixed model strings:

// One compatible endpoint, every provider behind it
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: "https://ai-gateway.vercel.sh/v1",
});

const completion = await client.chat.completions.create({
  model: "anthropic/claude-opus-4.8", // or "openai/gpt-5.5"
  messages: [{ role: "user", content: "..." }],
});

The Gateway charges no markup on tokens — including under bring-your-own-key — and supports automatic retries and model fallbacks to a different provider when the primary fails, which turns it into a resilience layer rather than a routing convenience. It also exposes a separate Anthropic-Messages- compatible surface alongside the Chat Completions one, so the dual-spec hedge from Section 05 works at the gateway tier too. We covered the Gateway's routing model from the creative-model angle in our Seedream 5 Pro routing post, and the AI SDK's provider layer shows up again in our MCP tool-drift security guide.

The ad-hoc nature of all this is exactly what a community specification launched on January 15, 2026 set out to fix. "Open Responses" is a vendor-neutral spec derived from OpenAI's newer Responses API — chosen over Chat Completions specifically because Responses was designed with the features of more recent models, such as reasoning traces, baked into the design. The launch partners are the plumbing of the ecosystem: OpenRouter, Hugging Face, LM Studio, vLLM, Ollama, and Vercel.

Willison's fuller framing names the disease this post has been cataloguing symptom by symptom: "There are dozens of slightly incompatible implementations of that not-quite-specification floating around already, it would be great to have a written spec accompanied by a compliance test suite." That is the Chat Completions ecosystem in one sentence — a not-quite-specification with no compliance suite, which is why the reality-check table in Section 04 has to exist at all.

Our forward read: Open Responses is the right idea arriving after the market already standardized on the wrong-but-good-enough shape. Voluntary specs win when the biggest implementers adopt them, and the launch-partner list is infrastructure players, not the model vendors themselves — as of mid-July, the five providers in our table still document Chat Completions compatibility as their primary surface. The likely path is coexistence: Chat Completions as the lingua franca for basic calls, Open Responses growing where reasoning traces and richer semantics matter. Plan for the messy middle, not for the clean convergence.

Everything above reduces to four working rules. They assume the thing this post has demonstrated: the wire format is shared, the contract is not.

The economic payoff of doing this well is real. List input pricing across the compatible field spans roughly 21x — from DeepSeek V4-Flash at $0.14 per million tokens to K3 at $3.00 — with cache-hit discounts that reshape the math again (K3 drops 90% to $0.30 on cache hits; V4-Flash falls to $0.0028). When switching is a config change, that spread is negotiating leverage you can actually use.

This is the engineering half of the argument we made in our second-source vendor-resilience playbook: hedging only works if the switching cost stays near zero, and compatible endpoints are what hold it there. The same logic runs through coding-agent cost comparisons across models — the models you can cheaply evaluate are the models that discipline your incumbent's pricing. If you want help designing this into your stack — capability manifests, gateway routing, provider evals on your own workloads — that's the core of our AI transformation engagements.