AI Inference Providers: Q2 2026 Pricing Matrix

By Q2 2026, the serverless inference market for open-weight models has consolidated around seven providers. Pricing on the same model spreads 6× across the field; P50 latency spreads 5-7×; throughput on specialty hardware (Groq LPU, Cerebras wafer-scale) spreads up to 10× over commodity H100 endpoints. The matrix matters because the wrong default costs more than the engineering work to switch.

This post compiles the per-token pricing, P50 latency, and output throughput across the seven providers and twelve popular open-weight models — Llama 4 70B / 405B, Qwen 3 72B / 235B-MoE, DeepSeek V4-Pro / V4-Flash, Mistral Large 2, Command-R+, Mixtral 8x22B, plus Phi-5 and Granite Code at smaller scale. Numbers are taken from public pricing pages, Artificial Analysis benchmarks, and direct Apr 2026 testing.

The seven providers in this matrix occupy three positioning bands. Price-leader (Together, Fireworks) compete on per-token cost and broad model coverage. Performance-leader (Groq, Cerebras) compete on throughput and time-to-first-token. Coverage-leader (Replicate, OctoAI) compete on model breadth and custom-fine-tune support. Anyscale Endpoints sits across bands — competitive on price, enterprise-strong on contracts.

The cleanest cross-provider comparison is on a single popular open-weight model — Llama 4 70B is the 2026 reference because every provider in this matrix hosts it and the architecture (GQA + 70B dense) is identical across endpoints. The 6× spread below is real; the question is what comes with each tier.

The pattern: prices reflect what you're buying. Together batch tier saves 75% per token but accepts 60-minute SLA. Groq LPU charges 5× more but delivers 5-7× throughput. The matrix makes the trade explicit instead of implicit.

Cost is one axis; latency is the other. The two move independently — cheap providers can have low latency (Together on-demand) and expensive providers can have high latency (Cerebras at low utilization). The numbers below are P50 measurements from Apr 2026 on Llama 4 70B output decode.

Below is the practitioner's read on each provider — strengths, gotchas, and the kind of workload they fit. Pricing is the entry point; operating fit is the deeper question.

Most production deployments end up using two or three providers, not one. The pattern that wins: a price-leader for steady-state volume, a speed-leader for streaming/critical paths, and a coverage-leader as a fallback for niche models. Below is the workload-class decision.

• Listed price ≠ paid price. Every provider has volume tiers and reserved-capacity discounts. Listed pricing is the starting point; expect 25-40% discount on $50K+/month steady-state commitments. Negotiate before scaling.
• Capacity tightens on hot model launches. When a new frontier model ships (Llama 4 launch, DeepSeek V4 launch), the cheapest providers run out of capacity for several days. Maintain a fallback provider for the first 2-3 weeks after any major launch.
• Tail latency P99 isn't advertised.P50 numbers are visible on every provider's page; P95 / P99 tail-latency under load is not. For production deployments, run a 24-hour load test against P99 before locking in.
• Region availability is uneven. US regions are fully covered by every provider; EU coverage varies (Anyscale and Together strongest); APAC is patchy outside Singapore and Tokyo. Map your traffic geography to provider regions before committing.
• Custom-model hosting has a cold-start surcharge. If you fine-tune a model and host it on Replicate, OctoAI, or Fireworks, expect 2-5× the per-token cost of a popular hosted model — capacity isn't pre-provisioned for long-tail customers. Bake this into the fine-tune ROI math.