Why a 770B-Parameter Model Can Be cheap to Run

Active parameters — the share of a model's weights that actually runs for each token — is the single most useful number on a 2026 model spec sheet, and the most misread. Tencent's Hy4 preview, released August 28, is 770 billion parameters in total, yet each token routes through only 49 billion of them. That one ratio is why a model this large can list under $1 per million input tokens — $0.834 on OpenRouter, listed August 28, 2026.

The stakes are practical. Four open-weight mixture-of-experts (MoE) releases landed in the last week of August 2026 alone — Hy4 preview, GLM-5.3's open weights, GLM-5.3-Flash, and Qwen3.8-Flash-Next — and every one of them leads its announcement with parameter counts. Buyers who read those counts as a proxy for price, quality, or hardware cost make expensive mistakes in both directions: overpaying for capacity they cannot use, or under-provisioning memory for weights that must be resident whether or not they fire.

This reference puts the week's releases side by side — total parameters, active parameters, context, licence, price, each cell labelled by source and date — and then walks through what the activation number predicts, what it does not, and how to read a spec sheet that vendors do not fill in consistently. For how the experts inside these models get chosen in the first place, see our MoE architecture comparison — this post deliberately stays on the buyer's side of that line.

The definition fits in one sentence, and Hugging Face's official MoE guide states it cleanly: model capacity depends on total parameters, but inference speed depends on active parameters. In an MoE model, most weights sit in routed "experts," and each token runs through only a selected few — in Hy4 preview's case, 8 of 256 routed experts plus one shared expert per token, which is how 770B total becomes 49B active.

Everything the number predicts follows from that. First, per-token serving cost: the floating-point work per token scales with the weights that actually run, so a 49B-active model prices its compute like a much smaller dense model, not like a 770B one. Second, throughput: fewer active weights per token means more tokens per second on the same hardware. Third, the hot working set: the weights that must move through the accelerator's fastest memory each step track the active count, which is what makes serving economical at batch scale.

Just as important is the list of things the number does not predict: output quality, context window, licence terms, and whether you can afford to self-host. Sections 03 through 07 take those one at a time, with August 2026 releases as the evidence.

No post we found lists these releases side by side on total parameters, active parameters, context, licence and dated price. The table below assembles those five attributes for the late-August 2026 MoE releases plus a DeepSeek V4 Pro reference row, from each vendor's Hugging Face model card, official announcement, and live pricing surface — every price labelled by where it applies and when it was listed.

* GLM-5.3's Hugging Face card does not publish an active-parameter count; secondary sources report a figure around 40B, but the card itself is silent. That silence is a data point in its own right — a major vendor shipping 753B open weights without stating the one number this post argues matters most. † Qwen3.8-Flash-Next's 125B is the backbone figure; NVIDIA's developer blog counts the same checkpoint at 176B once a 51B N-gram embedding table and a 4B multi-token-prediction module are included. "Total parameters" is not counted uniformly across vendors — the table uses backbone totals throughout because that is how Hy4 and GLM report theirs.

Two more caveats keep this table honest. Qwen3.8-Flash-Next (the open weights) and Qwen3.8-Flash (the hosted API) are different, separately documented artifacts that share most of a name — the hosted model's internals are undisclosed, so its parameter cells stay marked Undisclosed rather than inheriting the sibling's. The weights and the hosted API are not the same Qwen3.8-Flash — our sister post covers that split in full. And context-window cells reflect a specific serving surface: GLM-5.3-Flash's 1,310,720 tokens is one OpenRouter provider's ceiling, not a fixed property of the model.

If active parameters drove price directly, two models with the same activation count would cost about the same to use. August 2026 supplies a clean natural experiment: DeepSeek V4 Pro and Tencent's Hy4 preview both activate exactly 49 billion parameters per token. On OpenRouter in August 2026, V4 Pro lists at $0.435 per million input tokens; Hy4 preview, listed August 28, 2026, at $0.834 — roughly 1.9x — and the output gap is wider still, $0.87 against $2.501, about 2.9x. Identical activation, nearly triple the output rate.

Widen the frame and the correlation gets worse. GLM-5.3 — a 753B-total model in the same size class, whose card does not publish an activation count — lists at $1.40 in and $4.40 out on Z.ai's API in August 2026: more than 3x V4 Pro's input rate and about 5x its output rate. Per active billion, the two models where the number is published diverge sharply — V4 Pro's input rate works out to roughly $0.009 per million tokens per active billion (0.435 ÷ 49), Hy4's to roughly $0.017 (0.834 ÷ 49). Same numerator logic, same denominator, double the unit price.

The interpretation is the point of this post: the active-parameter count sets a vendor's cost floor, not its price. What a listed rate actually reflects is serving efficiency, margin strategy, subsidy appetite, and how badly the vendor wants developer attention that month.

The flip side of the activation discount is the part spec sheets whisper: every routed expert must be resident in memory whether or not it fires. Compute scales with active parameters; memory scales with total parameters. Hugging Face's original MoE explainer makes the canonical case — a sparse model built from eight roughly 7-billion-parameter experts needs enough VRAM to hold a dense 47B model, even though each token's compute resembles a ~12B dense model.

Scale that mistake up to the August releases and it stops being academic. Hy4 preview's 49B active count reads like something a modest cluster could serve — but 770B total parameters (plus a 10B-parameter multi-token-prediction layer the card lists on top of the headline figure) must sit in memory before the first token routes anywhere. The gap between the number that sets your speed and the number that sets your hardware bill can exceed an order of magnitude for the same model. We priced this out end to end for a 1.5TB checkpoint in the real hardware cost of holding a trillion-parameter model — the cost model there applies unchanged to every row in the table above, so this post will not rebuild it.

The strongest evidence that activation count says nothing about output quality comes, usefully, from a vendor arguing its own book. Alibaba's model card for Qwen3.8-Flash-Next — a 6B-active model — publishes a benchmark table pitting it directly against Claude Opus 4.6. The comparison is Alibaba's own: Alibaba chose the benchmarks, ran the evaluations, and picked the frontier model to stand next to. It is not an independent audit, and the numbers below should be read with that label attached.

Discount the table however heavily you like — vendor-selected benchmarks deserve heavy discounting — and the structural point survives: a lab was willing to publish, under its own name, a comparison in which a 6B-active model meets or beats a frontier model across six chosen benchmarks. If activation count predicted quality, that table could not exist even as marketing. Alibaba's own card also frames the model as "an experimental preview of the architecture that will underpin Qwen4" — a preview artifact, not a flagship, which makes the 6B activation figure more striking, not less.

Tencent, for its part, published its own internal evaluation of Hy4 preview against competing models — a vendor-run, vendor-graded exercise with no external audit, which is why this reference reports its existence rather than its numbers. The pattern to internalize: quality claims travel with evaluation methodology, and neither travels with the parameter table. Run your own evals on your own workload; read activation counts for cost, and only cost.

Nothing in a parameter count tells you what you are legally allowed to do with the weights, and August 2026 made that vivid: those four open-weight releases shipped under four different licence regimes, with no correlation to size in either direction.

Context windows are equally uncoupled from parameters. Hy4 preview's OpenRouter listing shows 1,048,576 tokens; 18B-active GLM-5.3-Flash shows an even larger 1,310,720 on one provider; and 6B-active Qwen3.8-Flash-Next is 262,144 native, reaching 1M only via a manual YaRN configuration step — while its hosted sibling gets 1M by default. Context is an engineering and serving decision, not a size dividend. For the licence side in depth, how we read licences across 30 open-weight models is the companion reference.

Even after you have the right model and the right numbers, "the price" is not one number — it is a surface and a date. GLM-5.3-Flash is the cleanest example in circulation, with three legitimate prices live simultaneously in the week of its launch.

The framing around that price is also worth labelling. Z.ai presents GLM-5.3-Flash as priced at roughly one-tenth of full GLM-5.3's API rate — arithmetic that roughly holds against list prices ($0.15/$0.50 versus $1.40/$4.40), but it is the vendor's own marketing comparison, not an audited ratio, and it quietly uses the post-promo list number. Meanwhile the hosted Qwen3.8-Flash lists at $0.15/$0.47 on OpenRouter in August 2026 — essentially GLM-5.3-Flash's list rate — despite the open sibling carrying one-third the active parameters (6B versus 18B). Activation counts do not set street prices even within the budget tier. GLM-5.3-Flash's launch pricing and promo window have their own write-up in our Ox Alpha reveal coverage.

Put the sections together and a reading order falls out — four questions, each answered by a different line of the spec sheet, none answered by the line vendors put in the headline.

Looking forward, the reporting gaps in this post are more likely to widen than close. GLM-5.3 shipped 753B open weights without publishing an activation count; Qwen's own checkpoint is quoted at two different totals depending on who is counting; hosted siblings disclose nothing at all. Expect spec sheets to get less comparable before they get more comparable, and expect price to keep decoupling from both numbers as serving stacks and subsidy strategies diverge.

For teams building on these models, the practical posture is the one this reference is built around: keep a dated, surface-labelled table of the models you actually route to, and re-verify it when a promo window closes or a licence changes. If you want help standing that discipline up — model selection, eval design, cost tracking across providers — that is exactly what our AI transformation engagements cover.