Self-Hosting a 1.5TB Model: The 2026 Cost Reality Check

Self-hosting a trillion-parameter model in 2026 is no longer a question of whether the weights are available — Kimi K3’s are, on Hugging Face, released July 27, 2026 — but whether you can afford the machine underneath them. The repository is 1,560,998,984,390 bytes across 118 files, roughly 1.56TB to pull down, and Moonshot’s own deployment guidance points at 64 or more accelerators. Open weights are free. The hardware is not.

There is a tell buried in the model card. On a Hugging Face API pull taken July 27, 2026, K3 shows 2,850 downloads against 5,637 likes — about 1.98 likes for every download. Both counters move constantly, so treat that as a point-in-time snapshot rather than a stable statistic. But the direction is the story: roughly twice as many people have applauded this release as have pulled it. Approval is cheap. A terabyte-and-a-half of high-bandwidth memory is not.

This guide works the arithmetic end to end. What 1.56TB on disk turns into once it is resident on GPUs, why a bare memory fit is not a deployment, what 64 accelerators cost per month at three providers’ own published rates, and — the number most cost writeups skip — how many tokens per month you need to be pushing before self-hosting beats simply paying $3.00 per million input tokens and $15.00 per million output tokens for the hosted API.

Start with the only number that is exactly knowable. Summing the siblings array of the Hugging Face API response for moonshotai/Kimi-K3 on July 27, 2026 gives 1,560,998,984,390 bytes across 118 files. In decimal units that is ≈1.56TB. In the binary units your operating system’s disk tools actually report, the same figure is ≈1.42TiB. Both are correct; they are the same bytes described in two different counting systems, and the mismatch between them is a reliable source of confusion when someone quotes a smaller-sounding number back at you.

The same API response reports a separate, independent figure: safetensors metadata totalling 2,779,931,837,184 parameters — about 2.78 trillion. That number arrives via a completely different field than the file-size sum, which makes it a genuine cross-check rather than a restatement.

The distinction that matters for budgeting is between download size and resident size. The 1.56TB figure is what crosses your network and lands on disk, including tokenizer files, configs, index shards and metadata. The roughly 1.4TB figure is what has to occupy accelerator memory once the MXFP4-packed weights are loaded and serving. You pay for the first once, in bandwidth and staging storage. You pay for the second every hour the cluster is up.

Moonshot’s own K3 blog states that the model applies quantization-aware training from the SFT stage onward, using MXFP4 weights with MXFP8 activations, for broad hardware compatibility. That is the vendor-confirmed part. What follows is our own reconciliation of the Hugging Face metadata against the confirmed repo total — useful for intuition, but explicitly an estimate rather than something Moonshot has published.

The safetensors metadata splits those 2.78T parameters across three stored dtypes: 2,722,740,830,208 tagged U8, 57,179,884,544 tagged BF16, and 11,122,432 tagged F32. The U8 tag is a storage-container label, not a logical bit-width — two 4-bit MXFP4 values pack into each byte. Reading it that way, the packed 4-bit block lands near 1.36TB rather than 2.72TB, the BF16 components at roughly 114GB (57.18B parameters × 2 bytes), and the MXFP4 block-scale factors at roughly 85GB (one shared scale byte per 32-element block).

Those three components sum to about 1.56TB — which is the point of running the exercise. An estimate that reconciles to the independently-confirmed repo total, via a different API field, is worth more than an estimate that does not. It is still arithmetic we performed, not a breakdown Moonshot released.

The format itself is not proprietary. MXFP4 and MXFP8 are defined in the OCP Microscaling Formats specification, co-developed by AMD, Arm, Intel, Meta, Microsoft, Nvidia and Qualcomm. The block-level detail below is taken from the alliance’s companion paper, Microscaling Data Formats for Deep Learning (arXiv 2310.10537): a 32-element block size with a shared 8-bit exponent (E8M0) scale factor per block, and an E2M1 layout for MXFP4 elements — one sign bit, two exponent bits, one mantissa bit. A 32-element MXFP4 block therefore occupies 136 bits including its shared scale, or 4.25 effective bits per weight; the MXFP8 equivalent occupies 264 bits. That extra quarter-bit per weight is exactly the ~85GB of scale-factor overhead in the chart above.

Quantization research published through 2026 suggests MXFP4 can be close to lossless on very large mixture-of-experts models when it is paired with quantization-aware training, with degradation more pronounced on smaller models — while also documenting real error sources, including scale-bias rounding and truncation of very small values, that need deliberate mitigation. That literature is general; it did not test K3 specifically. Treat it as a reason to run your own evaluation rather than as evidence that the quantization is free.

Here is the most seductive coincidence in this entire release. Nvidia’s B200 product page lists total memory of 1.4TB for an 8-GPU HGX B200 node — implying roughly 180GB of HBM3e per GPU, a figure corroborated by Fireworks listing its rental SKU as “B200 180 GB.” K3’s resident weights are also roughly 1.4TB. The arithmetic looks like a perfect fit, and it is a trap.

Filling a node’s memory with weights leaves nothing for the KV cache, nothing for activations, and nothing for the framework and runtime overhead every real deployment carries. Eight B200s at 180GB give 1,440GB against roughly 1,400GB of weights: a 40GB margin, under 3%, before a single request is served. For a model whose entire selling point involves long context — and therefore a large KV cache — that margin is not a tight fit. It is a non-starter.

The table below separates the two questions most coverage conflates: how few GPUs can physically hold the weights, and what the vendor actually recommends for serving. We compute the first column ourselves from published per-GPU memory; the last column is Moonshot’s own recommendation.

Read the H200 row twice. The bare fit is ten GPUs. The vendor recommendation is sixty-four. That is not a rounding difference — it is a 6.4× multiple on your largest recurring cost line, and it is the single biggest reason self-hosting estimates built on “does it fit in VRAM” arithmetic come in wildly under. If you are still sizing hardware for smaller models, our hardware price-bracket guide for running models locally covers the tiers below this one.

The most important sentence for anyone sizing a K3 deployment is not in a benchmark table or a spec sheet. It is in Moonshot’s own model blog, and it is a deployment recommendation rather than a requirement — which is precisely why it is easy to skip past.

The reasoning is bandwidth, not capacity. Moonshot’s own framing ties inference efficiency to larger high-bandwidth communication domains, and credits its attention and prefill-cache design with letting the company serve K3 at a competitive token price despite the model’s scale and long context. Read that as an admission worth taking seriously: the vendor is telling you that its own economics depend on a communication topology most self-hosters will not reproduce.

This is the asymmetry at the heart of the open-weights bargain. You get the same parameters Moonshot has. You do not get Moonshot’s interconnect, its serving stack, its cache-hit rate across millions of concurrent requests, or its ability to amortise idle capacity across an entire customer base. The weights are the cheap part of the transaction, and they are the only part that transfers for free.

If you have not yet decided whether K3 is the right model at all, settle that before you price hardware — start with the benchmark and hallucination picture, and check what actually shipped and under what licence. The K3 licence carries commercial-use restrictions; confirm they fit your deployment before you rent anything.

Three providers publish per-GPU or per-node H200 rates clearly enough to build arithmetic on. All three figures below were read from the provider’s own public pricing page on July 27, 2026, and all three assume 64 GPUs running continuously for a 730-hour month.

Modal lists serverless H200 capacity at $0.001261/second, explicitly per-GPU, which works out to about $4.54/GPU-hour. Fireworks AI lists on-demand dedicated H100 80GB and H200 141GB at $7.00/GPU-hour, billed per-second with scale-to-zero when idle, alongside B200 180GB at $10.00 and B300 288GB at $12.00. Google Cloud lists the a3-ultragpu-8g machine type — eight H200s — at $84.806908/hour on demand, which we round to $84.81/hour in every calculation below and which is about $10.60 per GPU-hour once you divide out the node.

One provider is deliberately missing from this comparison. Together AI publishes on-demand HGX H100 and HGX B200 rates, but its pricing page does not unambiguously state whether those hourly figures are per-GPU or per full 8-GPU node. That is an eightfold ambiguity, which is far too large to paper over in a cost table, so we have excluded those rates from every calculation here rather than guess at the unit.

K3’s hosted API is priced at $3.00 per million input tokens on a cache miss, $0.30 per million on a cache hit, and $15.00 per million output tokens. That rate is first-party — Moonshot’s own international pricing, matched by hosting partners and by trade-press coverage citing the Kimi API docs — not a reseller’s markup. It also sits level with Claude Sonnet 5’s $3/$15, which quietly undercuts the assumption that an open Chinese model is automatically the cheap option.

The break-even model is two lines of arithmetic, stated here so every cell below is auditable and recomputable when prices move:

• Blended API price ($/1M tokens) = (input share × $3.00) + (output share × $15.00)
• Break-even volume (M tokens/month) = monthly self-hosting infrastructure cost ÷ blended API price

Cache hits are excluded from the blend deliberately — they make the API cheaper, which pushes break-even higher still. If a meaningful share of your traffic is cacheable, the case for self-hosting weakens further; our write-up on cache-first agent economics walks through how large that effect can get at $0.30 per million.

The friendliest cell in that table is roughly 16.8 billion tokens per month — long-output agentic workloads on the cheapest infrastructure tier. The most demanding is nearly 118 billion. To make the middle row concrete: 23.6 billion tokens a month is about 786 million tokens every single day, sustained, with the cluster never idle. That is a serious platform business, not a team rolling out an internal assistant.

The forward-looking view matters more than any individual cell. Our read is that hosted inference prices are falling faster than GPU rental rates, and K3’s own $3/$15 placement — level with a closed frontier model rather than far beneath it — suggests the era of open Chinese models priced as loss leaders is closing. If API rates keep drifting down while GPU-hours stay sticky, every number in this table drifts upward. Anyone modelling a self-hosting payback period over twenty-four months should assume the target moves against them.

Every figure in the previous section is a floor, and it is worth being explicit about why. The table prices 64 accelerators running flat out, all month, with no other line items. Real deployments carry four categories of cost that the arithmetic never sees — and all four move break-even in the same direction.

Stripping out the enthusiasm, four situations account for nearly every defensible self-hosting decision at this model scale. Three of them are not about cost at all.

The strategic move for most organisations is not a binary. It is to route by workload — cheap models for bulk classification, the frontier for hard reasoning, self-hosted weights only where sovereignty or volume genuinely demands it. Our cost-optimisation routing playbook sets out that discipline, and the current API pricing tracker keeps the comparison rates current. If you would rather have someone build and defend the model with you, our AI transformation engagements start with exactly this kind of build-versus-buy analysis, and our web and platform development team handles the serving architecture when the answer comes back self-host.