AI DevelopmentDecision Matrix9 min readPublished

Should You Run Inkling 2 Mini on a 32 GB Mac or Use an API?

A Mac you already own can keep a repository task offline. Tinker can run the same model for a fraction of a cent on a small request. A failing test makes the trade-off concrete.

DA
Digital Applied
PublishedNovember 21, 2026
Read time9 min

A developer with a 32 GB Apple silicon Mac can run Inkling 2 Mini locally. Thinking Machines released the weights on November 20 and supports that memory configuration through Ollama. If the code can travel and the job is occasional, try the hosted model first. A request with 20,000 uncached input tokens and 2,000 output tokens costs $0.004 at Tinker’s listed sampling rates. The strongest reason to download roughly 20 GB is a task you need to finish offline or keep on the machine.

Consider a developer changing how a monthly report sorts two entries with the same date. One test fails. The assistant needs the ordering rule, a failing example and enough surrounding code to suggest a patch. It can do that work on either route. Where the model runs determines who receives the evidence, how much memory the editor has left, and how much work it takes to get an answer worth reviewing.

Key takeaways
  1. 01
    The supported minimum is 32 GB.Thinking Machines lists Apple silicon Macs with at least 32 GB. Its filmed offline demonstration used an M5 Pro MacBook Pro with 48 GB. The latter machine supplies no timing for the former.
  2. 02
    The checkpoint and the conversation share memory.The published checkpoint is about 20 GB. macOS, the model runtime, context, editor and tests also need memory. The model card’s million-token ceiling describes the model’s capability.
  3. 03
    Tinker charges for both sides of the request.At the November 21 direct sampling rates, $0.15 buys a million uncached input tokens and $0.50 buys a million output tokens. Count every turn needed to get an accepted change.
  4. 04
    Check the entire tool chain for offline work.The runtime, source files, dependencies and test fixtures must all be available on the Mac. A local model can still be paired with an editor or search tool that sends text elsewhere.

01 — The assignmentGive the assistant one failing test

Write the desired behaviour before opening either model. In this example the report sorts by date and, for a tie, by the entry’s existing identifier. Use two invented entries with equal dates and different identifiers. Feed them in reverse identifier order so the wrong implementation has a chance to fail. The output should be a proposed change to the comparison function, a regression test and a short account of why dates that differ still sort correctly.

This is a useful first task because the maintainer can tell when it is done. A plausible explanation alone earns little. Run the focused test against the original function, apply the suggested change in a scratch checkout and run the test again. Read the diff. An assistant that rewrites the fixture to match the existing behaviour can turn a test green while leaving the original defect in place. Use the same starting commit and acceptance conditions on each route.

Prepare a small packet

Collect the failing assertion, the comparison function, the input type and the caller that renders the report. Remove real records from the fixture. Save the packet locally so a second run starts with the same evidence.

02 — Memory budgetWhere the other gigabytes go

The lab’s model card describes a single checkpoint of about 20 GB. Subtract that rounded file size from the 32 GB printed on the box and the nominal gap is around 12 GB. File size and live allocation measure different things, so that subtraction cannot tell you how large a prompt will fit. The remaining demands include the operating system, model working buffers, the attention cache that grows with context, the IDE and the processes that actually run the tests. Thinking Machines model card

The parameter counts tell a similar story. Mini contains 26.4 billion parameters in total. The lab counts 3.4 billion active parameters and 5.4 billion effective activations per token because the model reuses a middle group of layers. Reusing a layer saves storing another copy of its weights while still doing computation. A download estimate based on just the active count misses the experts the runtime needs available as the conversation changes.

For this repository task, the critical moment may come after the first answer. You might have the editor, model, development server and test runner competing for memory just as the assistant reads a second file. Check memory pressure and swap in Activity Monitor before the request and again during the test. A Mac that gives a pleasant first reply with everything else closed may feel quite different during an ordinary working session. The Inkling 2 Small hardware guide covers the much larger checkpoint in this family.

03 — Local setupRead the context Ollama actually allocated

Ollama lists ollama run inkling-2-mini as the command to fetch and start its local package. Do the download while connected. With the model loaded, use ollama ps to inspect the running model and the context allocation. Record the model tag or ID alongside your Mac chip, installed memory and runtime version. Those details make a result repeatable when a colleague selects the same name in another editor. Ollama model library · Ollama context settings

Start the coding assistant against the local runtime with permission to read the scratch checkout. Ask it to explain the failure using the two sample rows and identify the file it would change. Then give it the test output and ask for the smallest patch that preserves the existing date ordering. If the assistant cannot run a test itself, run the test in your terminal and return only the failing lines. The model supplies text or tool requests. The surrounding application decides whether to execute them.

The card lists a context window of up to one million tokens. Raising the runtime allocation consumes memory, even when a longer window would be convenient. If the tool output grows large, keep the assertion and first relevant stack frame, then request another file by name. A full dependency tree or a long build log can occupy more space than the function you need to fix. Compare a larger window only when the smaller one drops evidence necessary to solve the task.

04 — Hosted setupWhat goes into a hosted request?

Thinking Machines released Mini for inference and fine-tuning on Tinker as well as for chat in the Tinker Playground. The playground is a quick way to hand the model a permitted example. A coding assistant that opens files and runs tests needs a client capable of reading a local file, sending the relevant text to Tinker, interpreting the reply and returning the result of the next tool action. Account for that integration if your editor has no suitable adapter. Thinking Machines release

Send the same ordering rule, synthetic rows and failing assertion. The source checkout can stay on your machine while the excerpt travels to the service. When the model asks for the caller, that second file becomes another disclosure if the client sends it. Test output may carry a full file path, a customer identifier or an environment value. Inspect what your adapter includes automatically, especially surrounding files and terminal logs. A prepared public fixture is enough to compare usefulness before authorizing private source.

For a quick experiment, paste the model’s proposed patch into a disposable checkout and run the tests locally. That arrangement requires very little plumbing and makes it obvious which snippets leave the device. An automated tool loop can reduce copying, but also changes the unit of work. One developer prompt can now trigger several model requests, each carrying prior messages, file excerpts or tool results. Track the whole exchange before quoting a price per fix.

05 — Token billA small request costs four tenths of a cent

Tinker lists $0.15 per million uncached prefill tokens, $0.015 per million cached prefill tokens and $0.50 per million sampled output tokens for Mini. Prefill is the input processed for a request. Sampling is the text the model generates. In the example, 20,000 uncached input tokens cost $0.003 and 2,000 output tokens cost $0.001. Add them to get $0.004, or four tenths of one US cent. These are direct Tinker sampling charges, with no training in the example.

Calculated using Tinker’s November 21 model prices. The examples assume all input is uncached and exclude tools, retries and engineering time.
Illustrative workloadUncached input tokensOutput tokensModel cost (USD)
One request20,0002,000$0.004
Three requests of that size60,0006,000$0.012
1,000 one-request tasks20,000,0002,000,000$4.00

A second turn often has a different size. If the whole first exchange is sent again, count the repeated input at the applicable rate on that turn. If a cache discount appears in the usage record, apply it only to the eligible tokens. A growing transcript and an extra round of tests can make three real requests cost more or less than the equal-sized example. Divide total cost by accepted changes when comparing assistants that take different numbers of attempts.

Now put the Mac in the same calculation. If it is already paid for and has spare capacity, the immediate costs are electricity and the time spent setting up and operating the runtime. If a developer needs half an hour to configure it, a thousand model calls at the example rate save just $4 before those local costs. Paying for new memory or a new workstation requires an even wider comparison. Privacy or dependable offline access can justify the work on their own. Token savings at this volume are a weak argument for buying a machine.

06 — Offline useWill the test run on the train?

Thinking Machines shows Mini building a flight simulator offline on an M5 Pro MacBook Pro with 48 GB. Its video is evidence of an offline demonstration on that configuration. Someone with 32 GB still needs to discover which context, tools and competing applications fit on their own Mac. There is no local throughput figure in this comparison. The time to an accepted patch also includes searching files, running tests and reading the result.

For travel, download the model, clone the repository, install its dependencies and start the test once before leaving. Then disconnect and repeat the entire exercise. A missing package or a test that calls a development service will reveal a dependency on the network even if the model itself answers perfectly. Configure search tools, remote embeddings, error reporting and cloud model fallback according to the project’s data rules. A sensitive repository deserves the same rehearsal. The on-device agent guide describes how local inference fits into that wider stack.

Ready offline
The synthetic fixture and test suite are local

An assistant can read the checkout, propose the change and inspect local test output. Keep a copy of the model package and the dependencies the test actually loads.

Choose this route for travel or repository policies that require local processing.
Needs a network
The test calls a remote reporting service

The model can reason about the function, but the full acceptance test needs connectivity. Supply a local fixture for the ordering rule, then schedule the integration test when connected.

Tell the reviewer which check still requires the service.

07 — Buying decisionKeep the route that produces a reviewable patch

Run the same starting commit in fresh checkouts. On the local route, write down the allocated context, peak memory pressure, time to first useful answer and time until the focused tests pass. On the hosted route, write down the files sent, total billed input and output, retry count and the same two times. On both routes inspect the patch for a hidden behaviour change, such as reordering entries whose dates differ. A faster response that changes the wrong comparison still needs another review.

Repeat with a task that genuinely uses your normal tool set. A one-function fix can flatter a tight context window. A cross-file rename with a failing integration test stresses it differently. An editor with several projects open changes the memory budget again. If the workload mostly consists of short confidential edits and the local tests run comfortably, keep the local route. If permitted tasks arrive in bursts or the Mac slows down during tests, keep an explicit hosted option for them. The guide to planning and coding with separate models explains how extra calls can change that arithmetic.

For a shared rollout, put the routing decision where developers can see it. Show which mode is active before the first file is read and let the user decline a switch when a local request grows too large. An AI integration brief can specify the one repository task, the files permitted outside the device and the conditions under which the team would pay to run it elsewhere. Those three facts are easier to review than a general preference for local or cloud AI.

Decision

Start with the machine and the repository you have

Try one representative task on the existing Mac if offline work or source control boundaries matter. Use a permitted fixture on Tinker to put a dollar figure beside the local result. Choose after you have a passing test, a readable patch and a record of what it took on each route.

Digital Applied

Choose where your coding assistant runs

Scope a pilot around repositories, test results and the costs of the actual tool loop.

Local inferenceHosted samplingSource access
Project brief

Set the acceptance test

  • →Which files can travel?
  • →What change must pass?
  • →How long can the developer wait?
Answers

Frequently asked questions

Thinking Machines names Apple silicon with at least 32 GB of memory for the supported Mac route. Check the chip as well as the memory in About This Mac when deciding whether this setup applies.

Digital Applied newsletter

Deep dives on AI, marketing and development.

Practical guides and fresh insights by email. No recycled takes.

Related dispatches

Related reading

AI Development

Run a 35B AI Model on a 24GB Mac: Two New Ways Compared

Edge0 streams a 35B mixture-of-experts model from SSD in under 3 GiB of RAM. Bonsai 2 shrinks a 27B model to 5.9 GB. Which to pick, and what to test first.

September 17, 2026 · 7 minRead
AI Development

Best Hardware to Run Local AI Models in 2026: Buyer Guide

Compare the RTX 5090, RTX PRO 6000, DGX Spark, M5 Max and Mac Studio for local LLMs by price, VRAM, memory bandwidth and real tokens per second.

June 28, 2026 · 12 minRead
AI Development

DGX Spark vs M5 Max vs RTX 6000: Local AI Showdown

DGX Spark, M5 Max and RTX PRO 6000 Blackwell compared for local AI: memory, bandwidth, FP4 compute, real tokens per second and current 2026 pricing.

June 28, 2026 · 11 minRead
AI Development

Gemini 4 Neon vs GPT-6.2 Sol vs Claude Sonnet 5.5 Pricing

Compare direct API prices, the 272K-token cutoff, cache costs and independent coding scores. See the bill for 100 jobs on Gemini 4 Neon, GPT-6.2 Sol and Sonnet.

November 19, 2026 · 9 minRead
AI Development

Kimi K2.8 Preview: What Changes for Kimi Code Users?

Check Kimi K2.8 Preview access, context limits and reasoning settings, then plan a coding trial that separates official specifications from performance claims.

September 13, 2026 · 7 minRead
AI Development

Building Games With Astra: A Practical Playtest Guide

Plan an Astra-assisted game around a playable loop, repeatable checks and human feedback. Use a practical playtest method before investing in more polish.

September 9, 2026 · 6 minRead
Google Search

See more Digital Applied analysis in your Google results by adding us as a preferred source.

Add as a preferred source