On September 16, 2026 the engineer Rohan Bansal published a detailed account of teaching a 4-billion-parameter model to do one thing: tell a Postgres database how to run a query faster than it would on its own. After a first round of training on about 500 example runs from a frontier model and a second round of reinforcement learning against four database containers on his desk, the small model cut the summed running time of a standard 113-query benchmark by 44.7%. The base model had been unable to produce a usable plan for 99 of those queries. The whole run cost about $1,200.
This post is not about databases. It is about the question a founder or engineering lead should ask before paying a frontier model to do the same repetitive job ten thousand times a day: is this a task where the answer is cheap to check? If it is, a small model trained for it can win on cost, speed and control. If it is not, the frontier model stays. The experiment is one person's work on one benchmark, and its author says so; the figures below are his, from the write-up, read on September 18, 2026.
- 01The task was chosen because its output is easy to grade.A query plan either runs faster or it does not, and the time can be measured. That single, checkable axis is what reinforcement learning needs, and it is the test to apply to any business task.
- 02The result is real but narrow.44.7% lower summed latency and a 1.81x geometric-mean speed-up across the 113 queries of the Join Order Benchmark, with up to three attempts per query. One database, one workload, one author.
- 03The bill was small; the setup was not trivial.About $800 of rented H100 time over 95 hours, about $400 of API calls to generate teacher runs, and a home measurement rig that took real engineering to make the reward signal trustworthy.
- 04Route by trait, not by fashion.Checkable, high-volume, narrow: train small. Open-ended, low-volume, judgement-heavy: prompt a frontier model. Structured or typed data with an existing specialist: use the specialist.
01 — The resultThe result, and what it measured
Postgres decides how to run a query by estimating how many rows each step will produce, and Bansal opens with the research showing that these estimates, and the plans built on them, have been leaving performance on the table for a decade. His model does not replace the database's planner. It writes hints, through an open-source extension that reads structured comments above a query, that steer the planner toward a different join order, scan type or degree of parallelism. The measure of success is the running time of the resulting plan against the default one.
Lower summed latency
Total running time of all 113 Join Order Benchmark queries with the model's plans, against Postgres defaults. The author's practical measure of what a batch of analytics would save.
Geometric-mean speed-up
Each query weighted equally, so one lucky plan cannot carry the score. The title's '81% faster' is this figure. Measured with three rollouts per query, up to five candidates each, so the best of 15 counts.
Queries the untrained model could not plan
Before training, only 15 of 113 runs produced a valid, compilable hint, and nine of those were identical to the default plan. The training taught the format first, then the skill.
The two headline numbers say different things and the author keeps them apart, which is worth copying. The workload figure is what an analyst running the whole batch would feel. The geometric mean is the evidence that the model learned something general rather than stumbling onto one dramatic plan; a value above one means the typical query got faster. The benchmark itself comes from the 2015 paper by Leis and colleagues that first asked how good query optimisers really are, revisited by the same group in 2025, and the model was trained on a different, larger query set from the same film database with no overlapping query shapes, so the 113 test queries were new to it.
02 — The traitThe trait that made it work
Choosing the best order to join tables is a famously hard problem; checking whether a chosen plan is fast is easy. Bansal builds the whole experiment on that asymmetry. Reinforcement learning needs a reward, and a reward that can be measured in milliseconds, on every attempt, without a human, is the best kind there is. The model was allowed several candidate plans per query, each one was run and timed, and the faster ones were reinforced.
Language models are particularly good at learning how to do tasks with easily verifiable outputs.Rohan Bansal, 'Training a 4B model to produce 81% faster query plans than Postgres', September 16, 2026
That sentence is the test for any business task. Ask three questions of the job you are paying a large model to do. Can a machine grade the output without a person, in seconds? Is the grade a number rather than an opinion? And is the job repeated often enough that a few percent per run adds up? Query plans, routing decisions, extraction into a fixed schema, classification against a checkable rule and code that has a test suite all pass. Writing a customer email, summarising a meeting or deciding a strategy does not, because nobody can score those in milliseconds, and those stay with the model that reasons best.
03 — The methodWhat the experiment took
The write-up is long because most of the work was not training. It was building an environment in which the reward could be trusted, and that is the part a business copying the idea will underestimate.
| Step | What was built | Why it mattered |
|---|---|---|
| A harness with six tools | A small agent loop that lets the model inspect tables, statistics and the default plan, then submit candidate plans as structured objects compiled into hints. | Gives the model something to act on and a fixed output format to grade; the untrained model mostly failed here. |
| A quiet measurement rig | Four Postgres containers on a home machine, each query warmed until its cache counters settled and then run 20 times, with a 5% tie zone so equal plans score equal. | Without it, timing noise fooled the reward about 40% of the time on one query; raising the database cache cut that error roughly fourfold. |
| Teacher runs from a frontier model | About 500 trajectories generated by OpenAI's GPT-6 Astra through the same harness, converted to the small model's format and used for supervised fine-tuning with low-rank adapters. | Taught the 4B model the harness's language before asking it to be good; the author thinks RL alone might have worked with many more attempts. |
| Reinforcement learning across two machines | Inference and training on a rented two-GPU H100 node; database measurements on the desk rig, linked over a private network, with a custom variant of GRPO for noisy scores. | The rented machine was too noisy for timing, so measurement stayed local. The split is the practical shape of most small-model RL projects. |
- Rented GPU nodeTwo H100s, about 95 hours
- ~$800
- Teacher trajectoriesAPI calls to generate about 500 runs
- ~$400
- Total stated costExcluding home electricity, about $9 a day at full load
- $1,200Author's figure, September 16, 2026
What the model learned is also instructive. In the final evaluation most searches began by inspecting statistics or the default plan before proposing anything, most used all five allowed attempts, and the gains came from three moves: rewriting the join order, fixing a single scan choice, and turning on parallelism. That is a narrow, legible skill, which is what a small model is for.
04 — The decisionRouting your own tasks
Three kinds of one-job model have crossed this blog in the last fortnight, and together they make a routing table. The rows below are ours; the examples are linked.
We covered the second row in our post on Jev and the third in our TabPFN guide. The first row is this post. What the three share is that each replaced a general model with something that knows one thing, and each was justified by a check that could be run without a human.
05 — The limitsWhat one experiment does not show
The author is careful, and the reader should be too. This is one person's work, on one benchmark, against one database with a fixed dataset, and its author lists comparisons he has not yet run, including whether a structured search over hints without any language model would do as well. A frontier model through the same harness was still stronger per query on the small slice he tested, which is why it served as the teacher. And the result is a research result about plans, not a production feature: a business that wanted this would still need to decide how hints are applied, monitored and rolled back when the data changes.
None of that weakens the lesson. It bounds it. The claim that survives is that a task with a cheap, honest check can be learned by a small model for a small sum, and that the check is the expensive part to build. The claim that does not survive is that small models beat large ones in general; the author says the opposite, that the distillation from a frontier model is proof large models are not going anywhere.
Before training anything, write the function that scores an output in under a second with no person involved. If you cannot write it, the task belongs to a frontier model and an evaluation set. If you can, the scorer is your reward, your test suite and your monitoring, and the small model is the easy part.
06 — For a businessCost and skills for a business
The experiment cost about what a company spends on a single frontier-model integration workshop, and the author notes that the training infrastructure he built by hand is now available as hosted services from several providers. The skills it needed are the honest part of the bill: someone who can build an agent harness, someone who can make a measurement trustworthy, and someone who understands enough reinforcement learning to handle a noisy reward. That is a small team for a month, not a research lab, and it is the same shape of work our guide to fine-tuning for business use cases describes for the supervised half.
The return depends on volume. A task run a hundred times a day does not pay back the checker. A task run a hundred thousand times a day, at frontier-model prices, pays it back in weeks, and gains a model you own, can run on your own hardware, and can retrain when the data moves. If you have a task like that and want the checker and the environment built before any training budget is spent, our AI transformation service starts with the scoring function.
07 — Next stepThe check is the product; the model is the easy part
List your ten most repeated AI calls and score each for checkability
Take the ten prompts your systems send most often. For each, write down whether a machine could grade the answer in a second, and how many times a day it runs. The ones that score yes on both are candidates for a model trained for that job alone, at a cost this experiment puts in the low thousands. The rest stay where they are, and you will have spent an afternoon learning which is which.