Bonsai 2 Squeezes a 27B Model Into 5.9GB, and SWE-bench Falls From 80.6 to 60.8

AI 요약

• PrismML shrank a 27B model to 5.9GB and shipped it under Apache 2.0.
• The 20-benchmark average holds at 98.2%, but SWE-bench Verified falls from 80.6 to 60.8.
• Stock llama.cpp and Ollama will not load the files.

Running a large model on a laptop has usually meant giving up one of two things: the size of the model, or the quality of what it answers with. Ternary Bonsai 2 27B, released September 17 by PrismML of Pasadena, California, recalculates that exchange rate. It takes Alibaba's open-weight Qwen3.8 27B and brings it from 53.8GB down to 5.9GB, roughly 9.3 times smaller. The license is Apache 2.0 and the weights are a free download on Hugging Face.

The number the announcement leads with is 98.2%. Averaged across a 20-benchmark suite, the compressed model scores 83.9 against the original's 85.4. Hacker News gave the release 557 points and 185 comments on day one. Break that average apart by task, though, and the answer depends entirely on what you were planning to hand the model.

What improved over the first generation, and what got dropped

PrismML already shipped the first Bonsai 27B on July 14. That build used Qwen3.6 27B as its base and retained 94.6% of the original's score. The second generation swaps the base to Qwen3.8 27B and lifts retention to 98.2%. The file size stayed put; the model inside it got better.

The compression stores each weight as one of three values: -1, 0, or +1, which is where the "ternary" name comes from. A typical model spends 16 bits per weight. Three values need 1.6 bits in theory, and once you add a separate FP16 scaling factor per group of weights, the real cost lands at 1.75 bits per weight. Two files ship:

One build from the first generation is missing here. The July release paired the ternary 5.9GB file with a 1-bit 3.9GB build that stored each weight as one of two values, and the selling point for that build was that it ran on iPhone and iPad. Bonsai 2 ships only the ternary build. If a phone is the target, the July 1-bit build remains the only option. At 5.93GB with vision processing layered in, this release does not fit the same devices.

98.2% is an average

Broken out by task, retention falls as low as 76% in places. These are PrismML's own measurements as published in the announcement.

Math gives up 0.49 points and coding 0.59. Instruction following comes in 1.41 points above the original. Judged on these rows alone, "near lossless" is not an overstatement.

One area moves nowhere near the average: tasks that chain many steps and call tools over and over. Editing code, running the tests, reading the failure, and editing again all land here.

For a second reference point: on the same Terminal-Bench 2.1, DeepSeek V4.1 Flash scored 90.6. A 52.8 from a 5.9GB model on your own machine is not a score that replaces a cloud coding agent.

That does not make the compression technique a failure. Take the same base model and shrink it with the ultra-low-bit scheme llama.cpp ships by default, and the resulting IQ2_XXS build uses 7.3GB to reach an aggregate of only 75.2. Bonsai 2 reaches 83.9 at 5.93GB: smaller by 1.4GB and higher by 8.7 points. Accounting for compression during training beats downloading a finished model and mechanically shrinking it afterward.

The graph PrismML published alongside the announcement converts how many gigabytes it takes to reach a given score into a metric of its own. Ternary Bonsai 2 27B lands at 0.444, the IQ2_XXS build of the same base at 0.275, and the FP16 original at 0.051. First place on that chart is not Bonsai 2 but the 1-bit Bonsai 27B from July, at 0.530. That is the build this release dropped.

One caveat belongs on all of it. The 98.2% figure is PrismML measuring its own benchmark suite on its own harness, and nobody outside the company has reproduced it yet. Hacker News commenters argued the suite was picked without long-context or long-horizon tasks. Alongside that are run reports of the model looping in place until it exhausted its output budget, and of it trailing the original on tasks that require recall.

Can you use it today

The download carries no conditions. No account, no approval, no waitlist, and no geographic gating on the Hugging Face repository, so the weights pull the same way from Singapore, the EU, or anywhere else. The actual gate is the runtime.

The model card is blunt about it: stock llama.cpp does not run these files. Hacker News comments returned repeatedly to the changes not being upstreamed. Keeping your existing local-inference setup and swapping in a new file is not an option here.

On throughput, PrismML measures 143 tokens per second on an RTX 5090 and 46.8 on an Apple M5 Max. Reports from Hacker News users spread wider across hardware: 40.6 tokens per second on an RTX 3070, 26.5 on an RTX 3060, 14.2 on an M1 Pro. On a 6GB card, only 44 of 65 layers fit and throughput collapsed to 0.67 tokens per second. A 5.9GB file does not mean 6GB of VRAM is enough, because the KV cache holding the conversation takes its own memory on top.

If you already build llama.cpp yourself and have a GPU with 8GB or more of VRAM, or an Apple silicon Mac, pull PrismML's fork and start with single-shot work such as summarizing local documents or explaining a piece of code. That is where this model sits closest to the original. If you were instead looking for somewhere to run a coding agent locally, 60.8 on SWE-bench is not yet a reason to take the cloud model out of that slot. Two things are worth checking back on: when PrismML lands its changes upstream in llama.cpp, and when a 1-bit Bonsai 2 build arrives to bring this down to phones and 8GB devices.