Ternary Bonsai 2 shrinks a 27B model to 5.9GB

What this is about

PrismML released Ternary Bonsai 2 27B on September 18, 2026. The model is based on Qwen3.8 27B, but its smallest offered GGUF package occupies only 5.93GB instead of 53.80GB in FP16. The weights are available under Apache 2.0, allowing local and commercial use in principle.

This matters to developers because a model in this size class can now run on a laptop with 16GB of memory or on a single graphics card. File size is only part of the story: PrismML reports that the compressed model retains 98.2 percent of the uncompressed model's average score across 20 benchmarks.

What Ternary Bonsai 2 actually does

The 27.36 billion parameters remain. PrismML instead reduces most model weights to three possible values: minus one, zero, or plus one. Groups of 128 weights share an FP16 scale, while a Hadamard rotation is used to spread outliers before compression. The shipped PTQ1_0 format uses an effective 1.76 bits per weight.

The model accepts text and images, offers a documented context window of 262,144 tokens, and supports tool calls. Its GGUF build currently requires a PrismML fork of llama.cpp because the new PTQ1_0 and PQ2_0 formats are not supported upstream. An MLX package is also available for Apple silicon.

Why it matters

Local models can keep sensitive prompts on the user's device and avoid recurring API fees. Reducing the package from 53.80GB to 5.93GB therefore makes a meaningful difference to the hardware that can run it. PrismML reports 142.5 tokens per second on an RTX 5090 and 46.8 tokens per second on an M5 Max. These are vendor measurements, not independent reproductions.

The aggregate score of 83.9 versus 85.4 for Qwen3.8 27B sounds nearly lossless. Longer agent tasks show a weaker picture: the published evaluation gives Bonsai 2 a score of 52.8 on Terminal-Bench 2.1, compared with 69.7 for the full model. SWE-bench Verified shows 60.8 versus 80.6. Anyone planning a coding agent should inspect task-level results rather than relying on the average.

In plain language

Imagine a large toolbox in which every tool comes in its own heavy case. PrismML replaces many cases with three standard sizes. The box becomes much lighter but still holds the same tools. The difference may be invisible during a quick repair, yet small inaccuracies can surface several steps later during a complicated assembly job.

A practical example

A small software team wants to search internal documents locally and generate code suggestions. The original 53.80GB package does not fit comfortably on its existing machine with 16GB of shared memory. The 5.93GB build leaves room for the runtime, context, and application.

The team then evaluates 100 real tasks: 70 short questions, 20 image analyses, and ten multi-step repository changes. If the model performs well on short tasks but takes incorrect follow-up actions on four of the ten repository jobs, the compression is suitable for document search but not yet for unattended code changes.

Scope and limits

First, most capability, speed, and energy figures come from PrismML; no independent reproduction was available at research time. Second, the 98.2 percent average hides large differences between individual tasks, especially long agent workflows. Third, the smallest GGUF build does not run with standard llama.cpp and requires PrismML's fork and custom kernels.

The model is therefore not an automatic substitute for cloud models. Teams need to test their own data, hardware, and cost of errors. Apache 2.0 grants broad software rights, but it does not establish that every training source of the base model is legally unproblematic for every use.

SEO & GEO keywords

Ternary Bonsai 2 27B, PrismML, Qwen3.8 27B, ternary weights, local AI, model compression, GGUF, llama.cpp, MLX, Apache 2.0, open-weight model, laptop AI