Ternary Bonsai 2 4B

Use it locally to generate text or respond to prompts about images.

Availability

GGUF files and setup instructions are provided; MLX weights are also linked

Pricing model

Free

Deployment

llama.cpp on CUDA, Metal, or CPU via the PrismML fork

Privacy

Not established

Licence

Apache 2.0

• The model supports reasoning, visual understanding, and tool-driven workflows. ^[1]
• The model is downloadable starting October 15, 2026. ^[2]

  Developers, researchers, and other users can download the new Bonsai 2 models under the Apache 2.0 license for free starting October 15, 2026.

• The model is optimized for low-latency inference on consumer-grade CPUs and edge GPUs. ^[2]

  They have been designed for seamless integration with existing AI workflows and are optimized for low-latency inference on consumer-grade CPUs, and edge GPUs.

• The model has a 0.95GB footprint. ^{[1] [2]}

  The models use Prism ML ’s ternary weight representation, which encodes model weights using three values: {−1, 0, +1}.

• The model accepts vision input through the official vision tower. ^[3]

  - **Vision input** through the official vision tower, carried unrotated in FP16.

• The model runs with MLX on Apple Silicon (Metal). ^[3]

  | Backends | MLX on Apple Silicon (Metal): mlx-vlm 0.7.8+ natively, or the loader in `runtime/` |

• The optional multi-token-prediction head is intended to help most on copy-heavy work such as code edits and rewrites. ^[3]

  It helps most on copy-heavy work such as code edits and rewrites; on general chat it is about as fast as plain decoding or slightly slower, so plain decoding stays the default.

• The model supports vision input through a separate mmproj pack. ^[4]

  - **Vision input** through a separate mmproj pack (Q8_0, or BF16 for reference precision), loaded only when an image arrives.

• The model has a 262,144-token context length inherited from its base model. ^{[3] [4]}

  | Context length | 262,144 tokens (inherited from the base model) |

• Three GGUF packings are available: PTQ1_0, PQ2_0, and Q2_0. ^[4]

  - **Three GGUF packings** of the same weights, consumed directly by custom kernels and never expanded back to FP16: **PTQ1_0** (dense trits, 0.95 GB), **PQ2_0** (2-bit slots, 1.15 GB) and **Q2_0** (1.21 GB).

• The PrismML llama.cpp fork runs every listed GGUF file on CUDA, Metal, and CPU.
• Upstream llama.cpp runs only the Q2_0 packing. ^[4]

  Upstream llama.cpp runs only `Q2_0` (recent `master`, which includes the Hadamard activation runtime); it rejects `PQ2_0` and `PTQ1_0` as unknown types.

• The model is licensed under Apache 2.0. ^{[3] [4]}

• Bespoke-Nimble-9B-v2 leads 12 AI developments on 15 October 2026 Daily brief 2026-10-15