Ternary Bonsai 2 2B

Use it locally to generate text and answer questions about images through llama.cpp.

Availability

GGUF files and running instructions are provided

Pricing model

Free

Deployment

llama.cpp on CUDA, Metal, or CPU

Privacy

Not established

Licence

Apache 2.0

• The model supports reasoning, visual understanding, and tool-driven workflows.
• The model is downloadable starting October 15, 2026. ^[2]

  Developers, researchers, and other users can download the new Bonsai 2 models under the Apache 2.0 license for free starting October 15, 2026.

• The model is optimized for low-latency inference on consumer-grade CPUs and edge GPUs. ^[2]

  They have been designed for seamless integration with existing AI workflows and are optimized for low-latency inference on consumer-grade CPUs, and edge GPUs.

• The model accepts vision input through the official vision tower. ^[3]

  - **Vision input** through the official vision tower, carried unrotated in FP16.

• The model is downloadable from Hugging Face.
• An OpenAI-compatible server can be run with mlx-vlm. ^[3]

  For an OpenAI-compatible server, run `python -m mlx_vlm.server --model bonsai2-2b-mlx` and send the sampling settings in each request.

• Its weights are ternary values stored in MLX affine 2-bit format. ^[3]

  - **Ternary language weights** stored in MLX's 2-bit grouped format (group 128).

• The model accepts vision input through a separate mmproj pack. ^[4]

  - **Vision input** through a separate mmproj pack (Q8_0, or BF16 for reference precision), loaded only when an image arrives.

• The model has a 262,144-token context length. ^{[3] [4]}

  | Context length | 262,144 tokens (inherited from the base model) |

• Three GGUF language-model packings are provided: PTQ1_0, PQ2_0, and Q2_0. ^[4]

  - **Three GGUF packings** of the same weights, consumed directly by custom kernels and never expanded back to FP16: **PTQ1_0** (dense trits, 0.43 GB), **PQ2_0** (2-bit slots, 0.52 GB) and **Q2_0** (0.55 GB).

• The model runs with the PrismML llama.cpp build on CUDA, Metal, and CPU. ^[4]

  A 2B-class vision-language model with ternary transformer weights, for llama.cpp (CUDA, Metal, CPU).

• The model is licensed under Apache 2.0. ^{[3] [4]}

• Bespoke-Nimble-9B-v2 leads 12 AI developments on 15 October 2026 Daily brief 2026-10-15