Needle 3

Developers can use it to run tool calls, extract structured data, or generate embeddings locally on supported devices.

Availability

Try it in the browser; Python package and source are on GitHub

Pricing model

Not established

Deployment

On-device; platform engines and air-gapped setup are documented

Privacy

All three jobs run on the device.

Licence

Not established

• Each argument is derived from a span of the request, and a required argument without a span or default withholds the call. ^[1]

  Each argument is copied from a span of the request, and the model writes that derivation as its reasoning line before the call.

• Ranges, enums, patterns, lengths, and item counts are compiled into the decode grammar. ^[1]

  Ranges, enums, patterns, lengths and item counts are compiled into the decode grammar, so an invalid value is unrepresentable rather than merely discouraged.

• Above five tools, retrieval selects the five closest tools for the request. ^[1]

  Above that, retrieval engages: every schema is embedded once, each request is embedded, and only the five closest tools enter the context with the grammar rebuilt over them.

• Needle 3 supports tool calls, structured extraction, and text embeddings. ^[2]

  - **Tool calls**: given the functions your app exposes, Needle picks the right ones and fills every argument from what the user said.

• The model uses a byte-level grammar compiled from schemas to constrain output. ^[2]

  The weights are compressed to CQ2-bit with Cactus Quants; a byte-level grammar compiled from your schemas constrains every token, and every response carries a calibrated confidence score from a learned head.

• The repo includes the 20-layer needle3.cact model and a needle3.safetensors checkpoint to fine-tune. ^[2]

  The repo holds the 20-layer `needle3.cact`, the `needle3.safetensors` checkpoint to fine-tune, and an engine per platform.

• Needle 3 runs on-device, with an engine provided for each platform folder. ^[2]
• The model can be tried in a browser, and the Python package can be installed with pip install cactus-needle. ^[2]