llama.cpp b11149

Use it to run, test and deploy local inference of LLMs on desktops, servers, mobile devices and embedded platforms across multiple GPU/CPU backends.

Availability

Pre-release b11714, released 17 Oct

Pricing model

open-source

Deployment

Builds listed for multiple operating systems and hardware backends

Privacy

local inference (run on user hardware)

Licence

Not established

• Adds a -b/--backend option to test-llama-archs for testing a specific backend ^[1]

  tests: add -b/--backend option to test-llama-archs for testing a specific backend ( #27372 )

• Release includes attestations and many compiled assets for multiple OS/architectures/backends listed on the release page ^[1]
• Includes platform-specific setup guidance for Snapdragon targets (CPU, Adreno GPU, Hexagon NPU) ^[1]

  Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide

• Provides builds and CI artifacts for macOS (arm64 and x64), iOS, Linux (CPU, Vulkan, CUDA, ROCm, OpenVINO, SYCL), Windows (CPU, CUDA, Vulkan, OpenVINO, SYCL, ROCm) and Android ^[1]
• Release published as pre-release tag b11149 on GitHub on 23 Sep 2026 with assets
• The release adds the llama_batch_ext API. ^[2]

  llama: add llama_batch_ext ( #24669 )

• Builds are listed for macOS, iOS, Linux, Android, Windows, and openEuler. ^{[1] [2]}
• The release is marked as a pre-release. ^{[1] [2]}
• The batch API supports token, embedding, and state inputs. ^[2]
• The release adds support for sparse FA on SYCL. ^[3]
• b11179 is marked as a pre-release. ^[3]
• The release lists builds for macOS, iOS, Linux, Android, and Windows. ^{[1] [3]}
• The listed builds include CPU and accelerator-specific variants.
• The server now permits chunked prefill for RANK pooling when the context is causal. ^[4]

  can_split() now permits chunked prefill for RANK pooling when the

• The change addresses long-document and multimodal reranking for causal models including Qwen3 and Qwen3-VL. ^[4]
• Release b11223 is marked as a pre-release. ^[4]
• The release lists builds for macOS, iOS, Linux, Android, Windows, and openEuler. ^{[1] [4]}
• The server supports typed vision, audio, and video content input for the /v1/embeddings endpoint. ^[5]

  server : support typed content (vision/audio/video) input for /v1/embeddings endpoint ( #29556 )

• The GitHub release is marked Pre-release. ^{[1] [5]}
• Bare content arrays are rejected with a migration message. ^[5]

  Bare content arrays (the unwrapped shape) are

• Legacy formats continue to work through tokenize_input_prompts. ^[5]
• Wrapped content arrays are accepted for multimodal embedding requests. ^[5]

  Accept the OpenAI-style wrapped content array format for multimodal

• The server supports multimodal input for Qwen3-VL-Embedding. ^[5]

  server : support multimodal input for /v1/embeddings (Qwen3-VL-Embedding)

• The release adds MiMo support for dflash conversion and feature extraction. ^[6]
• The release lists builds for macOS, Linux, Android, and Windows. ^{[1] [6]}
• The GitHub release is marked Pre-release. ^{[1] [6]}
• The release lists Android arm64 builds for Snapdragon CPU, Adreno GPU and Hexagon NPU. ^{[1] [7]}

  Android arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide

• The release lists builds for macOS, iOS, Linux, Android, Windows and openEuler. ^{[1] [7]}
• Release b11345 is marked as a pre-release. ^[7]
• The release adds q2_k and q3_k quant type support for Hexagon. ^[7]

  hexagon: add q2_k and q3_k quant type support ( #29717 )

• The release lists Linux arm64 builds for Snapdragon CPU, Adreno GPU and Hexagon NPU. ^{[1] [7]}

  Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide

• Rejection sampling supports grammar-constrained requests. ^[8]

  Support grammar-constrained requests in rejection sampling

• Probabilistic draft sampling is controlled by a flag whose default value is greedy. ^[8]

  Fallback to argmax sampling for grammar-constrained requests and adding flag for enabling probabilistic draft sampling.

• The release is marked pre-release and was released on 03 Oct. ^[8]
• The release adds probabilistic sampling for simple draft and MTP. ^[8]

  spec : add probabilistic sampling for simple draft and MTP ( #27694 )

• The target verifies drafts by rejection sampling. ^[8]

  Make the drafter probabilistic and the target verify by rejection sampling

• The Clef support is text-only. ^[9]

  model: add support for clef decision model (text-only) ( #29831 )

• The GitHub release is marked Pre-release. ^{[1] [9]}
• The release lists platform builds for macOS, iOS, Linux, Android, and Windows. ^{[1] [9]}
• The release adds initial support for the Clef decision model. ^[9]

  model: add support for clef decision model (text-only) ( #29831 )

• b11400 is marked as a pre-release. ^[10]
• The release lists Linux builds for CPU, Vulkan, CUDA, ROCm, OpenVINO and SYCL configurations. ^{[1] [10]}
• The release lists Android and Windows builds. ^{[1] [10]}
• The release adds support for both embeddings and raw tokens in a batch. ^[10]
• The release lists builds for macOS Apple Silicon and Intel, and an iOS XCFramework. ^{[1] [10]}
• b11408 adds a Q8_0 IME1 matrix kernel for the SpacemiT X60. ^[11]

  ggml-cpu : add Q8_0 IME1 matrix kernel for SpacemiT X60 ( #28479 )

• The release marks Q8_0 as supported on the X60. ^[11]
• The release reports Qwen2.5-0.5B-Instruct Q8_0 tested on a Milk-V Jupiter. ^[11]
• b11408 is marked as a pre-release on GitHub. ^[11]
• On that test, pp128 throughput increased from 10.70 to 93.87 tokens per second. ^[11]
• b11507 adds support for MoE cache over multiple GPUs. ^[12]

  llama: support MoE cache over multiple GPUs ( #30112 )

• b11507 is marked as a pre-release. ^[12]
• The release lists macOS, iOS, Linux, Android, and Windows builds. ^{[1] [12]}
• Linux and Windows builds are listed for CUDA 12 and CUDA 13.
• The release extends Gemma-4 GPU decode through OpenCL to E2B. ^[13]

  opencl: enable Gemma-4 E4B GPU decode

• The release improves OpenCL support for FA and DK512 for Gemma-4. ^[13]
• The release lists builds for macOS, iOS, Linux, Android, Windows, and openEuler. ^{[1] [13]}
• The release is marked Pre-release. ^{[1] [13]}
• The release enables Gemma-4 E4B GPU decode through OpenCL. ^[13]

  opencl: enable Gemma-4 E4B GPU decode

• The release adds inline RoPE construction in Mellum. ^[14]

  model : inline rope construction in Mellum

• The release adds MTP support for Mellum. ^[14]

  model : MTP support for Mellum2.1 ( #30612 )

• The release adds MTP support for Mellum2.1. ^[14]

  model : MTP support for Mellum2.1 ( #30612 )

• b11695 is marked as a pre-release. ^[14]
• Builds are listed for macOS/iOS, Linux, Android, and Windows. ^{[1] [14]}
• b11714 is marked as a pre-release. ^[15]
• The Discover Models surface includes a curated catalog list with search and a details pane. ^[15]

  The Discover surface: the curated-catalog list with its search and

• The Discover interface includes download options and a standalone download progress bar. ^[15]

  download options, and the standalone download progress bar that replaces

• Paused downloads can be resumed, and incomplete downloads are remembered across restarts. ^[15]
• The Discover setting gates browsing and downloading Hugging Face GGUF models. ^[15]

  The flag gates browsing and downloading HuggingFace GGUF models, so it is

• DeepSeek adds web search for apps and updates its Flash model Daily brief 2026-10-17
• Generate editable slides and dashboards from a prompt Daily brief 2026-10-11
• Build and govern workplace agents with North 2 Daily brief 2026-10-05
• Microsoft introduces FrogNano-4B-2609 for repository-level coding workflows Daily brief 2026-10-03
• Google introduces Gemini 4 Argon reasoning model Daily brief 2026-10-02
• ElevenLabs introduces two text-to-speech models Daily brief 2026-09-28