llama.cpp v0.5.0

Developers can use it to run supported models and serve them through its API, including setups that need multiple listening addresses or image inputs in function-call outputs.

Availability

Released 23 Sep 20:50

Pricing model

Not stated

Deployment

Builds listed for multiple operating systems and hardware backends

Privacy

Not stated

Licence

Not stated

• The release is marked as a pre-release. ^{[1] [16]}
• Builds are listed for macOS, iOS, Linux, Android, Windows, and openEuler. ^{[1] [16]}
• The batch API supports token, embedding, and state inputs. ^[1]
• The release adds the llama_batch_ext API. ^[1]

  llama: add llama_batch_ext ( #24669 )

• b11179 is marked as a pre-release. ^[2]
• The listed builds include CPU and accelerator-specific variants.
• The release lists builds for macOS, iOS, Linux, Android, and Windows. ^{[2] [16]}
• The release adds support for sparse FA on SYCL. ^[2]
• The release lists builds for macOS, iOS, Linux, Android, Windows, and openEuler. ^{[3] [16]}
• The server now permits chunked prefill for RANK pooling when the context is causal. ^[3]

  can_split() now permits chunked prefill for RANK pooling when the

• The change addresses long-document and multimodal reranking for causal models including Qwen3 and Qwen3-VL. ^[3]
• Release b11223 is marked as a pre-release. ^[3]
• Wrapped content arrays are accepted for multimodal embedding requests. ^[4]

  Accept the OpenAI-style wrapped content array format for multimodal

• The server supports multimodal input for Qwen3-VL-Embedding. ^[4]

  server : support multimodal input for /v1/embeddings (Qwen3-VL-Embedding)

• The server supports typed vision, audio, and video content input for the /v1/embeddings endpoint. ^[4]

  server : support typed content (vision/audio/video) input for /v1/embeddings endpoint ( #29556 )

• The GitHub release is marked Pre-release. ^{[4] [16]}
• Bare content arrays are rejected with a migration message. ^[4]

  Bare content arrays (the unwrapped shape) are

• Legacy formats continue to work through tokenize_input_prompts. ^[4]
• The release adds MiMo support for dflash conversion and feature extraction. ^[5]
• The GitHub release is marked Pre-release. ^{[5] [16]}
• The release lists builds for macOS, Linux, Android, and Windows. ^{[5] [16]}
• The release lists Linux arm64 builds for Snapdragon CPU, Adreno GPU and Hexagon NPU. ^{[6] [16]}

  Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide

• The release adds q2_k and q3_k quant type support for Hexagon. ^[6]

  hexagon: add q2_k and q3_k quant type support ( #29717 )

• Release b11345 is marked as a pre-release. ^[6]
• The release lists builds for macOS, iOS, Linux, Android, Windows and openEuler. ^{[6] [16]}
• The release lists Android arm64 builds for Snapdragon CPU, Adreno GPU and Hexagon NPU. ^{[6] [16]}

  Android arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide

• The target verifies drafts by rejection sampling. ^[7]

  Make the drafter probabilistic and the target verify by rejection sampling

• The release adds probabilistic sampling for simple draft and MTP. ^[7]

  spec : add probabilistic sampling for simple draft and MTP ( #27694 )

• Probabilistic draft sampling is controlled by a flag whose default value is greedy. ^[7]

  Fallback to argmax sampling for grammar-constrained requests and adding flag for enabling probabilistic draft sampling.

• The release is marked pre-release and was released on 03 Oct. ^[7]
• Rejection sampling supports grammar-constrained requests. ^[7]

  Support grammar-constrained requests in rejection sampling

• The GitHub release is marked Pre-release. ^{[8] [16]}
• The release lists platform builds for macOS, iOS, Linux, Android, and Windows. ^{[8] [16]}
• The release adds initial support for the Clef decision model. ^[8]

  model: add support for clef decision model (text-only) ( #29831 )

• The Clef support is text-only. ^[8]

  model: add support for clef decision model (text-only) ( #29831 )

• The release lists builds for macOS Apple Silicon and Intel, and an iOS XCFramework. ^{[9] [16]}
• b11400 is marked as a pre-release. ^[9]
• The release adds support for both embeddings and raw tokens in a batch. ^[9]
• The release lists Linux builds for CPU, Vulkan, CUDA, ROCm, OpenVINO and SYCL configurations. ^{[9] [16]}
• The release lists Android and Windows builds. ^{[9] [16]}
• On that test, pp128 throughput increased from 10.70 to 93.87 tokens per second. ^[10]
• The release reports Qwen2.5-0.5B-Instruct Q8_0 tested on a Milk-V Jupiter. ^[10]
• b11408 is marked as a pre-release on GitHub. ^[10]
• The release marks Q8_0 as supported on the X60. ^[10]
• b11408 adds a Q8_0 IME1 matrix kernel for the SpacemiT X60. ^[10]

  ggml-cpu : add Q8_0 IME1 matrix kernel for SpacemiT X60 ( #28479 )

• b11507 is marked as a pre-release. ^[11]
• The release lists macOS, iOS, Linux, Android, and Windows builds. ^{[11] [16]}
• Linux and Windows builds are listed for CUDA 12 and CUDA 13. ^[16]
• b11507 adds support for MoE cache over multiple GPUs. ^[11]

  llama: support MoE cache over multiple GPUs ( #30112 )

• The release improves OpenCL support for FA and DK512 for Gemma-4. ^[12]
• The release lists builds for macOS, iOS, Linux, Android, Windows, and openEuler. ^{[12] [16]}
• The release is marked Pre-release. ^{[12] [16]}
• The release enables Gemma-4 E4B GPU decode through OpenCL. ^[12]

  opencl: enable Gemma-4 E4B GPU decode

• The release extends Gemma-4 GPU decode through OpenCL to E2B. ^[12]

  opencl: enable Gemma-4 E4B GPU decode

• The release adds MTP support for Mellum. ^[13]

  model : MTP support for Mellum2.1 ( #30612 )

• The release adds MTP support for Mellum2.1. ^[13]

  model : MTP support for Mellum2.1 ( #30612 )

• The release adds inline RoPE construction in Mellum. ^[13]

  model : inline rope construction in Mellum

• Builds are listed for macOS/iOS, Linux, Android, and Windows. ^{[13] [16]}
• b11695 is marked as a pre-release. ^[13]
• b11714 is marked as a pre-release. ^[14]
• The Discover Models surface includes a curated catalog list with search and a details pane. ^[14]

  The Discover surface: the curated-catalog list with its search and

• The Discover interface includes download options and a standalone download progress bar. ^[14]

  download options, and the standalone download progress bar that replaces

• Paused downloads can be resumed, and incomplete downloads are remembered across restarts. ^[14]
• The Discover setting gates browsing and downloading Hugging Face GGUF models. ^[14]

  The flag gates browsing and downloading HuggingFace GGUF models, so it is

• The release is v0.5.0, published on 23 Sep. ^{[15] [16]}

  The release expands hyper-connection, flash-attention, and fused MoE/SSM support across backends, with robustness, quantization, data-layout, and RPC/meta improvements.

• The release accelerates CUDA conv2d with implicit GEMM. ^[15]

  Accelerate CUDA conv2d with implicit GEMM ( #29135 )

• The release adds HRM-Text (DFM Mimir 1B) support. ^[15]

  It adds HRM-Text (DFM Mimir 1B) support, MiMo-V2.6 and HunyuanOCR conversion support, ggml 0.25.0 backend improvements, multi-address HTTP binding, image outputs from function calls, and several chat parser/UI fixes.

• Server function-call outputs support input_image. ^[15]

  Add input_image support to server function-call outputs ( #22575 )

• The server can bind to multiple comma-separated TCP addresses and UNIX sockets. ^[15]

  Allow the server to bind to multiple addresses ( #28690 )

• The release adds conversion support for MiMo-V2.6. ^[15]

  It adds HRM-Text (DFM Mimir 1B) support, MiMo-V2.6 and HunyuanOCR conversion support, ggml 0.25.0 backend improvements, multi-address HTTP binding, image outputs from function calls, and several chat parser/UI fixes.

• DeepSeek adds web search for apps and updates its Flash model Daily brief 2026-10-17
• Generate editable slides and dashboards from a prompt Daily brief 2026-10-11
• Build and govern workplace agents with North 2 Daily brief 2026-10-05
• Microsoft introduces FrogNano-4B-2609 for repository-level coding workflows Daily brief 2026-10-03
• Google introduces Gemini 4 Argon reasoning model Daily brief 2026-10-02
• ElevenLabs introduces two text-to-speech models Daily brief 2026-09-28