DeepInfer
DeepInfer is an open-source inference framework for serving DeepSeek models at scale, with offline inference and features such as optimized kernels, asynchronous scheduling, and structured outputs. It supports DeepSeek-V4.1-Flash and runs on NVIDIA SM100 GPUs or Huawei Ascend 950 NPUs.
What it does
Developers can use DeepInfer to run offline inference for DeepSeek-V4.1-Flash on supported NVIDIA or Huawei hardware.
- Availability
- Open-source release; build from source
- Pricing model
- Apache License 2.0
- Deployment
- Offline inference; Linux; NVIDIA SM100 GPUs or Huawei Ascend 950 NPUs
- Privacy
- Not stated
- Licence
- Apache License 2.0
Capability
Modality
Access
Deployment
Pricing
Target user
Evidence
5 claims- DeepInfer is available as an open-source release. [1]
- The release supports DeepSeek-V4.1-Flash. [1]
This open-source release provides offline inference with the core features needed for serving at scale.
GitHub new repositories. Quoted text is outside CC BY 4.0 and keeps its original rights. - Structured outputs can be constrained to a regex, JSON schema, or EBNF grammar. [1]
- **Structured outputs:** constrain generation to a regex, JSON schema, or EBNF grammar.
GitHub new repositories. Quoted text is outside CC BY 4.0 and keeps its original rights. - Supported hardware includes NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. [1]
- A KV cache system and load balancing across data-parallel replicas are not included in the release. [1]
- Load balancing across data-parallel replicas for both prefill and decode
GitHub new repositories. Quoted text is outside CC BY 4.0 and keeps its original rights.
Sources
1
1
GitHub new repositoriesOwner-controlled
https://github.com/deepseek-ai/DeepInfer
Retrieved 2026-10-29 00:34 UTC
Verified
Versions
2| Version | Effective | Change | Changed fields | State |
|---|---|---|---|---|
| 2 | 2026-10-21 10:23 UTC | material change | capability, access, pricing model, deployment, privacy | Current |
| 1 | 2026-10-20 11:14 UTC | publication | None | Current |
In briefs
1- Voice agents can take actions during a live conversation Daily brief 2026-10-20
Original catalog content is licensed under CC BY 4.0, attributed to wowsuchai. Source quotations and linked content keep their original rights.