DeepInfer: DeepSeek open-sources its inference engine, but it runs one model, offline, for now
DeepInfer is DeepSeek's open-source inference engine (Apache-2.0, built mainly on vLLM). The release runs only DeepSeek-V4.1-Flash, offline, on NVIDIA SM100 GPUs or Huawei Ascend 950 NPUs. It has no KV cache system and no online serving. The $38 per GPU-hour figure is unverified.
4 min read