Baseten has released the public DeepSeek Ultrafast serving repository, targeting the existing open-weight DeepSeek-V4.1-Flash checkpoint rather than introducing a new foundation model. The engine is engineered for NVIDIA Blackwell hardware and an OpenAI-compatible local serving interface. Its importance is infrastructural: a model's weights, parameter count and license stay the same while the practical cost and latency of inference can change dramatically with better kernels, request scheduling and speculative decoding. Primary source ↗
The project's own performance table reports 1,740 coding-decode tokens per second on four GB300 GPUs and 2,060 tokens per second on eight B300 GPUs. In the eight-B300 configuration, it also reports prefill of one million tokens in 11.7 seconds, or approximately 85,000 input tokens per second. These figures are measured by the maintainers on a particular coding prompt and GPU topology. They do not establish speed on an arbitrary user workload, and Open Model Weights has not reproduced them.
The repository describes a Rust serving engine that accepts API-compatible clients and supports million-token context. It also provides a distributed KV-cache design: servers retain recent conversation state in host memory, while other servers can retrieve that state using RDMA instead of recomputing the whole prompt. Redis tracks cache ownership. The approach is directly relevant to long-running agents and repeatedly resumed sessions, but deploying it depends on supported Blackwell systems, network configuration and operational expertise.
The release belongs in our serving and runtime evidence layer, not as a new model-registry entry. Readers should check the exact upstream commit, build instructions, dependency versions and benchmark methodology before treating the vendor's throughput figures as portable. The project is source-available under Elastic License 2.0, which limits use as a hosted or managed third-party service; that is not an unrestricted open-source grant. Visible code enables independent experimentation but does not make vendor throughput claims independently measured, or turn an eight-GPU server into a local desktop solution.
What this changes for the evidence layer
This is a source-available serving-engine release, NOT an OSI-open-source runtime or a new weight checkpoint. Elastic License 2.0 restricts offering the software to third parties as a hosted or managed service. Monitor the exact code revision, rights for commercial serving, hardware requirements, and independently replicated benchmarks before comparing runtimes; the underlying DeepSeek model remains a separate registry record.
Baseten Labs — DeepSeek Ultrafast official repository and README
This brief is based on the cited primary source. Performance, benchmark and comparative claims remain attributed unless Open Model Weights publishes an independent measurement.
Open source article ↗