Open-weight serving and inference infrastructure

Baseten publishes source-available DeepSeek Ultrafast serving code

Baseten releases a Rust/CUDA serving engine for DeepSeek-V4.1-Flash. Its Blackwell throughput is vendor-reported, and Elastic License 2.0 restricts managed hosting.

Open Model Weights published15 Nov 2026
Primary sourceBaseten Labs — DeepSeek Ultrafast official repository and README
Source published2026-11-13
FOLLOW OUR COVERAGE

Follow Open Model Weights on Google See more of our open-weight AI reporting in Google Search.

Add as a Preferred Source on Google
Original premium CGI-style editorial art of a chrome Blackwell server accelerator cluster with glossy teal data ribbons and a dark, reflective studio backdrop.
Original high-gloss conceptual illustration of accelerated open-weight model serving; not a photograph of Baseten hardware or an independently measured benchmark. AI-generated editorial illustration; not a documentary photograph.

Baseten has released the public DeepSeek Ultrafast serving repository, targeting the existing open-weight DeepSeek-V4.1-Flash checkpoint rather than introducing a new foundation model. The engine is engineered for NVIDIA Blackwell hardware and an OpenAI-compatible local serving interface. Its importance is infrastructural: a model's weights, parameter count and license stay the same while the practical cost and latency of inference can change dramatically with better kernels, request scheduling and speculative decoding. Primary source ↗

The project's own performance table reports 1,740 coding-decode tokens per second on four GB300 GPUs and 2,060 tokens per second on eight B300 GPUs. In the eight-B300 configuration, it also reports prefill of one million tokens in 11.7 seconds, or approximately 85,000 input tokens per second. These figures are measured by the maintainers on a particular coding prompt and GPU topology. They do not establish speed on an arbitrary user workload, and Open Model Weights has not reproduced them.

The repository describes a Rust serving engine that accepts API-compatible clients and supports million-token context. It also provides a distributed KV-cache design: servers retain recent conversation state in host memory, while other servers can retrieve that state using RDMA instead of recomputing the whole prompt. Redis tracks cache ownership. The approach is directly relevant to long-running agents and repeatedly resumed sessions, but deploying it depends on supported Blackwell systems, network configuration and operational expertise.

The release belongs in our serving and runtime evidence layer, not as a new model-registry entry. Readers should check the exact upstream commit, build instructions, dependency versions and benchmark methodology before treating the vendor's throughput figures as portable. The project is source-available under Elastic License 2.0, which limits use as a hosted or managed third-party service; that is not an unrestricted open-source grant. Visible code enables independent experimentation but does not make vendor throughput claims independently measured, or turn an eight-GPU server into a local desktop solution.

OMW REGISTRY WATCH

What this changes for the evidence layer

This is a source-available serving-engine release, NOT an OSI-open-source runtime or a new weight checkpoint. Elastic License 2.0 restricts offering the software to third parties as a hosted or managed service. Monitor the exact code revision, rights for commercial serving, hardware requirements, and independently replicated benchmarks before comparing runtimes; the underlying DeepSeek model remains a separate registry record.

PRIMARY SOURCE

Baseten Labs — DeepSeek Ultrafast official repository and README

This brief is based on the cited primary source. Performance, benchmark and comparative claims remain attributed unless Open Model Weights publishes an independent measurement.

Open source article ↗