SGLang 0.5.23 adds Paged Experts and faster DeepSeek V4.1 inference

SGLang has released version 0.5.23, an open-source inference-engine update with meaningful implications for teams deploying large open-weight models. Its official changelog spans more than a thousand pull requests and reports new serving recipes for existing checkpoints including Muse Spark Open, Reflection Beam-1.0, MiniMax-M3.1-Flash and several diffusion models. These are runtime integrations, not newly trained foundation models. The standout engineering change is an option to serve mixtures of experts whose full parameter sets do not fit in GPU memory.

Paged Experts, explicitly opt-in, moves expert management beyond the assumption that all expert weights must remain on the accelerator. SGLang reports an improvement of up to thirty times relative to CPU offload in its selected configuration. That is a comparison against a particular baseline, not evidence that every oversized model can run efficiently on any graphics card. Weight storage, PCIe transfers, active-expert routing and remaining KV-cache memory still determine whether a practical local deployment works.

Data-center operators get a different set of gains. The SGLang team reports up to 1.5 times more DeepSeek V4.1 prefill throughput on NVIDIA GB300 under load and around twenty-eight percent lower first-token latency on selected 32K–64K prompts. Opt-in fused persistent-kernel plugins are reported to accelerate single-user decode by up to forty-four percent on GB300 and seventy-five percent on AMD MI350X. None of those figures is an independent Open Model Weights measurement; they are specific to publisher hardware, workloads and launch flags.

The release also adds an SGLang-TorchTPU path for Google TPU7x, LoRA adapters on quantized FP8 and NVFP4 MoE models, and HiCache file-backed prefix storage in selected distributed configurations. These features broaden deployment choices but raise compatibility questions around plugin maturity, memory accounting and adapter correctness. Administrators should pin the exact release, compare end-to-end request latency against their current server and test both accuracy and failure recovery before upgrading a production inference stack.