DeepSeek open sources DSpark, a new framework to speed up LLM inference by up to 85%

DeepSeek has announced the release of DSpark, a new open-source framework designed to significantly accelerate inference in large language models (LLMs). The system, released under the MIT license, is available to developers and researchers. DSpark employs a technique called speculative decoding, which allows a smaller draft model to predict multiple tokens ahead, enabling the larger model to verify them in parallel rather than generating each token sequentially. This reduces latency and improves user experience.

According to DeepSeek's tests, DSpark boosts per-user token generation speed by 60% to 85% on the DeepSeek-V4-Flash model and 57% to 78% on the larger DeepSeek-V4-Pro model. These improvements also allow the system to handle more concurrent users on the same hardware. In high-traffic scenarios, DSpark significantly outperforms previous methods, avoiding performance cliffs that older baselines encounter.

The core idea behind DSpark is to use a lightweight 'scout' model to guess the next few tokens, which the large model then quickly validates. This approach maintains accuracy while increasing speed. DeepSeek has published checkpoints showing that the technique is not limited to its own models; it can also be applied to other open-weight models such as Alibaba's Qwen and Google's Gemma.

While speculative decoding is not a new concept, DSpark's innovation lies in making it practical for large-scale production systems. DeepSeek tested DSpark on its latest models, DeepSeek-V4-Flash (284 billion parameters) and DeepSeek-V4-Pro (1.6 trillion parameters), both supporting context windows of up to one million tokens, and achieved substantial performance gains.

This development comes amid rising geopolitical tensions around AI, with the U.S. government imposing restrictions on models from companies like Anthropic and OpenAI. Despite this, DeepSeek's open-source strategy continues to influence global AI development. By releasing DSpark under the permissive MIT license, DeepSeek removes barriers for commercial and academic use, potentially accelerating the adoption of faster and more efficient AI systems.