Tencent has open-sourced AngelSpec, a unified training framework that supports six draft architectures for speculative decoding. The framework introduces innovative techniques like Training-Time Test (TTT) and block-diffusion drafting, achieving up to 2.4x speedups over autoregressive decoding on Hy3 models.
In the race to deploy large language models at scale, inference speed remains a critical bottleneck. Speculative decoding has emerged as a powerful technique to accelerate generation without sacrificing quality. However, existing implementations often lack flexibility, requiring different frameworks for different draft architectures. Tencent’s AngelSpec changes this by providing a unified, open-source training framework that supports multiple speculative decoding strategies—including multi-token prediction (MTP) and block-parallel methods—specifically designed for their Hy3 model family.
AngelSpec introduces a modular design that treats workload heterogeneity as a first-class constraint. It features specialized drafters for different domains, a config-driven pipeline that supports six distinct draft architectures, and cutting-edge training enhancements like Training-Time Test (TTT) and block-diffusion drafting. The result is significant inference speedups: up to 2.4× over autoregressive decoding on Hy3-295B-A21B models at scale.
As LLMs grow larger, autoregressive decoding—generating one token at a time—becomes progressively slower. Speculative decoding mitigates this by using a smaller, faster draft model to propose multiple tokens, which are then verified by the target model in parallel. This approach has become a standard technique in the industry from 2024 to 2026, with increasing adoption for production inference.
However, speculative decoding frameworks have often been fragmented. Different draft architectures—such as Eagle3, MTP, and block-based methods—require separate training pipelines. This fragmentation hinders adoption and optimization.
AngelSpec addresses this by providing a unified training framework that supports six draft architectures behind a single, config-driven pipeline. This flexibility allows practitioners to select and train the optimal draft model for their specific workload, whether code generation, math reasoning, or general text.
AngelSpec out of the box supports six draft architectures: DFly, DFlash, DFlare, Eagle3, DSpark, and MTP. These span multi-token prediction (MTP) and block-parallel speculative decoding methods. Users can switch between drafters by editing configuration files, making experimentation easy.
MTP drafters predict multiple future tokens simultaneously, improving generation parallelism. Block-parallel drafters like DFly propose entire blocks of tokens in one shot, enabling even larger speedups.
One of AngelSpec’s standout contributions is the application of Training-Time Test (TTT) and target-model rollout during MTP training. These techniques fine-tune the draft model’s predictions using on-the-fly test passes through the target model, improving coherence.
The results are compelling. According to the AngelSpec Technical Report (2026), the MTP drafter’s mean acceptance rate at temperature T=0 rose from 52.8% to 66.4% after TTT and target-model rollout. Similarly, the mean accepted length improved from 2.58 to 2.99 tokens. This represents a substantial reduction in generation stalls, directly translating to faster inference.
Perhaps the most impressive improvements come from DFly, a block-diffusion architecture designed for block-parallel speculative decoding. DFly generates entire blocks of draft tokens simultaneously, rather than sequentially.
On the Hy3-A21B model, DFly achieves a mean accepted length of 4.79 tokens—outperforming DFlash (3.69) and MTP (3.00). That’s a 29.8% relative gain over DFlash and a 59.7% improvement over MTP.
These gains become even more pronounced at larger model sizes. On HY3-295B-A21B with TP=8, DFly-8 delivers a 2.22× speedup over autoregressive decoding at concurrency 16. Across concurrency levels from 4 to 64, DFly-8 achieves between 1.98× and 2.40× speedup. This demonstrates strong scalability under real-world serving loads.
AngelSpec further improves throughput with D-cut, an optimization that dynamically shortens verification runs. By sacrificing a small amount of acceptance probability, D-cut increases throughput by 15.7%—reaching 981 tokens per second at concurrency 64—with only a 2.8% acceptance sacrifice.
This kind of fine-grained optimization is critical for production deployments where every millisecond counts.
AngelSpec’s performance has been thoroughly benchmarked on Tencent’s Hy3 models. The following table summarizes key statistics from the official technical report.
Key Metrics:
| Metric | DFly (Hy3-A21B) | DFlash | MTP |
|---|---|---|---|
| Mean Accepted Length | 4.79 | 3.69 | 3.00 |
| Relative Gain vs MTP | +59.7% | – | – |
| Speedup at c16 (HY3-295B) | 2.22× | – | – |
Training Data Expansion: For further improvement, 700K additional training prompts (500K code, 200K math) were used to expand the training dataset for DFly, ensuring robustness across diverse tasks.
These benchmarks highlight the practical advantages of using a block-diffusion drafter like DFly and the value of a unified framework that supports such architectures.
Reflecting the industry trend toward open-source tooling for disaggregated inference, Tencent has released AngelSpec along with seven pre-trained checkpoints. These are available on both Hugging Face and ModelScope.
The checkpoint lineup includes:
This release allows the community to leverage AngelSpec on compatible models, not just Hy3. The config-driven architecture means users can adapt the framework for other model families with minimal code changes.
“We believe that open-sourcing AngelSpec will accelerate research and adoption of speculative decoding,” said the Tencent team. This openness is part of a broader trend in 2026 toward standardized, reusable infrastructure for speculative decoding—a shift from bespoke implementations to unified frameworks.
AngelSpec clearly positions itself at the forefront of efficient LLM inference. Its unified approach reduces complexity, enabling more teams to adopt speculative decoding. The advancements in MTP training (TTT) and block-parallel drafting (DFly) push the state of the art.
For technology professionals evaluating speculative decoding solutions, AngelSpec offers:
As LLMs continue to scale, efficient generation techniques like those in AngelSpec will become indispensable. By making these tools open source, Tencent not only benefits its own Hy3 deployment but also empowers the broader AI community to build faster, more cost-effective inference systems.
AngelSpec represents a notable step forward in unifying speculative decoding training and deployment. With its support for multiple draft architectures, innovative training techniques like TTT, and block-diffusion drafting achieving significant speedups, it addresses many of the fragmentation issues that have slowed adoption. The open-source release of the framework and pre-trained checkpoints invites collaboration and further innovation.
Key takeaways:
For developers and ML engineers looking to accelerate LLM inference, AngelSpec provides a comprehensive, open-source solution ready for exploration and adaptation.
AngelSpec is an open-source unified training framework from Tencent that accelerates large language model inference through speculative decoding. It addresses the fragmentation in existing speculative decoding frameworks by supporting six different draft architectures in a single config-driven pipeline, achieving up to 2.4× speedups over standard autoregressive decoding.
Autoregressive decoding generates one token at a time sequentially, which becomes slower as models grow larger. Speculative decoding uses a smaller draft model to propose multiple tokens in parallel, which are then verified by the target model, allowing for faster generation without sacrificing output quality.
AngelSpec supports six draft architectures out of the box: DFly, DFlash, DFlare, Eagle3, DSpark, and MTP (multi-token prediction). This variety allows users to select the optimal drafting strategy for different workloads like code generation or mathematical reasoning.
Training-Time Test (TTT) is a technique that incorporates test-time evaluation signals into the training process of draft models. It enhances the draft model's ability to propose tokens that the target model is more likely to accept, thereby improving speculative decoding efficiency.
Developers can access AngelSpec through Tencent's open-source repository, where they can train draft models using the provided configurable pipeline, select from the six supported architectures, and evaluate speedups on compatible models like Tencent's Hy3 family. Detailed documentation and examples are available in the repository.
