Speculative decoding accelerates large language model generation by allowing multiple tokens to be drafted swiftly by a lightweight model before being verified by a larger, more powerful one. This ...

Researchers from Intel Labs and the Weizmann Institute of Science have introduced a major advance in speculative decoding. The new technique, presented at the International Conference on Machine ...

The ReDrafter software is designed to significantly speed up the execution of large language models on Nvidia GPUs. The tool is open source. Apple has launched a project in collaboration with Nvidia ...