满血版DeepSeek R1部署A100，基于INT8量化，相比BF16实现50%吞吐提升！ 美团搜推机器学习团队最新开源，实现对DeepSeek R1模型基本无损的INT8精度量化。 要知道，DeepSeek R1原生版本的模型权重为FP8数据格式，对GPU芯片类型有严格限制，仅能被英伟达新型GPU支持（如Ada ...

Investigations, conducted together with scientists at CERN, show promising results – with breakthrough performance – in their pursuit of faster Monte Carlo based simulations, which are an important ...

Essentially all AI training is done with 32-bit floating point. But doing AI inference with 32-bit floating point is expensive, power-hungry and slow. And quantizing models for 8-bit-integer, which is ...

The best kinds of research are those that test new ideas and that also lead to practical innovations in real products. It takes a keen eye to differentiate science projects, which can be fun but which ...

The general definition of quantization states that it is the process of mapping continuous infinite values to a smaller set of discrete finite values. In this blog, we will talk about quantization in ...

FriendliAI also offers a unique take on the current memory crisis hitting the industry, especially as inference becomes the dominant AI use case. As recently explored by SDxCentral, 2026 is tipped to ...