Forbes

Here’s How Big LLMs Teach Smaller AI Models Via Leveraging Knowledge Distillation

Forbes contributors publish independent expert analyses and insights. Dr. Lance B. Eliot is a world-renowned AI scientist and consultant. In today’s column, I examine the rising tendency of employing ...
CNBC

How DeepSeek used distillation to train its artificial intelligence model, and what it means for companies such as OpenAI

Chinese artificial intelligence lab DeepSeek roiled markets in January, setting off a massive tech and semiconductor selloff after unveiling AI models that it said were cheaper and more efficient than ...
Wired

Distillation Can Make AI Models Smaller and Cheaper

The original version of this story appeared in Quanta Magazine. The Chinese AI company DeepSeek released a chatbot earlier this year called R1, which drew a huge amount of attention. Most of it ...
InfoWorld

Researchers propose a self-distillation fix for ‘catastrophic forgetting’ in LLMs

LLMs tend to lose prior skills when fine-tuned for new tasks. A new self-distillation approach aims to reduce regression and ...
Computer Weekly

DeepSeek shows enterprises model distillation opportunity

Model distillation is one of the technology trends that has reached a level of maturity identified in Gartner’s 2025 Hype Cycle for artificial intelligence (AI) as “the slope of enlightenment”.