If you’ve ever used a neural network to solve a complex problem, you know they can be enormous in size, containing millions of parameters. For instance, the famous BERT model has about ~110 million.

DeepSeek’s R1 release has generated heated discussions on the topic of model distillation and how companies may protect against unauthorized distillation. Model distillation has broad IP implications ...

Knowledge distillation is an increasingly influential technique in deep learning that involves transferring the knowledge embedded in a large, complex “teacher” network to a smaller, more efficient ...