PURSUhInT

Our study titled “PURSUhInT: In Search of Informative Hint Points Based on Layer Clustering for Knowledge Distillation” is available online in 'Expert Systems with Applications' journal

Reyhan Kevser Keser is the leading author of a recent paper on knowledge distillation titled as “PURSUhInT: In Search of Informative Hint Points Based on Layer Clustering for Knowledge Distillation” which is available online in 'Expert Systems with Applications' journal.

In this study, we propose a clustering based hint selection methodology, where the layers of teacher model are clustered with respect to several metrics and the cluster centers are used as the hint points. Our method is applicable for any student network, once it is applied on a chosen teacher network. The figure below shows the steps of our approach.

This research is supported by Arcelik ITU R&D Center, The Scientific and Technological Research Council of Turkey (TUBITAK) under the grant number 121E378 and ITU Scientific Research Projects Fund under the grant number MOA-2019-42321.