Softmax Öznitelikleri ile Özbilgi Damıtma Yoluyla Derin Sinir Ağlarının Görüntü Sınıflandırma Performansının İyileştirilmesi

Abstract

This study proposes a new self-knowledge distillation method to enhance the classification performance of deep learning models. Compared to traditional knowledge distillation, which relies on the softmax outputs of a teacher model, the proposed method provides greater flexibility in learning. In standard distillation, the student model is constrained to mimic the teacher, whereas the proposed approach integrates the teacher's predictions as an additional input, allowing the model to preserve its own learning dynamics. This approach balances the teacher's guidance with independent feature learning, strengthening decision boundaries. The proposed method has been tested on CIFAR-10 and CIFAR-100 datasets and evaluated using basic convolutional neural network and ResNet50 architectures. The results demonstrate that this method offers a valuable performance improvement in applications where classification accuracy is crucial. © 2025 Elsevier B.V., All rights reserved.