Skip to content

Repository files navigation

Feature fusion mechanisms play a crucial role in deep learning, enabling the extraction of diverse features from various networks. Historically, addition and concatenation have been the most commonly employed techniques for feature fusion. However, addition fails to preserve discriminative information, while concatenation combines features by increasing the dimensionality of the feature vector. Both methods are simple operations and are unable to discover the relationships or interactions between different features. In this paper, we propose a novel approach to feature fusion, known as Multihead Self-Attention Feature Fusion (MHSAFF), which effectively fuses the features of Convolutional Neural Networks and Transformer networks. This method, originally introduced in this work, demonstrates superior performance compared to addition and concatenation. MHSAFF is a robust and efficient approach for capturing long-range dependencies or interactions among different input features. To assess the efficacy of our proposed method, MHSAFF, we conducted experiments on a variety of benchmark datasets and custom data, including CIFAR10, Flower 102, Imagenette-which is a subset of 10 readily classifiable classes from ImageNet, and two cattle datasets as custom data. Our experiments not only revealed that MHSAFF outperformed other methods in terms of performance, but also highlighted the importance of diverse features, which are derived from different networks. Additionally, our research explores various key, query, and value parameters in our proposed feature fusion technique, and we observed that generating these parameters from different inputs can significantly impact performance. Interestingly, we found that using the same patch for both the CNN and Transformer does not significantly affect accuracy. Overall, our study provides novel insights into feature fusion and offers an innovative method for this task.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages