ConvMixer-torch2tf

This repository hosts code for converting the original ConvMixer ¹ (PyTorch) models to TensorFlow. In this repo we convert these 6 ConvMixer models to TensorFlow SavedModels:

The models can also be found on TensorFlow Hub

Model Name	Fine-tunable	Resolution	acc@1	#params	File Size	TensorFlow Model
ConvMixer-1536/20	❌	224x224	81.37	51.6 x 10^6	184MB	github/drive/bucket
ConvMixer-768/32	❌	224x224	80.16	21.1 x 10^6	75MB	github/drive/bucket
ConvMixer-1024/20	❌	224x224	76.94	24.4 x 10^6	87MB	github/drive/bucket
Feature Extractor ConvMixer-1536/20	✅	224x224	-	-	178MB	github/drive/bucket
Feature Extractor ConvMixer-768/32	✅	224x224	-	-	75MB	github/drive/bucket
Feature Extractor ConvMixer-1024/20	✅	224x224	-	-	75MB	github/drive/bucket

You can use the Feature Extractor models for fine-tuning on your own datasets.

ConvMixer is a simple model, proposed in the paper "Patches Are All You Need?" ² for image classification pre-trained on ImageNet-1K. ConvMixer uses only standard convolutions to achieve the mixing steps. Despite it's simplicity it outperforms ViT and MLP-Mixer. Using the conversion.ipynb notebook, one should be able to take a model from the pre-trained models and convert that to TensorFlow and use that with TensorFlow Hub and Keras.

The original model classes and weights were converted using the onnx-tf tool ³.

About the Notebooks

conversion.ipynb

This notebook demonstrates how the models were converted to TensorFlow SavedModel and uses onnx-tf ³ tool to do so. This notebook loads the ConvMixer model using timm (PyTorch Image Models) ⁴ modifying some of the code to allow doing so as shown in the timm_convmixer.py file in this repo.

classification.ipynb

This noteboook demonstrates how to load a ConvMixer Model varaint and run image classification using TensorFlow and TensorFlow Hub ⁵. This notebook will soon be updated to fetch models from tfhub.dev. These models can be used in the exact same manner with TensorFlow Keras too.

Want to Contribute 🙋‍♂️?

Awesome! If you want to contribute to this project, you're always welcome! See Contributing Guidelines. You can also take a look at open issues for getting more information about current or upcoming tasks.

Want to discuss? 💬

Have any questions, doubts or want to present your opinions, views? You're always welcome. You can start discussions.

License

Copyright 2020 Rishit Dagli

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

References

Footnotes

1. Official Code Implementation: https://github.com/tmp-iclr/convmixer ↩

2. Anonymous. Patches Are All You Need? 2021. openreview.net, https://openreview.net/forum?id=TVHS5Y4dNvM. ↩

3. onnx-tensorflow. Tensorflow Backend for ONNX. https://github.com/onnx/onnx-tensorflow ↩ ↩²

4. Ross Wightman, . "PyTorch Image Models." https://github.com/rwightman/pytorch-image-models. (2019). ↩

5. TensorFlow Hub. https://www.tensorflow.org/hub ↩