Poor Results on CIFAR-100 with Smaller Models (e.g., ResNet34, ResNet18)

[tsunghan-wu]

Hello,

I am currently experimenting with the FixMatch algorithm with this repo, using smaller models (specifically ResNet34 and ResNet18) instead of the larger models (WideResNet and ResNeXt) mentioned in the original paper. While I have managed to obtain promising results on the CIFAR-10 dataset, I am encountering significant challenges when applying the same algorithm to the CIFAR-100 dataset. Despite my best efforts, the performance of the smaller models on CIFAR-100 is consistently poor (specifically, ~60 acc under 2500 labels).

I would appreciate any guidance or insights regarding the following questions:

1. Is there a known limitation or issue when using smaller models (e.g., ResNet34, ResNet18) with the FixMatch algorithm? Could the algorithm's design or hyperparameters be better suited for larger models?

2. Are there any specific modifications or adjustments required in the codebase or training procedure when using smaller models with CIFAR-100?

3. Are there any recommended techniques or best practices for achieving reasonable results with smaller models on CIFAR-100 using the FixMatch algorithm?

I have thoroughly reviewed the documentation and existing issues but have not found any information specifically addressing this issue. Any assistance or suggestions would be highly appreciated in resolving this challenge. Thank you for your time and support!