PixelSelfAttention

Trying to adapt BERT for images... w

Work in progress. Although it works much better than MLP, it's still worse than vanilla CNN.

I think it will need to use CNN for learning higher-dimensional embeddings, otherwise it's not powerful enough.

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
cifar-10		cifar-10
cifar-100		cifar-100
.gitignore		.gitignore
README.md		README.md
bert.py		bert.py
experiments.xlsx		experiments.xlsx
get_results.py		get_results.py
main_loop.py		main_loop.py
print_metrics.py		print_metrics.py
run_experiment.py		run_experiment.py
train.py		train.py
trainv2.py		trainv2.py

Provide feedback