Steam BiT: match Steam Banners with Google's BiT

This repository contains Python code to retrieve Steam games with similar store banners, using Google's BiT.

Image similarity is assessed by the cosine similarity between image features encoded by BiT.

Model

BiT is a convolutional neural network based on a ResNet architecture:

• pre-trained for classification on ImageNet-1k (1.28M images with 1000 classes) for BiT-S,
• pre-trained for classification on ImageNet-21k (14M images with ~21k classes) for BiT-M.

In this repository, the image encoder is a BiT-R50x1 model, which results from the distillation of a BiT-M-R152x2. First, the teacher model is fine-tuned on ImageNet-1k. Then, the distillation is performed on ImageNet-1k.

Data

Data consists of vertical Steam banners (300x450 resolution), resized to 256x384 resolution.

This is performed with rom1504/img2dataset.

Usage

• To download image data, run download_steam_webdataset.ipynb.

Alternatively, you can find the data as v0.1 in the "Releases" section of this repository.

• To match images, run match_steam_banners_with_BiT.ipynb.

References

• Google's Big Transfer (BiT):
  • Official blog post
  • Official Github repository
  • Kolesnikov, Alexander, et al. Big Transfer (BiT): General Visual Representation Learning. ECCV 2020.
  • Beyer, Lucas, et al. Knowledge distillation: A good teacher is patient and consistent. arXiv 2021.
• A generic repository to match images:
  • match-steam-banners: retrieve games with similar banners,
• My usage of Facebook's DINO:
  • steam-DINO: retrieve games with similar banners, using Facebook's DINO (resolution 224),
• My usage of OpenAI's CLIP:
  • steam-CLIP: retrieve games with similar banners, using OpenAI's CLIP (resolution 224),
  • steam-image-search: retrieve games using natural language queries,
  • heroku-flask-api: serve the matching results through an API built with Flask on Heroku.