CraftVision 👓💬

Judge not, that you be not judged. For with the judgment you pronounce you will be judged, and with the measure you use it will be measured to you. Matthew 7:1-2

Ever accidentally enjoyed a sub-standard beverage? Ever wondered whether your premium IPA was fresh and delicious — or simply your brain trying to desperately justify the exuberant asking price? Well, fear no longer.

Now with the wisdom of the crowd, you too can calibrate a uniquely typical palette for craft brews, with CraftVision.
Identify craft beers that pass the Critic's judgement ★★★★★

Methodology

1. A first pass segments classes using the YOLOv3 model.
2. Bottles are sent to a custom ResNet34 classifier, fine-tuned on a collated label dataset (pre-trained on imagenet).
3. Ratings from BeerAdvocate are collected and displayed with a OpenCV overlay.

→ 📔 See nb_controller.ipynb for notebook

→ 📉 Training and hyperparameters are in 02_Dataset_and_training.ipynb

→ 📑 CraftVision/detector.py for inference code

Installation from AWS

Deep Learning AMI (Ubuntu) Version 25.0, GPU p2.xlarge for training ☑️, 100 GB

SSH into new linux box, activate pytorch conda environment

$ ssh -i "<key>.pem" ubuntu@ec2-<public-ip>.us-east-2.compute.amazonaws.com
$ source activate pytorch_p36

Clone and install requirements

$ git clone https://github.com/lukexyz/CraftVision.git
$ cd CraftVision
$ sudo pip3 install -r requirements.txt

Download pretrained weights

$ cd weights/
$ bash download_weights.sh

Run notebooks

$ jupyter notebook --ip=0.0.0.0 --no-browser

Access notebooks through a browser with the aws public IPv4 address (found in ec2 instance console), not the private IP as shown in the notebook terminal. Do use the provided token from the terminal.

http://<public IP>:8888/?token=<token>

Development Notes

1. ✅ Get Fastai custom dataset trained for labels
2. ✅ Get YOLOv3 implementation running
3. ✅ Extract bottle bounding box from YOLOv3 and send them to the brand classifier CNN
4. ✅ Run repo from WSL for development and use aws GPUs for training
5. ✅ Run on saved video and tune up for real-time stream later
6. ✅ Improve HUD visuals when critic score is returned and display beverage info
   • Resize image output so text is readable
7. Increase number of brands → Run YoloV3 on training corpus to extract bottles before training
8. Refactor detector to optimise for stream capture
9. Upgrade segmentation model from YOLO to detectron2

MIT License

Acknowledgements

• eriklindernoren/PyTorch-YOLOv3
• Jeremy Howard - Practical Deep Learning for Coders, v3 - Lesson 2