intermediate/realtime_rpi: a tutorial on realtime CV inference on a raspberry pi #1821
Conversation
|
✔️ Deploy Preview for pytorch-tutorials-preview ready! 🔨 Explore the source changes: 8a57a77 🔍 Inspect the deploy log: https://app.netlify.com/sites/pytorch-tutorials-preview/deploys/6201ddf34b45690008843867 😎 Browse the preview: https://deploy-preview-1821--pytorch-tutorials-preview.netlify.app |
| .. code:: shell | ||
|
|
||
| $ python3 -c "import torch; print(torch.__version__)" | ||
| 1.10.0+cpu |
There was a problem hiding this comment.
I'd remove this line so you don't have to keep updating tutorial with new version
| # convert opencv output from BGR to RGB | ||
| image = image[:, :, [2, 1, 0]] | ||
|
|
||
| NOTE: You can get even more performance by training the model directly with OpenCV's BGR data format to remove the conversion step. |
There was a problem hiding this comment.
I don't really have any docs here, this would be on the training dataset.
|
|
||
| preprocess = transforms.Compose([ | ||
| transforms.ToTensor(), | ||
| transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]), |
There was a problem hiding this comment.
maybe explain briefly what this is. This tutorial would be great for someone even if they don't know what pytorch is. Think smart high school students
| transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]), | ||
| ]) | ||
| input_tensor = preprocess(image) | ||
| input_batch = input_tensor.unsqueeze(0) # create a mini-batch as expected by the model |
There was a problem hiding this comment.
can also briefly elaborate here - can add shape annotations on top of input_tensor and input_batch
| MobileNetV2: Quantization and JIT | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| For optimal performance we want a model that's quantized and fused. Quantized |
There was a problem hiding this comment.
It's kinda cool that raspberry pi supports int8, definitely something I was surprised to learn
There was a problem hiding this comment.
Thankfully it doesn't require specialized hardware support. Most CPUs support 8bit operations since it's cheap to add hardware wise
There was a problem hiding this comment.
qnnpack is designed for mobile arm devices so it works surprisingly well on an Arm64 Raspberry Pi
| from torchvision import models | ||
| net = models.quantization.mobilenet_v2(pretrained=True, quantize=True) | ||
|
|
||
| We then want to jit the model to reduce Python overhead and fuse any ops. Jit gives us ~30fps instead of ~20fps without it. |
There was a problem hiding this comment.
Kinda interesting that vanilla pytorch worked fine. curious if you tried using pytorch live
There was a problem hiding this comment.
I think PyTorch Live is only for iOS/Android https://pytorch.org/live/docs/tutorials/get-started/ -- RPi is just aarch64 Linux so standard PyTorch works out of the box
| most of the work to fuse and quantize has already been done for you so you can | ||
| directly deploy with good performance on a Raspberry Pi. | ||
|
|
||
| See more: |
There was a problem hiding this comment.
I'd like to see a gif of the model working, you can even showcase what it's seeing - will also be nice to share in tweet threads
There was a problem hiding this comment.
yup, definitely planning on it. no more mysterious gdrive python packages
This adds a tutorial on how to use PyTorch for real time inference on a Raspberry Pi 4. This requires a number of specialized steps to get it all setup correctly and running permanently so seems worthwhile to add to the tutorials.
This requires a 64-bit operating system and RPi OS 64-bit just came out of beta so seems like a good time to document this. https://www.raspberrypi.com/news/raspberry-pi-os-64-bit/
There's a number of existing tutorials but they're way more complicated than they need to be since
pip install torchworks out of the box on 64-bit images. Much better to justpip installthan install from Google Drive which the two top Google searches recommend.This also seems to be the fastest known way of running mobilenetv2 on a Raspberry Pi 4 (without accelerators) that I've seen by an order of magnitude. At ~30-32fps that's ~33ms per inference including camera capture and processing. A quick Google search shows:
The images are currently hosted on githubusercontent, let me know if I should place them in the repo instead.
Test plan:
I've tested all of these steps from scratch using the official Raspberry Pi OS (64-bit) images.
Screenshots: