- MegaDetector overview
- Our ask to MegaDetector users
- Downloading the model(s)
- Using the models
- Tell me more about why detectors are a good first step for camera trap images
- Pretty picture
- Mesmerizing video
- Can you share the training data?
Conservation biologists invest a huge amount of time reviewing camera trap images, and – even worse – a huge fraction of that time is spent reviewing images they aren’t interested in. This primarily includes empty images, but for many projects, images of people and vehicles are also “noise”, or at least need to be handled separately from animals.
Machine learning can accelerate this process, letting biologists spend their time on the images that matter.
To this end, this page hosts a model we’ve trained to detect animals, people, and vehicles in camera trap images, using several hundred thousand bounding boxes from a variety of ecosystems. It does not identify animals, it just finds them. The current model is based on Faster-RCNN with an InceptionResNetv2 base network, and was trained with the TensorFlow Object Detection API. We use this model as our first stage for classifier training and inference.
MegaDetector is free, and it makes us super-happy when people use it, so we put it out there as a downloadable model that is easy to use in a variety of conservation scenarios. That means we don’t know who’s using it unless you contact us (or we happen to run into you), so please please pretty-please email us at cameratraps@microsoft.com if you find it useful!
In addition to incorporating additional data, this release adds a preliminary “human” class. Our animal training data is still far more comprehensive than our humans-in-camera-traps data, so if you’re interested in using our detector but find that it works better on animals than people, stay tuned.
- Frozen model (.pb)
- TFODAPI config file
- Last checkpoint (for resuming training)
- TensorFlow SavedModel (inputs in TF common image format,
defaultoutput signature) - Tensorflow SavedModel for TFServing (inputs in uint8 format,
serving_defaultoutput signature)
First MegaDetector release!
We provide three ways to apply this model to new images - see below. For the two scripts, we describe how you can get set up if you have Python 3 and pip installed on a Linux computer or if you are on Windows and are starting from scratch. If your computer is also used for other Python projects, we recommend you set up the environment as described in the Installation section of our main readme, and use conda to set up a virtual environment in which to run scripts from this repo. This reduces potential version conflict headaches with your other projects. The environment file you should use to run the two scripts below is environment-detector.yml. You will still need to add the required repos to PYTHONPATH, but don't have to worry about installing Python, pip or any packages yourself. If you do not have a GPU on your computer, change tensorflow-gpu to tensorflow in environment-detector.yml.
When we describe how to run our two inference scripts below, we assume the following:
- You have Python installed. If you’re working on Windows, we recommend installing Anaconda, which is Python plus a zillion useful packages.
- You have checked out this git repo, and the AI for Earth Utilities repo. If you’re not familiar with git, we recommend installing Git for Windows. Specific instructions for checking out the repo will be rolled into the next step.
- You have put the base folder of both repos on your Python path. If you are using Windows, for example, you would do this by finding the directory to which you cloned each repo, and adding that directory to your PYTHONPATH environment variable. Here’s a good page about editing environment variables in Windows. You will need administrative access to your PC to set an environment variable.
Here are instructions for steps 2 and 3 that assume you don’t have administrative access to your PC (if you set the environment variable as per above, you can skip the “set PYTHONPATH” step here). We’re going to put things in “c:\git”, but you can use any folder you like.
After installing git and Anaconda, open an Anaconda Prompt, and run:
cd c:\git
git clone https://github.com/Microsoft/cameratraps
git clone https://github.com/Microsoft/ai4eutils
set PYTHONPATH=c:\git\cameratraps;c:\git\ai4eutils
Subsequent times you open your Anaconda prompt, you'll just need to do:
set PYTHONPATH=c:\git\cameratraps;c:\git\ai4eutils
python find_repeat_detections.py
To “test drive” this model on small sets of images and get super-satisfying visual output, we provide run_tf_detector.py, an example script for invoking this detector on new images. This script doesn’t depend on anything else in our repo, so you can download it and give it a try. Let us know how it works on your images!
To try this out (on Linux), assuming you have Python 3 and pip installed, you can do:
# Download the script and the MegaDetector model file
wget https://raw.githubusercontent.com/microsoft/CameraTraps/master/detection/run_tf_detector.py
wget https://lilablobssc.blob.core.windows.net/models/camera_traps/megadetector/megadetector_v3.pb
# Install TensorFlow
#
# If you have a GPU on your computer, change "tensorflow" to "tensorflow-gpu"
pip install tensorflow==1.13.1
# Install other dependencies
pip install Pillow humanfriendly matplotlib tqdm jsonpickle
# Run MegaDetector
python run_tf_detector.py megadetector_v3.pb --image_file some_image_file.jpg
Run python run_tf_detector.py for a full list of options.
If you’re using Windows, you’ll need to install Python; if you’re starting from scratch, we recommend installing Anaconda, which is Python plus a zillion useful packages. The “git”, “pip”, and ”Python” lines above will then work fine for Windows too, and you will probably want to download the script and the MegaDetector model file through your browser.
Then you can do the following, changing "tensorflow" to "tensorflow-gpu" if you have a GPU on your computer:
pip install tensorflow==1.13.1
pip install Pillow humanfriendly matplotlib tqdm
python wherever_you_downloaded_the_script/run_tf_detector.py wherever_you_downloaded_the_detector_file/megadetector_v3.pb --image_file some_image_file.jpg
To apply this model to larger image sets on a single machine, we recommend a slightly different script, run_tf_detector_batch.py. This outputs data in the same format as our batch processing API, so you can leverage all of our post-processing tools.
To try this out (on Linux), assuming you have Python 3 and pip installed, you can do:
# Clone our two required git repos
git clone https://github.com/microsoft/CameraTraps/
git clone https://github.com/microsoft/ai4eutils/
# Add those repos to your Python path
export PYTHONPATH="$PYTHONPATH:$PWD/ai4eutils:$PWD/CameraTraps"
# Download the MegaDetector model file
wget -O ~/megadetector_v3.pb https://lilablobssc.blob.core.windows.net/models/camera_traps/megadetector/megadetector_v3.pb
# Install TensorFlow
#
# If you have a GPU on your computer, change "tensorflow" to "tensorflow-gpu"
pip install tensorflow==1.13.1
# Install other dependencies
pip install humanfriendly Pillow pandas tqdm
# Run MegaDetector
python CameraTraps/detection/run_tf_detector_batch.py ~/megadetector_v3.pb some_image_file.jpg some_output_file.json
Run python run_tf_detector_batch.py for a full list of options.
If you’re using Windows:
- You’ll need to install Python; if you’re starting from scratch, we recommend installing Anaconda, which is Python plus a zillion useful packages.
- You’ll need to install git; if you don’t have git installed, we recommend installing Git for Windows.
Then all the git, pip, and python steps in the above instructions should work fine on Windows. You’ll probably want to just download the MegaDetector model file in your browser. For the step where you add our repos to your Python path, you can do this by finding the directory to which you cloned each repo, and adding that directory to your PYTHONPATH environment variable. Here’s a good page about editing environment variables in Windows. You can also do all of this at the command prompt.
Putting all of this together, if you’ve installed Anaconda, this might look like the following (in your Anaconda prompt):
cd c:\git
git clone https://github.com/microsoft/CameraTraps/
git clone https://github.com/microsoft/ai4eutils/
set PYTHONPATH=c:\git\cameratraps;c:\git\ai4eutils
pip install tensorflow==1.13.1
pip install humanfriendly Pillow pandas tqdm
python CameraTraps/detection/run_tf_detector_batch.py wherever_you_put_the_detector_file/megadetector_v3.pb some_image_file.jpg some_output_file.json
Speaking of which, when we process loads of images from collaborators, we use our batch processing API, which we can make available externally on request. Email us for more information.
If you use the MegaDetector in a publication, please cite:
@article{beery2019efficient,
title={Efficient Pipeline for Camera Trap Image Review},
author={Beery, Sara and Morris, Dan and Yang, Siyu},
journal={arXiv preprint arXiv:1907.06772},
year={2019}
}
Can do! See these “slides”.
Here’s a “teaser” image of what detector output looks like:
Image credit University of Washington.
Here’s a neat video of our v2 detector running in a variety of ecosystems, on locations unseen during training.
Image credit eMammal.
This model is trained on bounding boxes from a variety of ecosystems, and many of the images we use in training are not publicly-shareable for license reasons. We do train in part on bounding boxes from two public data sets:
...so if our detector performs really well on those data sets, that’s great, but it’s a little bit cheating, because we haven’t published the set of locations from those data sets that we use during training.