From c104e80c6e9caa598ae5acbc0181db9bc5b6e82e Mon Sep 17 00:00:00 2001 From: Dan Morris Date: Thu, 25 Aug 2022 10:16:34 -0700 Subject: [PATCH 01/17] Updating sync API for MDv5 (simplifying base image, updating docs) --- api/synchronous/README.md | 66 ++++++++++------------- api/synchronous/api_core/Dockerfile | 16 +++--- api/synchronous/api_core/supervisord.conf | 2 +- 3 files changed, 37 insertions(+), 47 deletions(-) diff --git a/api/synchronous/README.md b/api/synchronous/README.md index b37c979e8..d8f228a2b 100644 --- a/api/synchronous/README.md +++ b/api/synchronous/README.md @@ -13,67 +13,52 @@ The most notable prerequisite is nvidia-docker; install according to: -### Set up the repo +### Clone this repo -- Clone the camera traps repo +```bash +git clone "https://github.com/microsoft/CameraTraps/" +cd CameraTraps +``` - ```bash - git clone "https://github.com/microsoft/CameraTraps/" - cd CameraTraps - ``` -- During this testing phase, switch to the api-flask-redis-v1 branch - - ```bash - git checkout api-flask-redis-v1 - ```` - - ### Download the model file -- Download the MegaDetector model file to `api_flask_redis/api_core/animal_detection_api/model` +Download the MegaDetector model file(s) to `api_flask_redis/api_core/animal_detection_api/model`. We will download both MDv5a and MDv5b here, though currently the API is hard-coded to use MDv5a. - ```bash - wget "https://lilablobssc.blob.core.windows.net/models/camera_traps/megadetector/md_v4.1.0/md_v4.1.0.pb" -O api_flask_redis/api_core/animal_detection_api/model/md_v4.1.0.pb - ``` +```bash +wget "https://github.com/microsoft/CameraTraps/releases/download/v5.0/md_v5a.0.0.pt" -O api/synchronous/api_core/animal_detection_api/model/md_v5a.0.0.pt +wget "https://github.com/microsoft/CameraTraps/releases/download/v5.0/md_v5b.0.0.pt" -O api/synchronous/api_core/animal_detection_api/model/md_v5b.0.0.pt +``` ### Enable API key authentication (optional) -- To authenticate the API via a key, create a file with name `allowed_keys.txt`, add it to the folder `api_flask_redis/api_core/animal_detection_api`, then add a list of keys to the file, with one key per line. +To authenticate the API via a key, create a file with name `allowed_keys.txt`, add it to the folder `api/synchronous/api_core/animal_detection_api`, then add a list of keys to the file, with one key per line. ### Build the Docker image -- Switch to the `api_flask_redis/api_core` folder, from which the Docker image expects to be built +- Switch to the `api/synchronous/api_core` folder, from which the Docker image expects to be built ```bash - cd api_flask_redis/api_core + cd api/synchronous/api_core ``` -- Name the API's Docker image (the name doesn't matter, this is just a convenience if you are experimenting with multiple versions) +- Name the API's Docker image (the name doesn't matter, having a name is just a convenience if you are experimenting with multiple versions, but subsequent steps will assume you have set this environment variable to something) ```bash export API_DOCKER_IMAGE=camera-trap-api:1.0 ``` -- Set the base TensorFlow image +- Select the Docker base image... the following is the image against which we've tested, so if you aren't particularl about your Docker environment, we recommend this one (for GPU support): - For GPU environments: - - ```bash - export BASE_IMAGE=tensorflow/tensorflow:1.14.0-gpu-py3 - ``` - - Nota bene: we have historically run MegaDetector v4 in a TF 1.1x environment (since that's how it was trained), but TF 1.1x is incompatible with some newer GPUs, and you may find that things hang after loading CuDNN. If you experience that, try a TF2 environment instead: - ```bash - export BASE_IMAGE=tensorflow/tensorflow:latest-gpu + export BASE_IMAGE=nvidia/cuda:11.3.1-cudnn8-runtime-ubuntu20.04 ``` - For non-GPU environments: - +- ...or this one (for CPU-only deployments): + ```bash - export BASE_IMAGE=tensorflow/tensorflow:1.14.0-py3 + export BASE_IMAGE=ubuntu:20.04 ``` ### Build the Docker image @@ -89,18 +74,18 @@ The following will run the API locally on port 5050. - For GPU environments: ```bash - sudo nvidia-docker run -it -p 5050:1212 $API_DOCKER_IMAGE + sudo nvidia-docker run -it -p 5050:1213 $API_DOCKER_IMAGE ``` - For non-GPU environments: ```bash - sudo docker run -it -p 5050:1212 $API_DOCKER_IMAGE + sudo docker run -it -p 5050:1213 $API_DOCKER_IMAGE ``` ## Test the API in Postman -- To test in Postman, in a Postman tab enter the URL of the API, e.g.: +- To test in Postman, in a Postman tab, enter the URL of the API, e.g.: `http://100.100.200.200:5050/v1/camera-trap/sync/detect` @@ -112,14 +97,17 @@ The following will run the API locally on port 5050. - Under `Body` select `form-data`, create one key/value pair per image, with values of type "file" (to upload an image file) - Click `Send` +### Setting header options + ![Test in postman](images/postman_url_params.jpg) -
+### Specifying an API key ![Test in postman](images/postman_api_key.jpg) -
+### Sending an image ![Test in postman](images/postman_formdata_images.jpg)
+ diff --git a/api/synchronous/api_core/Dockerfile b/api/synchronous/api_core/Dockerfile index 3511ea44d..2ce802162 100644 --- a/api/synchronous/api_core/Dockerfile +++ b/api/synchronous/api_core/Dockerfile @@ -1,5 +1,6 @@ - -ARG BASE_IMAGE=tensorflow/tensorflow:1.14.0-gpu-py3 +# Default to an image with CUDA and NVIDIA drivers installed; the +# README recommends a different image for CPU-only installations. +ARG BASE_IMAGE=nvidia/cuda:11.3.1-cudnn8-runtime-ubuntu20.04 FROM $BASE_IMAGE # Install and expose redis default port @@ -22,6 +23,7 @@ RUN apt-get update --fix-missing \ python3-setuptools \ python3.7-dev \ python3.7 \ + python3-pip \ apt-transport-https \ build-essential \ gcc \ @@ -44,7 +46,7 @@ RUN pip3 install \ jsonpickle -# Install yolo5 requirements +# Install YOLOv5 requirements COPY ./yolov5/requirements.txt /yolov5_requirements.txt RUN pip3 install -r ./yolov5_requirements.txt @@ -55,7 +57,7 @@ RUN pip3 install -r ./requirements.txt COPY ./animal_detection_api /app/animal_detection_api/ RUN true -# Copy yolo5 code +# Copy YOLOv5 code COPY ./yolov5 /app/yolov5/ RUN true @@ -66,14 +68,14 @@ RUN true COPY ./startup.sh / RUN chmod +x /startup.sh - ENV PYTHONPATH="${PYTHONPATH}:/app/animal_detection_api/:/app/yolov5/" ENV PYTHONUNBUFFERED=TRUE -# Expose the port that is to be used when calling your API - +# Expose the API port +# # If you change the port here, remember to update supervisord.conf EXPOSE 1213 # Starts up the detector, Flask app and Redis server ENTRYPOINT [ "/startup.sh" ] + diff --git a/api/synchronous/api_core/supervisord.conf b/api/synchronous/api_core/supervisord.conf index da53667ee..4c9c89262 100644 --- a/api/synchronous/api_core/supervisord.conf +++ b/api/synchronous/api_core/supervisord.conf @@ -10,7 +10,7 @@ stdout_logfile=/var/log/redis/stdout.log stderr_logfile=/var/log/redis/stderr.log [program:detector] -command=python /app/animal_detection_api/api_backend.py +command=python3 /app/animal_detection_api/api_backend.py stdout_logfile=/dev/stdout stdout_logfile_maxbytes=0 stderr_logfile=/dev/stdout From 0cc857611b30591566ce76f84b0611b8a228fa28 Mon Sep 17 00:00:00 2001 From: Dan Morris Date: Thu, 25 Aug 2022 14:56:47 -0700 Subject: [PATCH 02/17] Finished moving sync API to PyTorch --- .gitignore | 6 +++ api/synchronous/README.md | 38 +++++++-------- api/synchronous/api_core/Dockerfile | 47 ++++++++++++------- api/synchronous/api_core/build_docker.sh | 6 ++- api/synchronous/api_core/requirements.txt | 5 +- api/synchronous/api_core/supervisord.conf | 3 +- .../camera_trap_flask_api_test.ipynb | 27 ++++++----- 7 files changed, 78 insertions(+), 54 deletions(-) diff --git a/.gitignore b/.gitignore index ca4852422..13c2fb27b 100644 --- a/.gitignore +++ b/.gitignore @@ -53,3 +53,9 @@ api_config*.py *.pth *.o debug.log +*.swp + +# Things created when building the sync API +yolov5 +api/synchronous/api_core/animal_detection_api/detection + diff --git a/api/synchronous/README.md b/api/synchronous/README.md index d8f228a2b..a3b90a1fe 100644 --- a/api/synchronous/README.md +++ b/api/synchronous/README.md @@ -23,7 +23,7 @@ cd CameraTraps ### Download the model file -Download the MegaDetector model file(s) to `api_flask_redis/api_core/animal_detection_api/model`. We will download both MDv5a and MDv5b here, though currently the API is hard-coded to use MDv5a. +Download the MegaDetector model file(s) to `api/synchronous/animal_detection_api/model`. We will download both MDv5a and MDv5b here, though currently the API is hard-coded to use MDv5a. ```bash wget "https://github.com/microsoft/CameraTraps/releases/download/v5.0/md_v5a.0.0.pt" -O api/synchronous/api_core/animal_detection_api/model/md_v5a.0.0.pt @@ -37,39 +37,35 @@ To authenticate the API via a key, create a file with name `allowed_keys.txt`, a ### Build the Docker image -- Switch to the `api/synchronous/api_core` folder, from which the Docker image expects to be built +- Switch to the `api/synchronous/api_core` folder, from which the Docker image expects to be built. ```bash cd api/synchronous/api_core ``` -- Name the API's Docker image (the name doesn't matter, having a name is just a convenience if you are experimenting with multiple versions, but subsequent steps will assume you have set this environment variable to something) +- Name the API's Docker image (the name doesn't matter, having a name is just a convenience if you are experimenting with multiple versions, but subsequent steps will assume you have set this environment variable to something). ```bash export API_DOCKER_IMAGE=camera-trap-api:1.0 ``` -- Select the Docker base image... the following is the image against which we've tested, so if you aren't particularl about your Docker environment, we recommend this one (for GPU support): +- Select the Docker base image... we recommend this one: ```bash - export BASE_IMAGE=nvidia/cuda:11.3.1-cudnn8-runtime-ubuntu20.04 + export BASE_IMAGE=pytorch/pytorch:1.10.0-cuda11.3-cudnn8-runtime ``` -- ...or this one (for CPU-only deployments): +- Build the Docker image using build_docker.sh. ```bash - export BASE_IMAGE=ubuntu:20.04 + sudo sh build_docker.sh $BASE_IMAGE $API_DOCKER_IMAGE ``` -### Build the Docker image - -```bash -sudo sh build_docker.sh $BASE_IMAGE $API_DOCKER_IMAGE -``` +Building may take 5-10 minutes. ### Run the Docker image -The following will run the API locally on port 5050. +The following will run the API on port 5050, but you can change that to the port on which you want to run the API. - For GPU environments: @@ -89,13 +85,13 @@ The following will run the API locally on port 5050. `http://100.100.200.200:5050/v1/camera-trap/sync/detect` - - Select `POST` - - Optionally add the `min_confidence` parameter, which sets the minimum detection confidence that's returned to the caller (defaults to 0.1) - - Optionally add the `min_rendering_confidence` parameter, which sets the minimum detection confidence that's rendered to returned images (defaults to 0.8) (not meaningful if "render" is False) - - Optionally add the `render` parameter, set to `true` if you would like the images to be rendered with bounding boxes - - If you enabled authentication by adding the file `allowed_keys.txt` under `api_flask_redis/api_core/animal_detection_api`then in the headers tab add the `key` parameter and enter the key value (this would be one of the keys that you saved to the file `allowed_keys.txt`) - - Under `Body` select `form-data`, create one key/value pair per image, with values of type "file" (to upload an image file) - - Click `Send` + - Select `POST`. + - Optionally add the `min_confidence` parameter, which sets the minimum detection confidence that's returned to the caller (defaults to 0.1). + - Optionally add the `min_rendering_confidence` parameter, which sets the minimum detection confidence that's rendered to returned images (defaults to 0.8) (not meaningful if "render" is False). + - Optionally add the `render` parameter, set to `true` if you would like the images to be rendered with bounding boxes. + - If you enabled authentication by adding the file `allowed_keys.txt` under `api/synchronous/api_core/animal_detection_api`then in the headers tab add the `key` parameter and enter the key value (this would be one of the keys that you saved to the file `allowed_keys.txt`). + - Under `Body` select `form-data`, and create one key/value pair per image, with values of type "file" (to upload an image file). To create a k/v pair of type "file", hover over the right side of the box where it says "key"; a drop-down will appear where you can select "file". + - Click `Send`. ### Setting header options @@ -105,7 +101,7 @@ The following will run the API locally on port 5050. ![Test in postman](images/postman_api_key.jpg) -### Sending an image +### Sending one or more images ![Test in postman](images/postman_formdata_images.jpg) diff --git a/api/synchronous/api_core/Dockerfile b/api/synchronous/api_core/Dockerfile index 2ce802162..cc02f5025 100644 --- a/api/synchronous/api_core/Dockerfile +++ b/api/synchronous/api_core/Dockerfile @@ -1,9 +1,10 @@ -# Default to an image with CUDA and NVIDIA drivers installed; the -# README recommends a different image for CPU-only installations. -ARG BASE_IMAGE=nvidia/cuda:11.3.1-cudnn8-runtime-ubuntu20.04 +# Default to an image with PyTorch installed, for GPU installations. +# +# The README recommends a different image for CPU-only installations. +ARG BASE_IMAGE=pytorch/pytorch:1.10.0-cuda11.3-cudnn8-runtime FROM $BASE_IMAGE -# Install and expose redis default port +# Install Redis and expose the default Redis port RUN apt-get update && apt-get install -y redis-server EXPOSE 6379 @@ -31,40 +32,54 @@ RUN apt-get update --fix-missing \ libsm6 \ libxext6 -# Install additional packages +# Prepare to install Python dependencies RUN pip3 install --upgrade pip +# These are the standard MDv5 dependencies, other than PyTorch. +# +# PyTorch, CUDA, and CuDNN come from the base image. RUN pip3 install \ redis \ numpy \ requests \ matplotlib \ - pillow \ requests_toolbelt \ + pillow \ tqdm \ - humanfriendly\ - jsonpickle + humanfriendly \ + jsonpickle \ + opencv-python \ + pandas \ + seaborn==0.11.0 \ + PyYAML==6.0 +# Commenting this out, but leaving it here for posterity. We don't +# want to use YOLOv5's requirements file, because it will choose a +# version of PyTorch that - ironically - won't work with YOLOv5 (at +# least as of the time this Dockerfile was last edited). +# +# COPY ./yolov5/requirements.txt /yolov5_requirements.txt +# RUN pip3 install -r ./yolov5_requirements.txt -# Install YOLOv5 requirements -COPY ./yolov5/requirements.txt /yolov5_requirements.txt -RUN pip3 install -r ./yolov5_requirements.txt +WORKDIR / +# This requirements file contains that are specific to the API, i.e., +# not used by other parts of the repo, e.g. Flask. COPY ./requirements.txt / -RUN pip3 install -r ./requirements.txt +RUN TRUE +RUN pip3 install -r /requirements.txt -# Copy your API code +# Copy API code into the container COPY ./animal_detection_api /app/animal_detection_api/ RUN true -# Copy YOLOv5 code +# Copy YOLOv5 code into the container COPY ./yolov5 /app/yolov5/ RUN true +# Copy startup scripts/config into the container COPY ./supervisord.conf /etc/supervisord.conf RUN true - -# startup.sh is a helper script COPY ./startup.sh / RUN chmod +x /startup.sh diff --git a/api/synchronous/api_core/build_docker.sh b/api/synchronous/api_core/build_docker.sh index 965313cfb..46d4abfc8 100644 --- a/api/synchronous/api_core/build_docker.sh +++ b/api/synchronous/api_core/build_docker.sh @@ -4,13 +4,15 @@ # use the COPY action in the Dockerfile to copy them into the Docker image. # This is the main dependency -cp ../../../detection/tf_detector.py animal_detection_api/ mkdir animal_detection_api/detection/ cp -a ../../../detection/. animal_detection_api/detection/ -# Copy yolo5 dependencies +# Copy YOLOv5 dependencies git clone https://github.com/ultralytics/yolov5/ cd yolov5 + +# To ensure forward-compatibility, we pin a particular version +# of YOLOv5 git checkout c23a441c9df7ca9b1f275e8c8719c949269160d1 cd ../ diff --git a/api/synchronous/api_core/requirements.txt b/api/synchronous/api_core/requirements.txt index c00ef366a..ee0c16264 100644 --- a/api/synchronous/api_core/requirements.txt +++ b/api/synchronous/api_core/requirements.txt @@ -1,5 +1,4 @@ -Flask==2.1.0 +Flask==2.0.1 Flask-RESTful==0.3.8 gunicorn==20.0.4 -torch==1.10.1 -torchvision==0.11.2 + diff --git a/api/synchronous/api_core/supervisord.conf b/api/synchronous/api_core/supervisord.conf index 4c9c89262..59e03fd0a 100644 --- a/api/synchronous/api_core/supervisord.conf +++ b/api/synchronous/api_core/supervisord.conf @@ -16,7 +16,8 @@ stdout_logfile_maxbytes=0 stderr_logfile=/dev/stdout stderr_logfile_maxbytes=0 -# runs the flask app via gunicorn +# Runs the flask app via gunicorn. The port here needs to match the port +# exposed in the Dockerfile. [program:gunicorn] directory=/app/animal_detection_api/ command=gunicorn -b 0.0.0.0:1213 --workers 1 --workers 8 --threads 4 --timeout 6000 --graceful-timeout 1000 api_frontend:app diff --git a/api/synchronous/camera_trap_flask_api_test.ipynb b/api/synchronous/camera_trap_flask_api_test.ipynb index 2992c7c8c..1e3a12b95 100644 --- a/api/synchronous/camera_trap_flask_api_test.ipynb +++ b/api/synchronous/camera_trap_flask_api_test.ipynb @@ -29,16 +29,14 @@ "from requests_toolbelt.multipart import decoder\n", "from PIL import Image\n", "\n", - "# sample_input_dir should point to a folder full of png/jpg files\n", - "sample_input_root = os.getcwd()\n", - "sample_input_root = sample_input_root.replace('\\\\','/').replace('/api_flask_redis', '/api/synchronous/sample_input')\n", - "sample_input_dir = os.path.join(sample_input_root, 'png')\n", - "# sample_input_dir = os.path.join(sample_input_root, 'bad_inputs')\n", + "sample_input_dir = os.path.expanduser('~/git/CameraTraps/test_images/test_images')\n", "\n", - "ip_address = 'my_api_vm.southcentralus.cloudapp.azure.com'\n", + "ip_address = 'localhost'\n", "port = '5050'\n", "detect_endpoint = 'http://{}:{}/v1/camera-trap/sync/detect'.format(ip_address, port)\n", - "print(detect_endpoint)" + "print(detect_endpoint)\n", + "\n", + "max_images_per_call = 8" ] }, { @@ -78,7 +76,7 @@ "source": [ "params = {\n", " 'min_confidence': 0.15,\n", - " 'min_rendering_confidence': 0.8,\n", + " 'min_rendering_confidence': 0.2,\n", " 'render': True,\n", " 'key': None\n", "}\n", @@ -88,7 +86,14 @@ " return s\n", " \n", "file_handles = {}\n", - "for fn in filenames:\n", + "\n", + "if len(filenames) > max_images_per_call:\n", + " import random\n", + " filenames_to_submit = random.sample(filenames,max_images_per_call)\n", + "else:\n", + " filenames_to_submit = filenames\n", + " \n", + "for fn in filenames_to_submit:\n", " file_handles[fn] = (clean_filename(fn), open(fn, 'rb'), 'image/jpeg')\n", "\n", "m = MultipartEncoder(fields=file_handles)\n", @@ -183,7 +188,7 @@ "metadata": { "anaconda-cloud": {}, "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -197,7 +202,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.8" + "version": "3.9.7" } }, "nbformat": 4, From 1bdc9c8c23a0e62e28cb05fa30f8c4946983f522 Mon Sep 17 00:00:00 2001 From: Christopher Yeh Date: Sun, 28 Aug 2022 12:14:12 -0700 Subject: [PATCH 03/17] Simplify typing annotations in json_validator.py --- classification/json_validator.py | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/classification/json_validator.py b/classification/json_validator.py index 1ecd19bbb..b5857fdef 100644 --- a/classification/json_validator.py +++ b/classification/json_validator.py @@ -66,7 +66,7 @@ import os import pprint import random -from typing import Any, Optional +from typing import Any import pandas as pd import path_utils # from ai4eutils @@ -83,11 +83,11 @@ def main(label_spec_json_path: str, allow_multilabel: bool = False, single_parent_taxonomy: bool = False, check_blob_exists: bool | str = False, - min_locs: Optional[int] = None, - output_dir: Optional[str] = None, - json_indent: Optional[int] = None, + min_locs: int | None = None, + output_dir: str | None = None, + json_indent: int | None = None, seed: int = 123, - mislabeled_images_dir: Optional[str] = None) -> None: + mislabeled_images_dir: str | None = None) -> None: """Main function.""" # input validation assert os.path.exists(label_spec_json_path) @@ -251,7 +251,7 @@ def validate_json(input_js: dict[str, dict[str, Any]], def get_output_json(label_to_inclusions: dict[str, set[tuple[str, str]]], - mislabeled_images_dir: Optional[str] = None + mislabeled_images_dir: str | None = None ) -> dict[str, dict[str, Any]]: """Queries MegaDB to get image paths matching dataset_labels. @@ -432,7 +432,7 @@ def remove_non_images(js: MutableMapping[str, dict[str, Any]], def remove_nonexistent_images(js: MutableMapping[str, dict[str, Any]], log: MutableMapping[str, Any], - check_local: Optional[str] = None, + check_local: str | None = None, num_threads: int = 50) -> None: """Remove images that don't actually exist locally or on Azure Blob Storage. Modifies [js] and [log] in-place. @@ -518,7 +518,7 @@ def remove_images_insufficient_locs(js: MutableMapping[str, dict[str, Any]], def filter_images(output_js: Mapping[str, Mapping[str, Any]], label: str, - datasets: Optional[Container[str]] = None) -> set[str]: + datasets: Container[str] | None = None) -> set[str]: """Finds image files from output_js that have a given label and are from a set of datasets. From 042d457e9c7db41fc170ec24da32fd538345135c Mon Sep 17 00:00:00 2001 From: Christopher Yeh Date: Sun, 28 Aug 2022 12:22:50 -0700 Subject: [PATCH 04/17] Minor doc updates to merge_classification_detection_output.py --- .../merge_classification_detection_output.py | 40 ++++++++++--------- 1 file changed, 21 insertions(+), 19 deletions(-) diff --git a/classification/merge_classification_detection_output.py b/classification/merge_classification_detection_output.py index 7483a8272..c93783b40 100644 --- a/classification/merge_classification_detection_output.py +++ b/classification/merge_classification_detection_output.py @@ -5,7 +5,7 @@ 1) Either a "dataset CSV" (output of create_classification_dataset.py) or a "classification results CSV" (output of evaluate_model.py). The CSV is expected to have columns listed below. The 'label' and [label names] columns - are optional, but at least one of them must be proided. + are optional, but at least one of them must be provided. - 'path': str, path to cropped image - if passing in a detections JSON, must match ___cropXX_mdvY.Y.jpg @@ -66,7 +66,7 @@ import datetime import json import os -from typing import Any, Optional +from typing import Any import pandas as pd from tqdm import tqdm @@ -77,7 +77,7 @@ def row_to_classification_list(row: Mapping[str, Any], label_names: Sequence[str], contains_preds: bool, - label_pos: Optional[str], + label_pos: str | None, threshold: float, relative_conf: bool = False ) -> list[tuple[str, float]]: @@ -129,14 +129,14 @@ def main(classification_csv_path: str, output_json_path: str, classifier_name: str, threshold: float, - datasets: Optional[Sequence[str]], - detection_json_path: Optional[str], - queried_images_json_path: Optional[str], - detector_output_cache_base_dir: Optional[str], - detector_version: Optional[str], - samples_per_label: Optional[int], + datasets: Sequence[str] | None, + detection_json_path: str | None, + queried_images_json_path: str | None, + detector_output_cache_base_dir: str | None, + detector_version: str | None, + samples_per_label: int | None, seed: int, - label_pos: Optional[str], + label_pos: str | None, relative_conf: bool, typical_confidence_threshold: float) -> None: """Main function.""" @@ -196,7 +196,7 @@ def main(classification_csv_path: str, os.makedirs(os.path.dirname(output_json_path), exist_ok=True) with open(output_json_path, 'w') as f: json.dump(classification_js, f, indent=1) - + print('Wrote merged classification/detection results to {}'.format(output_json_path)) @@ -205,8 +205,8 @@ def process_queried_images( queried_images_json_path: str, detector_output_cache_base_dir: str, detector_version: str, - datasets: Optional[Sequence[str]] = None, - samples_per_label: Optional[int] = None, + datasets: Sequence[str] | None = None, + samples_per_label: int | None = None, seed: int = 123 ) -> dict[str, Any]: """Creates a detection JSON object roughly in the Batch API detection @@ -328,7 +328,7 @@ def combine_classification_with_detection( classifier_name: str, classifier_timestamp: str, threshold: float, - label_pos: Optional[str] = None, + label_pos: str | None = None, relative_conf: bool = False, typical_confidence_threshold: float = None ) -> dict[str, Any]: @@ -353,18 +353,20 @@ def combine_classification_with_detection( relative_conf: bool, if True then for each class, outputs its relative confidence over the confidence of the true label, requires 'label' to be in CSV + typical_confidence_threshold: float, useful default confidence + threshold; not used directly, just passed along to the output file Returns: dict, detections JSON file updated with classification results """ classification_metadata = { 'classifier': classifier_name, - 'classification_completion_time': classifier_timestamp + 'classification_completion_time': classifier_timestamp } - + if typical_confidence_threshold is not None: classification_metadata['classifier_metadata'] = \ {'typical_classification_threshold':typical_confidence_threshold} - + detection_js['info'].update(classification_metadata) detection_js['classification_categories'] = idx_to_label @@ -420,8 +422,8 @@ def _parse_args() -> argparse.Namespace: 'that image paths are given as /.') parser.add_argument( '--typical-confidence-threshold', type=float, default=None, - help='useful default confidence threshold; not used directly, just passed along to the output file') - + help='useful default confidence threshold; not used directly, just ' + 'passed along to the output file') detection_json_group = parser.add_argument_group( 'arguments for passing in a detections JSON file') From 2588cab9dff8b16896e4cd34f7ad34759c894568 Mon Sep 17 00:00:00 2001 From: Christopher Yeh Date: Sun, 28 Aug 2022 15:24:06 -0700 Subject: [PATCH 05/17] Better doc in run_classifier.py --- classification/run_classifier.py | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/classification/run_classifier.py b/classification/run_classifier.py index 94098d57e..b86ae1828 100644 --- a/classification/run_classifier.py +++ b/classification/run_classifier.py @@ -42,8 +42,8 @@ class SimpleDataset(torch.utils.data.Dataset): """Very simple dataset.""" def __init__(self, img_files: Sequence[str], - images_dir: Optional[str] = None, - transform: Optional[Callable[[PIL.Image.Image], Any]] = None): + images_dir: str | None = None, + transform: Callable[[PIL.Image.Image], Any] | None = None): """Creates a SimpleDataset.""" self.img_files = img_files self.images_dir = images_dir @@ -68,7 +68,7 @@ def __len__(self) -> int: def create_loader(cropped_images_dir: str, - detections_json_path: Optional[str], + detections_json_path: str | None, img_size: int, batch_size: int, num_workers: int @@ -77,7 +77,11 @@ def create_loader(cropped_images_dir: str, Args: cropped_images_dir: str, path to image crops - detections: optional dict, detections JSON + detections_json_path: optional str, path to detections JSON + img_size: int, resizes smallest side of image to img_size, + then center-crops to (img_size, img_size) + batch_size: int, batch size in dataloader + num_workers: int, # of workers in dataloader """ crop_files = [] @@ -96,7 +100,7 @@ def create_loader(cropped_images_dir: str, js = json.load(f) detections = {img['file']: img for img in js['images']} detector_version = js['info']['detector'] - + for img_file, info_dict in tqdm(detections.items()): if 'detections' not in info_dict or info_dict['detections'] is None: continue @@ -127,12 +131,12 @@ def create_loader(cropped_images_dir: str, def main(model_path: str, cropped_images_dir: str, output_csv_path: str, - detections_json_path: Optional[str], - classifier_categories_json_path: Optional[str], + detections_json_path: str | None, + classifier_categories_json_path: str | None, img_size: int, batch_size: int, num_workers: int, - device_id:int=None) -> None: + device_id: int | None = None) -> None: """Main function.""" # evaluating with accimage is much faster than Pillow or Pillow-SIMD try: @@ -155,7 +159,7 @@ def main(model_path: str, # create model print('Loading saved model') model = torch.jit.load(model_path) - model, device = train_classifier.prep_device(model,device_id=device_id) + model, device = train_classifier.prep_device(model, device_id=device_id) test_epoch(model, loader, device=device, label_names=label_names, output_csv_path=output_csv_path) @@ -164,7 +168,7 @@ def main(model_path: str, def test_epoch(model: torch.nn.Module, loader: torch.utils.data.DataLoader, device: torch.device, - label_names: Optional[Sequence[str]], + label_names: Sequence[str] | None, output_csv_path: str) -> None: """Runs for 1 epoch. @@ -249,5 +253,5 @@ def _parse_args() -> argparse.Namespace: classifier_categories_json_path=args.classifier_categories, img_size=args.image_size, batch_size=args.batch_size, - num_workers=args.num_workers, + num_workers=args.num_workers, device_id=args.device) From 64735467d0722b06b246c955cace860cee6cf193 Mon Sep 17 00:00:00 2001 From: Christopher Yeh Date: Sun, 28 Aug 2022 15:35:23 -0700 Subject: [PATCH 06/17] Update docs for train_classifier.py --- classification/train_classifier.py | 38 +++++++++++++++++------------- 1 file changed, 21 insertions(+), 17 deletions(-) diff --git a/classification/train_classifier.py b/classification/train_classifier.py index 202eef0b3..393255fd7 100644 --- a/classification/train_classifier.py +++ b/classification/train_classifier.py @@ -23,7 +23,7 @@ from datetime import datetime import json import os -from typing import Any, Optional +from typing import Any import numpy as np import PIL.Image @@ -77,10 +77,10 @@ class SimpleDataset(torch.utils.data.Dataset): def __init__(self, img_files: Sequence[str], labels: Sequence[Any], - sample_weights: Optional[Sequence[float]] = None, + sample_weights: Sequence[float] | None = None, img_base_dir: str = '', - transform: Optional[Callable[[PIL.Image.Image], Any]] = None, - target_transform: Optional[Callable[[Any], Any]] = None): + transform: Callable[[PIL.Image.Image], Any] | None = None, + target_transform: Callable[[Any], Any] | None = None): """Creates a SimpleDataset.""" self.img_files = img_files self.labels = labels @@ -171,7 +171,7 @@ def create_dataloaders( is_train = (split == 'train') and augment_train split_df = df[df['dataset_location'].isin(locs)] - sampler: Optional[torch.utils.data.Sampler] = None + sampler: torch.utils.data.Sampler | None = None weights = None if label_weighted or weight_by_detection_conf: # weights sums to: @@ -261,29 +261,33 @@ def build_model(model_name: str, num_classes: int, pretrained: bool | str, return model -def prep_device(model: torch.nn.Module, device_id:int=None) -> tuple[torch.nn.Module, torch.device]: +def prep_device(model: torch.nn.Module, device_id: int | None = None + ) -> tuple[torch.nn.Module, torch.device]: """Place model on appropriate device. Args: model: torch.nn.Module, not already wrapped with DataParallel + device_id: optional int, GPU device to use + if None, then uses DataParallel when possible + if specified, then only uses specified device Returns: model: torch.nn.Module, model placed on , wrapped with DataParallel if more than 1 GPU is found - device: torch.device, 'cuda:0' if GPU is found, otherwise 'cpu' + device: torch.device, 'cuda:{device_id}' if GPU is found, otherwise 'cpu' """ # detect GPU, use all if available if torch.cuda.is_available(): print('CUDA available') + torch.backends.cudnn.benchmark = True if device_id is not None: - print('Starting CUDA device {}'.format(device_id)) - device = torch.device('cuda:{}'.format(str(device_id))) + print(f'Starting CUDA device {device_id}') + device = torch.device(f'cuda:{device_id}') else: device = torch.device('cuda:0') - torch.backends.cudnn.benchmark = True device_ids = list(range(torch.cuda.device_count())) if len(device_ids) > 1: - print('Found multiple devices, enabling data parallelism ({})'.format(str(device_ids))) + print(f'Found multiple devices, enabling data parallelism ({device_ids})') model = torch.nn.DataParallel(model, device_ids=device_ids) else: print('CUDA not available, running on the CPU') @@ -307,7 +311,7 @@ def main(dataset_dir: str, num_workers: int, logdir: str, log_extreme_examples: int, - seed: Optional[int] = None) -> None: + seed: int | None = None) -> None: """Main function.""" # input validation assert os.path.exists(dataset_dir) @@ -463,7 +467,7 @@ def main(dataset_dir: str, def log_run(split: str, epoch: int, writer: tensorboard.SummaryWriter, label_names: Sequence[str], metrics: MutableMapping[str, float], - heaps: Optional[Mapping[str, Mapping[int, list[HeapItem]]]], + heaps: Mapping[str, Mapping[int, list[HeapItem]]] | None, cm: np.ndarray) -> None: """Logs the outputs (metrics, confusion matrix, tp/fp/fn images) from a single epoch run to Tensorboard. @@ -583,7 +587,7 @@ def track_extreme_examples(tp_heaps: dict[int, list[HeapItem]], def correct(outputs: torch.Tensor, labels: torch.Tensor, - weights: Optional[torch.Tensor] = None, + weights: torch.Tensor | None = None, top: Sequence[int] = (1,)) -> dict[int, float]: """ Args: @@ -614,13 +618,13 @@ def run_epoch(model: torch.nn.Module, weighted: bool, device: torch.device, top: Sequence[int] = (1, 3), - loss_fn: Optional[torch.nn.Module] = None, + loss_fn: torch.nn.Module | None = None, finetune: bool = False, - optimizer: Optional[torch.optim.Optimizer] = None, + optimizer: torch.optim.Optimizer | None = None, k_extreme: int = 0 ) -> tuple[ dict[str, float], - Optional[dict[str, dict[int, list[HeapItem]]]], + dict[str, dict[int, list[HeapItem]]] | None, np.ndarray ]: """Runs for 1 epoch. From 5c09905d50a8ae4a0922df8797852333e047bc6b Mon Sep 17 00:00:00 2001 From: Dan Morris Date: Sun, 28 Aug 2022 20:00:19 -0700 Subject: [PATCH 07/17] Box rendering options in run_detector.py --- README.md | 12 +++++----- detection/run_detector.py | 50 ++++++++++++++++++++++++++++++++------- megadetector.md | 6 +++-- 3 files changed, 52 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index 5f0c4ac2e..b41d5a60d 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,7 @@ If you're already familiar with MegaDetector and you're ready to run it on your We work with ecologists all over the world to help them spend less time annotating images and more time thinking about conservation. You can read a little more about how this works on our [getting started with MegaDetector](collaborations.md) page. -Here are a few of the organizations that have used MegaDetector... we're only listing organizations who (a) we know about and (b) have kindly given us permission to refer to them here, so if you're using MegaDetector or other tools from this repo and would like to be added to this list, email us! +Here are a few of the organizations that have used MegaDetector... we're only listing organizations who (a) we know about and (b) have kindly given us permission to refer to them here (or have posted publicly about their use of MegaDetector), so if you're using MegaDetector or other tools from this repo and would like to be added to this list, email us! * Idaho Department of Fish and Game * San Diego Zoo Global @@ -76,7 +76,6 @@ Here are a few of the organizations that have used MegaDetector... we're only li * Conservation X Labs * The Nature Conservancy in Wyoming * Seattle Urban Carnivore Project -* Road Ecology Center, University of California, Davis * Blackbird Environmental * UNSW Sydney * Taronga Conservation Society @@ -85,13 +84,14 @@ Here are a few of the organizations that have used MegaDetector... we're only li * Capitol Reef National Park and Utah Valley University * University of Victoria Applied Conservation Macro Ecology (ACME) Lab * Université du Québec en Outaouais Institut des Science de la Forêt Tempérée (ISFORT) -* University of British Columbia Wildlife Coexistence Lab ([OSS tool](https://github.com/WildCoLab/WildCo_Face_Blur)) -* Alberta Biodiversity Monitoring Institute (ABMI) ([blog post](https://wildcams.ca/blog/the-abmi-visits-the-zoo/)) -* Felidae Conservation Fund ([platform](https://wildepod.org/)) ([blog post](https://abhaykashyap.com/blog/ai-powered-camera-trap-image-annotation-system/)) +* University of British Columbia Wildlife Coexistence Lab ([WildCo-FaceBlur tool](https://github.com/WildCoLab/WildCo_Face_Blur)) +* Road Ecology Center, University of California, Davis ([Wildlife Observer Network platform](https://wildlifeobserver.net/)) +* The Nature Conservancy in California ([Animl platform](https://github.com/tnc-ca-geo/animl-frontend)) +* Felidae Conservation Fund ([WildePod platform](https://wildepod.org/)) ([blog post](https://abhaykashyap.com/blog/ai-powered-camera-trap-image-annotation-system/)) +* Alberta Biodiversity Monitoring Institute (ABMI) ([WildTrax platform](https://www.wildtrax.ca/)) ([blog post](https://wildcams.ca/blog/the-abmi-visits-the-zoo/)) * Irvine Ranch Conservancy ([story](https://www.ocregister.com/2022/03/30/ai-software-is-helping-researchers-focus-on-learning-about-ocs-wild-animals/)) * Wildlife Protection Solutions ([story](https://customers.microsoft.com/en-us/story/1384184517929343083-wildlife-protection-solutions-nonprofit-ai-for-earth)) * [TrapTagger](https://wildeyeconservation.org/trap-tagger-about/) -* [WildTrax](https://www.wildtrax.ca/) * [Camelot](https://camelotproject.org/) # Data diff --git a/detection/run_detector.py b/detection/run_detector.py index d753ee160..d8f1967d8 100644 --- a/detection/run_detector.py +++ b/detection/run_detector.py @@ -109,6 +109,9 @@ DEFAULT_RENDERING_CONFIDENCE_THRESHOLD = DETECTOR_METADATA['v5b.0.0']['typical_detection_threshold'] DEFAULT_OUTPUT_CONFIDENCE_THRESHOLD = 0.005 +DEFAULT_BOX_THICKNESS = 4 +DEFAULT_BOX_EXPANSION = 0 + #%% Classes @@ -261,7 +264,9 @@ def load_detector(model_file, force_cpu=False): def load_and_run_detector(model_file, image_file_names, output_dir, render_confidence_threshold=DEFAULT_RENDERING_CONFIDENCE_THRESHOLD, - crop_images=False): + crop_images=False, box_thickness=DEFAULT_BOX_THICKNESS, + box_expansion=DEFAULT_BOX_EXPANSION, + ): """Load and run detector on target images, and visualize the results.""" if len(image_file_names) == 0: @@ -378,10 +383,11 @@ def input_file_to_detection_file(fn, crop_index=-1): else: - # image is modified in place + # Image is modified in place viz_utils.render_detection_bounding_boxes(result['detections'], image, label_map=DEFAULT_DETECTOR_LABEL_MAP, - confidence_threshold=render_confidence_threshold) + confidence_threshold=render_confidence_threshold, + thickness=box_thickness, expansion=box_expansion) output_full_path = input_file_to_detection_file(im_file) image.save(output_full_path) @@ -405,6 +411,8 @@ def input_file_to_detection_file(fn, crop_index=-1): print('- inference took {}, std dev is {}'.format(humanfriendly.format_timespan(ave_time_infer), std_dev_time_infer)) +# ...def load_and_run_detector() + #%% Command-line driver @@ -412,43 +420,67 @@ def main(): parser = argparse.ArgumentParser( description='Module to run an animal detection model on images') + parser.add_argument( 'detector_file', help='Path to TensorFlow (.pb) or PyTorch (.pt) detector model file') - group = parser.add_mutually_exclusive_group(required=True) # must specify either an image file or a directory + + # Must specify either an image file or a directory + group = parser.add_mutually_exclusive_group(required=True) group.add_argument( '--image_file', help='Single file to process, mutually exclusive with --image_dir') group.add_argument( '--image_dir', help='Directory to search for images, with optional recursion by adding --recursive') + parser.add_argument( '--recursive', action='store_true', help='Recurse into directories, only meaningful if using --image_dir') + parser.add_argument( '--output_dir', help='Directory for output images (defaults to same as input)') + parser.add_argument( '--threshold', type=float, default=DEFAULT_RENDERING_CONFIDENCE_THRESHOLD, - help=('Confidence threshold between 0 and 1.0; only render boxes above this confidence' - ' (but only boxes above 0.005 confidence will be considered at all)')) + help=('Confidence threshold between 0 and 1.0; only render' + + ' boxes above this confidence (defaults to {})'.format( + DEFAULT_RENDERING_CONFIDENCE_THRESHOLD))) + parser.add_argument( '--crop', default=False, action="store_true", help=('If set, produces separate output images for each crop, ' 'rather than adding bounding boxes to the original image')) + + parser.add_argument( + '--box_thickness', + type=int, + default=DEFAULT_BOX_THICKNESS, + help=('Line width (in pixels) for box rendering (defaults to {})'.format( + DEFAULT_BOX_THICKNESS))) + + parser.add_argument( + '--box_expansion', + type=int, + default=DEFAULT_BOX_EXPANSION, + help=('Number of pixels to expand boxes by (defaults to {})'.format( + DEFAULT_BOX_EXPANSION))) + if len(sys.argv[1:]) == 0: parser.print_help() parser.exit() args = parser.parse_args() - assert os.path.exists(args.detector_file), 'detector file {} does not exist'.format(args.detector_file) - assert 0.0 < args.threshold <= 1.0, 'Confidence threshold needs to be between 0 and 1' # Python chained comparison + assert os.path.exists(args.detector_file), 'detector file {} does not exist'.format( + args.detector_file) + assert 0.0 < args.threshold <= 1.0, 'Confidence threshold needs to be between 0 and 1' if args.image_file: image_file_names = [args.image_file] @@ -470,6 +502,8 @@ def main(): image_file_names=image_file_names, output_dir=args.output_dir, render_confidence_threshold=args.threshold, + box_thickness=args.box_thickness, + box_expansion=args.box_expansion, crop_images=args.crop) diff --git a/megadetector.md b/megadetector.md index f71788b1a..0ff757762 100644 --- a/megadetector.md +++ b/megadetector.md @@ -7,7 +7,7 @@ 5. [Downloading the model](#downloading-the-model) 6. [Using the model](#using-the-model) 7. [Is there a GUI?](#is-there-a-gui) -8. [How do I use the results?](#how-do-i-use-the-results) +8. [How do I use the results?](#how-do-i-use-the-results) 9. [Have you evaluated MegaDetector's accuracy?](#have-you-evaluated-megadetectors-accuracy) 10. [Citing MegaDetector](#citing-megadetector) 11. [Tell me more about why detectors are a good first step for camera trap images](#tell-me-more-about-why-detectors-are-a-good-first-step-for-camera-trap-images) @@ -407,7 +407,9 @@ It's not quite as simple as "these platforms all run MegaDetector on your images * [Camelot](https://camelotproject.org/) * [WildePod](https://wildepod.org/) * [wpsWatch](https://wildlifeprotectionsolutions.org/wpswatch/) -* The [Zooniverse ML Subject Assistant](https://subject-assistant.zooniverse.org/#/intro) allows Zooniverse camera trap project owners to run MegaDetector and get "AI votes" on their camera trap images +* [Animl](https://github.com/tnc-ca-geo/animl-frontend) +* [Cam-WON](https://wildlifeobserver.net/) +* [Zooniverse ML Subject Assistant](https://subject-assistant.zooniverse.org/#/intro) ## How do I use the results? From 34da53a02b0a6fca5f7101165cce63d1a32b814b Mon Sep 17 00:00:00 2001 From: Dan Morris Date: Mon, 29 Aug 2022 08:21:35 -0700 Subject: [PATCH 08/17] Added a json-splitting cell to manage_local_batch --- .../data_preparation/manage_local_batch.py | 42 +++++++++++++++++-- 1 file changed, 38 insertions(+), 4 deletions(-) diff --git a/api/batch_processing/data_preparation/manage_local_batch.py b/api/batch_processing/data_preparation/manage_local_batch.py index 7cd2ded4f..c9ba12c85 100644 --- a/api/batch_processing/data_preparation/manage_local_batch.py +++ b/api/batch_processing/data_preparation/manage_local_batch.py @@ -338,7 +338,9 @@ def split_list(L, n): assert task_results['detection_categories'] == combined_results['detection_categories'] combined_results['images'].extend(copy.deepcopy(task_results['images'])) -assert len(combined_results['images']) == len(all_images) +assert len(combined_results['images']) == len(all_images), \ + 'Expected {} images in combined results, found {}'.format( + len(all_images),len(combined_results['images'])) result_filenames = [im['file'] for im in combined_results['images']] assert len(combined_results['images']) == len(set(result_filenames)) @@ -1218,7 +1220,39 @@ def remove_overflow_folders(relativePath): d = categorize_detections_by_size.categorize_detections_by_size(input_file,size_separated_file,options) -#%% Subsetting +#%% .json splitting + +data = None + +from api.batch_processing.postprocessing.subset_json_detector_output import ( + subset_json_detector_output, SubsetJsonDetectorOutputOptions) + +input_filename = filtered_output_filename +output_base = os.path.join(filename_base,'json_subsets') + +if False: + if data is None: + with open(input_filename) as f: + data = json.load(f) + print('Data set contains {} images'.format(len(data['images']))) + +print('Processing file {} to {}'.format(input_filename,output_base)) + +options = SubsetJsonDetectorOutputOptions() +# options.query = None +# options.replacement = None + +options.split_folders = True +options.make_folder_relative = True +options.split_folder_mode = 'bottom' # 'top', 'n_from_top', 'n_from_bottom' +options.split_folder_param = 0 +options.overwrite_json_files = False +options.confidence_threshold = 0.01 + +subset_data = subset_json_detector_output(input_filename, output_base, options, data) + + +#%% Custom splitting/subsetting data = None @@ -1268,13 +1302,13 @@ def remove_overflow_folders(relativePath): subset_json_detector_output(input_filename,output_filename,options) -#%% Folder splitting +#%% Splitting images into folders from api.batch_processing.postprocessing.separate_detections_into_folders import ( separate_detections_into_folders, SeparateDetectionsIntoFoldersOptions) default_threshold = 0.2 -base_output_folder = r'e:\{}-{}-separated'.format(base_task_name,default_threshold) +base_output_folder = os.path.expanduser('~/data/{}-{}-separated'.format(base_task_name,default_threshold)) options = SeparateDetectionsIntoFoldersOptions(default_threshold) From 7e397e097889de9265b7543c7dbbbbbdfa544aa0 Mon Sep 17 00:00:00 2001 From: Dan Morris Date: Tue, 30 Aug 2022 07:25:36 -0700 Subject: [PATCH 09/17] Update megadetector.md --- megadetector.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/megadetector.md b/megadetector.md index 0ff757762..385e1a2bc 100644 --- a/megadetector.md +++ b/megadetector.md @@ -389,7 +389,7 @@ But we recognize that Python tools can be a bit daunting, so we're excited that ### Tools for running MegaDetector locally -* [EcoAssist](https://github.com/PetervanLunteren/EcoAssist) is a GUI-based tool for running MegaDetector in MacOS environments (MDv4 only as far as we know) +* [EcoAssist](https://github.com/PetervanLunteren/EcoAssist) is a GUI-based tool for running MegaDetector in MacOS environments (supports MDv5) * [MegaDetector-GUI](https://github.com/petargyurov/megadetector-gui) is a GUI-based tool for running MegaDetector in Windows environments (MDv4 only as far as we know) ### Interactive demos/APIs From 9cc7e3cc2ee41f2b74a25410d1581f13dc29808d Mon Sep 17 00:00:00 2001 From: Dan Morris Date: Tue, 30 Aug 2022 15:32:06 -0700 Subject: [PATCH 10/17] supporting "starts with" queries in subset_json... --- .../data_preparation/manage_local_batch.py | 6 +- .../subset_json_detector_output.py | 98 +++++++++---------- 2 files changed, 49 insertions(+), 55 deletions(-) diff --git a/api/batch_processing/data_preparation/manage_local_batch.py b/api/batch_processing/data_preparation/manage_local_batch.py index c9ba12c85..5df50c361 100644 --- a/api/batch_processing/data_preparation/manage_local_batch.py +++ b/api/batch_processing/data_preparation/manage_local_batch.py @@ -1280,9 +1280,11 @@ def remove_overflow_folders(relativePath): options = SubsetJsonDetectorOutputOptions() options.confidence_threshold = 0.01 options.overwrite_json_files = True - options.make_folder_relative = True - options.query = folder_name + '\\' + options.query = folder_name + '/' + # This doesn't do anything in this case, since we're not splitting folders + # options.make_folder_relative = True + subset_data = subset_json_detector_output(input_filename, output_filename, options, data) diff --git a/api/batch_processing/postprocessing/subset_json_detector_output.py b/api/batch_processing/postprocessing/subset_json_detector_output.py index f02b9e7fe..1f06fbe50 100644 --- a/api/batch_processing/postprocessing/subset_json_detector_output.py +++ b/api/batch_processing/postprocessing/subset_json_detector_output.py @@ -8,6 +8,9 @@ # optionally replacing that query with a replacement token. If the query is blank, # can also be used to prepend content to all filenames. # +# Does not support regex's, but supports a special case of ^string to indicate "must start with +# to match". +# # 2) Create separate .jsons for each unique path, optionally making the filenames # in those .json's relative paths. In this case, you specify an output directory, # rather than an output path. All images in the folder blah\foo\bar will end up @@ -59,7 +62,6 @@ from tqdm import tqdm from ct_utils import args_to_object -from data_management.annotations import annotation_constants #%% Helper classes @@ -118,42 +120,6 @@ class SubsetJsonDetectorOutputOptions: #%% Main function -def add_missing_detection_results_fields(data): - """ - Temporary fix for a sort-of-bug that is causing us to remove fields other than "images" - from detection output in certain scenarios. - - Modifies *data* in-place, also returns *data*. - """ - - # Format spec: - # - # https://github.com/microsoft/CameraTraps/tree/master/api/batch_processing - - if 'images' not in data: - data['images'] = [] - - if 'info' not in data: - print('Adding "info" field to .json') - info = { - "detector": "unknown", - "detection_completion_time": "unknown", - "classifier": "unknown", - "classification_completion_time": "unknown" - } - data['info'] = info - - if 'classification_categories' not in data: - print('Adding "classification_categories" field to .json') - data['classification_categories'] = {} - - if 'detection_categories' not in data: - print('Adding "detection_categories" field to .json') - data['detection_categories'] = annotation_constants.bbox_category_id_to_name - - return data - - def write_detection_results(data, output_filename, options): """ Write the detector-output-formatted dict *data* to *output_filename*. @@ -178,6 +144,8 @@ def write_detection_results(data, output_filename, options): f.write(s) print(' ...done') +# ...write_detection_results() + def subset_json_detector_output_by_confidence(data, options): """ @@ -233,6 +201,8 @@ def subset_json_detector_output_by_confidence(data, options): return data +# ...subset_json_detector_output_by_confidence() + def remove_failed_images(data,options): """ @@ -262,6 +232,8 @@ def remove_failed_images(data,options): return data +# ...remove_failed_images() + def subset_json_detector_output_by_query(data, options): """ @@ -274,18 +246,31 @@ def subset_json_detector_output_by_query(data, options): print('Subsetting by query {}, replacement {}...'.format(options.query, options.replacement), end='') + query_string = options.query + query_starts_with = False + + # Support a special case regex-like notation for "starts with" + if query_string is not None and query_string.startswith('^'): + query_string = query_string[1:] + query_starts_with = True + # i_image = 0; im = images_in[0] for i_image, im in tqdm(enumerate(images_in), total=len(images_in)): fn = im['file'] # Only take images that match the query - if (options.query is not None) and (options.query not in fn): - continue + if query_string is not None: + if query_starts_with: + if (not fn.startswith(query_string)): + continue + else: + if query_string not in fn: + continue if options.replacement is not None: - if options.query is not None: - fn = fn.replace(options.query, options.replacement) + if query_string is not None: + fn = fn.replace(query_string, options.replacement) else: fn = options.replacement + fn @@ -300,6 +285,8 @@ def subset_json_detector_output_by_query(data, options): return data +# ...subset_json_detector_output_by_query() + def split_path(path, maxdepth=100): """ @@ -318,6 +305,8 @@ def split_path(path, maxdepth=100): if maxdepth and head and head != path \ else [head or tail] +# ...split_path() + def top_level_folder(p): """ @@ -342,8 +331,12 @@ def top_level_folder(p): return os.path.join(parts[0], parts[1]) else: return parts[0] + +# ...top_level_folder() + -if False: +if False: + p = 'blah/foo/bar'; s = top_level_folder(p); print(s); assert s == 'blah' p = '/blah/foo/bar'; s = top_level_folder(p); print(s); assert s == '/blah' p = 'bar'; s = top_level_folder(p); print(s); assert s == 'bar' @@ -383,10 +376,10 @@ def subset_json_detector_output(input_filename, output_filename, options, data=N print('Trimming to {} images'.format(options.debug_max_images)) data['images'] = data['images'][:options.debug_max_images] else: + print('Copying data') data = copy.deepcopy(data) + print('...done') - # data = add_missing_detection_results_fields(data) - if options.query is not None: data = subset_json_detector_output_by_query(data, options) @@ -456,6 +449,8 @@ def subset_json_detector_output(input_filename, output_filename, options, data=N folders_to_images.setdefault(dirname, []).append(im) + # ...for each image + print('Found {} unique folders'.format(len(folders_to_images))) # Optionally make paths relative @@ -470,7 +465,9 @@ def subset_json_detector_output(input_filename, output_filename, options, data=N fn = im['file'] relfn = os.path.relpath(fn, dirname).replace('\\', '/') im['file'] = relfn - + + # ...if we need to convert paths to be folder-relative + print('Finished converting to json-relative paths, writing output') os.makedirs(output_filename, exist_ok=True) @@ -501,6 +498,8 @@ def subset_json_detector_output(input_filename, output_filename, options, data=N # ...if we're splitting folders +# ...subset_json_detector_output() + #%% Interactive driver @@ -522,7 +521,7 @@ def subset_json_detector_output(input_filename, output_filename, options, data=N #%% Subset and split, but don't copy to individual folders - input_filename = r"C:\temp\tnc-hardage-20201028_detections.filtered_rde_0.60_0.85_10_0.05_r2_export\tnc-hardage-20201028_detections.filtered_rde_0.60_0.85_10_0.05_r2_export.json" + input_filename = r"C:\temp\xxx-20201028_detections.filtered_rde_0.60_0.85_10_0.05_r2_export\xxx-20201028_detections.filtered_rde_0.60_0.85_10_0.05_r2_export.json" output_filename = r"c:\temp\out" options = SubsetJsonDetectorOutputOptions() @@ -546,13 +545,6 @@ def subset_json_detector_output(input_filename, output_filename, options, data=N data = subset_json_detector_output(input_filename,output_filename,options,data) - - #%% Just do a filename replacement - - # python subset_json_detector_output.py "D:\temp\idfg\detections_idfg_20190625_refiltered.json" "D:\temp\idfg\detections_idfg_20190625_refiltered_renamed.json" --query "20190625-hddrop/" --replacement "" - - # python subset_json_detector_output.py "D:\temp\idfg\detections_idfg_20190625_refiltered_renamed.json" "D:\temp\idfg\output" --split_folders --make_folder_relative --copy_jsons_to_folders - #%% Command-line driver From 951167a21010cab4b144a03fa08ac149236c00f1 Mon Sep 17 00:00:00 2001 From: Dan Morris Date: Wed, 31 Aug 2022 12:41:56 -0700 Subject: [PATCH 11/17] Supporting non-default sizes in inference --- .../data_preparation/manage_local_batch.py | 12 +++- detection/pytorch_detector.py | 18 +++-- detection/run_detector.py | 16 ++++- detection/run_detector_batch.py | 66 +++++++++++++------ detection/tf_detector.py | 3 +- 5 files changed, 84 insertions(+), 31 deletions(-) diff --git a/api/batch_processing/data_preparation/manage_local_batch.py b/api/batch_processing/data_preparation/manage_local_batch.py index 5df50c361..07b6f36a5 100644 --- a/api/batch_processing/data_preparation/manage_local_batch.py +++ b/api/batch_processing/data_preparation/manage_local_batch.py @@ -46,6 +46,9 @@ quiet_mode = True +# Specify a target image size when running MD... strongly recommended to leave this at "None" +image_size = None + # Only relevant when running on CPU ncores = 1 @@ -181,9 +184,13 @@ def split_list(L, n): if quiet_mode: quiet_string = '--quiet' + image_size_string = '' + if image_size is not None: + image_size_string = '--image_size {}'.format(image_size) + # Generate the script to run MD - cmd = f'{cuda_string} python run_detector_batch.py "{model_file}" "{chunk_file}" "{output_fn}" {checkpoint_frequency_string} {checkpoint_path_string} {use_image_queue_string} {ncores_string} {quiet_string}' + cmd = f'{cuda_string} python run_detector_batch.py "{model_file}" "{chunk_file}" "{output_fn}" {checkpoint_frequency_string} {checkpoint_path_string} {use_image_queue_string} {ncores_string} {quiet_string} {image_size_string}' cmd_file = os.path.join(filename_base,'run_chunk_{}_gpu_{}.sh'.format(str(i_task).zfill(2), str(gpu_number).zfill(2))) @@ -255,7 +262,8 @@ def split_list(L, n): results=None, n_cores=ncores, use_image_queue=use_image_queue, - quiet=quiet_mode) + quiet=quiet_mode, + image_size=image_size) elapsed = time.time() - start_time print('Task {}: finished inference for {} images in {}'.format( diff --git a/detection/pytorch_detector.py b/detection/pytorch_detector.py index ef2871015..c39f29b7d 100644 --- a/detection/pytorch_detector.py +++ b/detection/pytorch_detector.py @@ -4,7 +4,6 @@ """ #%% Imports -import sys import torch import numpy as np @@ -16,7 +15,7 @@ # import pre- and post-processing functions from the YOLOv5 repo https://github.com/ultralytics/yolov5 from utils.general import non_max_suppression, scale_coords, xyxy2xywh from utils.augmentations import letterbox -except ModuleNotFoundError as e: +except ModuleNotFoundError: raise ModuleNotFoundError('Could not import YOLOv5 functions.') print(f'Using PyTorch version {torch.__version__}') @@ -45,7 +44,7 @@ def _load_model(model_pt_path, device): model = checkpoint['model'].float().fuse().eval() # FP32 model return model - def generate_detections_one_image(self, img_original, image_id, detection_threshold): + def generate_detections_one_image(self, img_original, image_id, detection_threshold, image_size=None): """Apply the detector to an image. Args: @@ -71,8 +70,19 @@ def generate_detections_one_image(self, img_original, image_id, detection_thresh img_original = np.asarray(img_original) # padded resize - img = letterbox(img_original, new_shape=PTDetector.IMAGE_SIZE, + target_size = PTDetector.IMAGE_SIZE + + # Image size can be an int (which translates to a square target size) or (h,w) + if image_size is not None: + assert isinstance(image_size,int) or (len(image_size)==2) + if False: + if (not isinstance(image_size,int)) or (image_size != PTDetector.IMAGE_SIZE): + print('Warning: using non-standard image size {}'.format(image_size)) + target_size = image_size + + img = letterbox(img_original, new_shape=target_size, stride=PTDetector.STRIDE, auto=True)[0] # JIT requires auto=False + img = img.transpose((2, 0, 1)) # HWC to CHW; PIL Image is RGB already img = np.ascontiguousarray(img) img = torch.from_numpy(img) diff --git a/detection/run_detector.py b/detection/run_detector.py index d8f1967d8..2d1968f24 100644 --- a/detection/run_detector.py +++ b/detection/run_detector.py @@ -265,7 +265,7 @@ def load_detector(model_file, force_cpu=False): def load_and_run_detector(model_file, image_file_names, output_dir, render_confidence_threshold=DEFAULT_RENDERING_CONFIDENCE_THRESHOLD, crop_images=False, box_thickness=DEFAULT_BOX_THICKNESS, - box_expansion=DEFAULT_BOX_EXPANSION, + box_expansion=DEFAULT_BOX_EXPANSION, image_size=None ): """Load and run detector on target images, and visualize the results.""" @@ -339,6 +339,8 @@ def input_file_to_detection_file(fn, crop_index=-1): fn = os.path.join(output_dir, fn) return fn + # ...def input_file_to_detection_file() + for im_file in tqdm(image_file_names): try: @@ -362,7 +364,8 @@ def input_file_to_detection_file(fn, crop_index=-1): start_time = time.time() result = detector.generate_detections_one_image(image, im_file, - detection_threshold=DEFAULT_OUTPUT_CONFIDENCE_THRESHOLD) + detection_threshold=DEFAULT_OUTPUT_CONFIDENCE_THRESHOLD, + image_size=image_size) detection_results.append(result) elapsed = time.time() - start_time @@ -443,6 +446,12 @@ def main(): '--output_dir', help='Directory for output images (defaults to same as input)') + parser.add_argument( + '--image_size', + type=int, + default=None, + help=('Force image resizing to a (square) integer size (not recommended to change this)')) + parser.add_argument( '--threshold', type=float, @@ -504,7 +513,8 @@ def main(): render_confidence_threshold=args.threshold, box_thickness=args.box_thickness, box_expansion=args.box_expansion, - crop_images=args.crop) + crop_images=args.crop, + image_size=args.image_size) if __name__ == '__main__': diff --git a/detection/run_detector_batch.py b/detection/run_detector_batch.py index dd0975f9d..fed7e2b87 100644 --- a/detection/run_detector_batch.py +++ b/detection/run_detector_batch.py @@ -121,7 +121,7 @@ def producer_func(q,image_files): print('Finished image loading'); sys.stdout.flush() -def consumer_func(q,return_queue,model_file,confidence_threshold): +def consumer_func(q,return_queue,model_file,confidence_threshold,image_size=None): """ Consumer function; only used when using the (optional) image queue. @@ -149,13 +149,16 @@ def consumer_func(q,return_queue,model_file,confidence_threshold): image = r[1] if verbose: print('De-queued image {}'.format(im_file)); sys.stdout.flush() - results.append(process_image(im_file,detector,confidence_threshold,image)) + results.append(process_image(im_file=im_file,detector=detector, + confidence_threshold=confidence_threshold, + image=image,quiet=True,image_size=image_size)) if verbose: print('Processed image {}'.format(im_file)); sys.stdout.flush() q.task_done() -def run_detector_with_image_queue(image_files,model_file,confidence_threshold,quiet=False): +def run_detector_with_image_queue(image_files,model_file,confidence_threshold, + quiet=False,image_size=None): """ Driver function for the (optional) multiprocessing-based image queue; only used when --use_image_queue is specified. Starts a reader process to read images from disk, but processes images in the @@ -185,13 +188,15 @@ def run_detector_with_image_queue(image_files,model_file,confidence_threshold,qu if run_separate_consumer_process: if use_threads_for_queue: - consumer = Thread(target=consumer_func,args=(q,return_queue,model_file,confidence_threshold,)) + consumer = Thread(target=consumer_func,args=(q,return_queue,model_file, + confidence_threshold,image_size,)) else: - consumer = Process(target=consumer_func,args=(q,return_queue,model_file,confidence_threshold,)) + consumer = Process(target=consumer_func,args=(q,return_queue,model_file, + confidence_threshold,image_size,)) consumer.daemon = True consumer.start() else: - consumer_func(q,return_queue,model_file,confidence_threshold) + consumer_func(q,return_queue,model_file,confidence_threshold,image_size) producer.join() print('Producer finished') @@ -224,7 +229,8 @@ def chunks_by_number_of_chunks(ls, n): #%% Image processing functions -def process_images(im_files, detector, confidence_threshold, use_image_queue=False, quiet=False): +def process_images(im_files, detector, confidence_threshold, use_image_queue=False, + quiet=False, image_size=None): """ Runs MegaDetector over a list of image files. @@ -245,15 +251,18 @@ def process_images(im_files, detector, confidence_threshold, use_image_queue=Fal print('Loaded model (batch level) in {}'.format(humanfriendly.format_timespan(elapsed))) if use_image_queue: - run_detector_with_image_queue(im_files, detector, confidence_threshold, quiet=quiet) + run_detector_with_image_queue(im_files, detector, confidence_threshold, + quiet=quiet, image_size=image_size) else: results = [] for im_file in im_files: - results.append(process_image(im_file, detector, confidence_threshold, quiet=quiet)) + results.append(process_image(im_file, detector, confidence_threshold, + quiet=quiet, image_size=image_size)) return results -def process_image(im_file, detector, confidence_threshold, image=None, quiet=False): +def process_image(im_file, detector, confidence_threshold, image=None, + quiet=False, image_size=None): """ Runs MegaDetector over a single image file. @@ -285,7 +294,7 @@ def process_image(im_file, detector, confidence_threshold, image=None, quiet=Fal try: result = detector.generate_detections_one_image( - image, im_file, detection_threshold=confidence_threshold) + image, im_file, detection_threshold=confidence_threshold, image_size=image_size) except Exception as e: if not quiet: print('Image {} cannot be processed. Exception: {}'.format(im_file, e)) @@ -301,8 +310,9 @@ def process_image(im_file, detector, confidence_threshold, image=None, quiet=Fal #%% Main function def load_and_run_detector_batch(model_file, image_file_names, checkpoint_path=None, - confidence_threshold=DEFAULT_OUTPUT_CONFIDENCE_THRESHOLD, checkpoint_frequency=-1, - results=None, n_cores=0, use_image_queue=False, quiet=False): + confidence_threshold=DEFAULT_OUTPUT_CONFIDENCE_THRESHOLD, + checkpoint_frequency=-1, results=None, n_cores=0, + use_image_queue=False, quiet=False, image_size=None): """ Args - model_file: str, path to .pb model file @@ -318,7 +328,7 @@ def load_and_run_detector_batch(model_file, image_file_names, checkpoint_path=No Returns - results: list of dict, each dict represents detections on one image """ - + # Handle the case where image_file_names is not yet actually a list if isinstance(image_file_names,str): @@ -361,7 +371,9 @@ def load_and_run_detector_batch(model_file, image_file_names, checkpoint_path=No if use_image_queue: assert n_cores <= 1 - results = run_detector_with_image_queue(image_file_names, model_file, confidence_threshold, quiet) + results = run_detector_with_image_queue(image_file_names, model_file, + confidence_threshold, quiet, + image_size=image_size) elif n_cores <= 1: @@ -384,7 +396,9 @@ def load_and_run_detector_batch(model_file, image_file_names, checkpoint_path=No count += 1 - result = process_image(im_file, detector, confidence_threshold, quiet=quiet) + result = process_image(im_file, detector, + confidence_threshold, quiet=quiet, + image_size=image_size) results.append(result) # Write a checkpoint if necessary @@ -425,7 +439,8 @@ def load_and_run_detector_batch(model_file, image_file_names, checkpoint_path=No image_batches = list(chunks_by_number_of_chunks(image_file_names, n_cores)) results = pool.map(partial(process_images, detector=detector, - confidence_threshold=confidence_threshold), image_batches) + confidence_threshold=confidence_threshold,image_size=image_size), + image_batches) results = list(itertools.chain.from_iterable(results)) @@ -434,7 +449,8 @@ def load_and_run_detector_batch(model_file, image_file_names, checkpoint_path=No return results -def write_results_to_file(results, output_file, relative_path_base=None, detector_file=None, info=None): +def write_results_to_file(results, output_file, relative_path_base=None, + detector_file=None, info=None): """ Writes list of detection results to JSON output file. Format matches: @@ -503,9 +519,10 @@ def write_results_to_file(results, output_file, relative_path_base=None, detecto checkpoint_frequency = -1 results = None ncores = 1 - use_image_queue = True + use_image_queue = False quiet = False image_dir = r'G:\temp\demo_images\ssmini' + image_size = None image_file_names = image_file_names = ImagePathUtils.find_images(image_dir, recursive=False) # image_file_names = image_file_names[0:2] @@ -521,7 +538,8 @@ def write_results_to_file(results, output_file, relative_path_base=None, detecto results=results, n_cores=ncores, use_image_queue=use_image_queue, - quiet=quiet) + quiet=quiet, + image_size=image_size) elapsed = time.time() - start_time @@ -555,6 +573,11 @@ def main(): '--quiet', action='store_true', help='Suppress per-image console output') + parser.add_argument( + '--image_size', + type=int, + default=None, + help=('Force image resizing to a (square) integer size (not recommended to change this)')) parser.add_argument( '--use_image_queue', action='store_true', @@ -693,7 +716,8 @@ def main(): results=results, n_cores=args.ncores, use_image_queue=args.use_image_queue, - quiet=args.quiet) + quiet=args.quiet, + image_size=args.image_size) elapsed = time.time() - start_time print('Finished inference for {} images in {}'.format( diff --git a/detection/tf_detector.py b/detection/tf_detector.py index 7837701a2..4b3eaef2c 100644 --- a/detection/tf_detector.py +++ b/detection/tf_detector.py @@ -99,7 +99,7 @@ def _generate_detections_one_image(self, image): return box_tensor_out, score_tensor_out, class_tensor_out - def generate_detections_one_image(self, image, image_id, detection_threshold): + def generate_detections_one_image(self, image, image_id, detection_threshold, image_size=None): """Apply the detector to an image. Args: @@ -114,6 +114,7 @@ def generate_detections_one_image(self, image, image_id, detection_threshold): - 'detections', which is a list of detection objects containing keys 'category', 'conf' and 'bbox' - 'failure' """ + assert image_size is None, 'Image sizing not supported for TF detectors' result = { 'file': image_id } From dbb1833be5206d1ae359d558cce09bdfcf1bd7f1 Mon Sep 17 00:00:00 2001 From: Dan Morris Date: Wed, 31 Aug 2022 16:10:20 -0700 Subject: [PATCH 12/17] repo additions --- megadetector.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/megadetector.md b/megadetector.md index 0ff757762..6187d2375 100644 --- a/megadetector.md +++ b/megadetector.md @@ -411,6 +411,10 @@ It's not quite as simple as "these platforms all run MegaDetector on your images * [Cam-WON](https://wildlifeobserver.net/) * [Zooniverse ML Subject Assistant](https://subject-assistant.zooniverse.org/#/intro) +### Other ways of running MegaDetector that don't fit easily into one of those categories + +* [FastAPI/Streamlit package for serving MD and visualizing results](https://github.com/abhayolo/megadetector-fastapi) + ## How do I use the results? See the ["How do people use MegaDetector results?"](https://github.com/microsoft/CameraTraps/blob/main/collaborations.md#how-people-use-megadetector-results) section of our "getting started" page. From 1bf18e7ab631423373664b5400c54eb7a8a18e74 Mon Sep 17 00:00:00 2001 From: Dan Morris Date: Sat, 3 Sep 2022 08:42:03 -0700 Subject: [PATCH 13/17] Update README.md --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index b41d5a60d..fdd1d942d 100644 --- a/README.md +++ b/README.md @@ -89,6 +89,7 @@ Here are a few of the organizations that have used MegaDetector... we're only li * The Nature Conservancy in California ([Animl platform](https://github.com/tnc-ca-geo/animl-frontend)) * Felidae Conservation Fund ([WildePod platform](https://wildepod.org/)) ([blog post](https://abhaykashyap.com/blog/ai-powered-camera-trap-image-annotation-system/)) * Alberta Biodiversity Monitoring Institute (ABMI) ([WildTrax platform](https://www.wildtrax.ca/)) ([blog post](https://wildcams.ca/blog/the-abmi-visits-the-zoo/)) +* Shan Shui Conservation Center ([blog post](https://mp.weixin.qq.com/s/iOIQF3ckj0-rEG4yJgerYw?fbclid=IwAR0alwiWbe3udIcFvqqwm7y5qgr9hZpjr871FZIa-ErGUukZ7yJ3ZhgCevs)) ([translated blog post](https://mp-weixin-qq-com.translate.goog/s/iOIQF3ckj0-rEG4yJgerYw?fbclid=IwAR0alwiWbe3udIcFvqqwm7y5qgr9hZpjr871FZIa-ErGUukZ7yJ3ZhgCevs&_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=en&_x_tr_pto=wapp)) * Irvine Ranch Conservancy ([story](https://www.ocregister.com/2022/03/30/ai-software-is-helping-researchers-focus-on-learning-about-ocs-wild-animals/)) * Wildlife Protection Solutions ([story](https://customers.microsoft.com/en-us/story/1384184517929343083-wildlife-protection-solutions-nonprofit-ai-for-earth)) * [TrapTagger](https://wildeyeconservation.org/trap-tagger-about/) From 76ac13900d7eec39dfadd2158ff20ed119d507e0 Mon Sep 17 00:00:00 2001 From: Dan Morris Date: Sun, 4 Sep 2022 10:20:57 -0700 Subject: [PATCH 14/17] Threshold validation in separate_detections_into_folders --- .../separate_detections_into_folders.py | 22 +++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/api/batch_processing/postprocessing/separate_detections_into_folders.py b/api/batch_processing/postprocessing/separate_detections_into_folders.py index dafb3c330..ca83b1b27 100644 --- a/api/batch_processing/postprocessing/separate_detections_into_folders.py +++ b/api/batch_processing/postprocessing/separate_detections_into_folders.py @@ -416,6 +416,10 @@ def separate_detections_into_folders(options): if 'detector_metadata' in results['info'] and \ 'typical_detection_threshold' in results['info']['detector_metadata']: default_threshold = results['info']['detector_metadata']['typical_detection_threshold'] + elif ('detector' not in results['info']) or (results['info']['detector'] is None): + print('Warning: detector version not available in results file, using MDv5 defaults') + detector_metadata = get_detector_metadata_from_version_string('v5a.0.0') + default_threshold = detector_metadata['typical_detection_threshold'] else: print('Warning: detector metadata not available in results file, inferring from MD version') detector_filename = results['info']['detector'] @@ -634,6 +638,24 @@ def main(): args_to_object(args, options) + def validate_threshold(v,name): + print('{} {}'.format(v,name)) + if v is not None: + assert v >= 0.0 and v <= 1.0, \ + 'Illegal {} threshold {}'.format(name,v) + + validate_threshold(args.threshold,'default') + validate_threshold(args.animal_threshold,'animal') + validate_threshold(args.vehicle_threshold,'vehicle') + validate_threshold(args.human_threshold,'human') + + if args.threshold is not None: + if args.animal_threshold is not None \ + and args.human_threshold is not None \ + and args.vehicle_threshold is not None: + raise ValueError('Default threshold specified, but all category thresholds also specified... not exactly wrong, but it\'s likely that you meant something else.') + + options.category_name_to_threshold['animal'] = args.animal_threshold options.category_name_to_threshold['person'] = args.human_threshold options.category_name_to_threshold['vehicle'] = args.vehicle_threshold From 5a943733ef6e990a30e1c6d4a1d46faa6810495f Mon Sep 17 00:00:00 2001 From: Dan Morris Date: Mon, 5 Sep 2022 09:24:14 -0700 Subject: [PATCH 15/17] Handling failed images in subset_json_detector_output --- api/batch_processing/data_preparation/manage_local_batch.py | 4 +++- .../postprocessing/subset_json_detector_output.py | 3 +++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/api/batch_processing/data_preparation/manage_local_batch.py b/api/batch_processing/data_preparation/manage_local_batch.py index 07b6f36a5..031013820 100644 --- a/api/batch_processing/data_preparation/manage_local_batch.py +++ b/api/batch_processing/data_preparation/manage_local_batch.py @@ -97,7 +97,7 @@ filename_base = os.path.join(base_output_folder_name, base_task_name) combined_api_output_folder = os.path.join(filename_base, 'combined_api_outputs') -postprocessing_output_folder = os.path.join(filename_base, 'postprocessing') +postprocessing_output_folder = os.path.join(filename_base, 'preview') os.makedirs(filename_base, exist_ok=True) os.makedirs(combined_api_output_folder, exist_ok=True) @@ -1252,6 +1252,8 @@ def remove_overflow_folders(relativePath): options.split_folders = True options.make_folder_relative = True + +# Reminder: 'n_from_bottom' with a parameter of zero is the same as 'bottom' options.split_folder_mode = 'bottom' # 'top', 'n_from_top', 'n_from_bottom' options.split_folder_param = 0 options.overwrite_json_files = False diff --git a/api/batch_processing/postprocessing/subset_json_detector_output.py b/api/batch_processing/postprocessing/subset_json_detector_output.py index 1f06fbe50..49632ba5d 100644 --- a/api/batch_processing/postprocessing/subset_json_detector_output.py +++ b/api/batch_processing/postprocessing/subset_json_detector_output.py @@ -165,6 +165,9 @@ def subset_json_detector_output_by_confidence(data, options): # iImage = 0; im = images_in[0] for iImage, im in tqdm(enumerate(images_in), total=len(images_in)): + if ('detections' not in im) or (im['detections'] is None): + continue + p_orig = im['max_detection_conf'] # Find all detections above threshold for this image From 1ef65fda4d65fd433fb048e8480ca9a8091e84c7 Mon Sep 17 00:00:00 2001 From: Dan Morris Date: Thu, 8 Sep 2022 09:16:56 -0700 Subject: [PATCH 16/17] Fixed harmless file deletion error on very small jobs --- detection/run_detector_batch.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/detection/run_detector_batch.py b/detection/run_detector_batch.py index fed7e2b87..8e75ecc25 100644 --- a/detection/run_detector_batch.py +++ b/detection/run_detector_batch.py @@ -729,7 +729,7 @@ def main(): write_results_to_file(results, args.output_file, relative_path_base=relative_path_base, detector_file=args.detector_file) - if checkpoint_path: + if checkpoint_path and os.path.isfile(checkpoint_path): os.remove(checkpoint_path) print('Deleted checkpoint file {}'.format(checkpoint_path)) From fd180470facaefaf29c9605347ba2399af7be0db Mon Sep 17 00:00:00 2001 From: songsparrow Date: Thu, 8 Sep 2022 09:24:11 -0700 Subject: [PATCH 17/17] Adding preliminary environment file for M1 Macs --- environment-detector-m1.yml | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 environment-detector-m1.yml diff --git a/environment-detector-m1.yml b/environment-detector-m1.yml new file mode 100644 index 000000000..13979af5d --- /dev/null +++ b/environment-detector-m1.yml @@ -0,0 +1,36 @@ +name: cameratraps-detector + +channels: + - conda-forge + - pytorch + +dependencies: + - nomkl + - python=3.8 + - Pillow=9.1.0 + - nb_conda_kernels + - ipykernel + - tqdm + - jsonpickle + - humanfriendly + - numpy + - matplotlib + - nb_conda_kernels + - ipykernel + - opencv + - requests + + # for running MegaDetector v4 + # - tensorflow>=2.0 + + # for running MegaDetector v5 + - pandas + - seaborn>=0.11.0 + - PyYAML>=5.3.1 + # - pytorch::pytorch=1.10.1 + # - pytorch::torchvision=0.11.2 + # - conda-forge::cudatoolkit=11.3 + # - conda-forge::cudnn=8.1 + +# the `nb_conda_kernels` and `ipykernel` packages are installed so that we +# can use Jupyter Notebooks with this environment as a kernel