Merge pull request microsoft#7 from AAU-SceneGraph/ResolveConflicts

Resolve conflicts between main and KERN_NEMESIS branches + updates

Dockerfile

@@ -0,0 +1,40 @@
+ARG CUDA="10.1"
+ARG CUDNN="7"
+
+FROM nvidia/cuda:${CUDA}-cudnn${CUDNN}-devel-ubuntu18.04
+
+RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
+
+WORKDIR /sgb
+
+# Temporary FIX
+# https://forums.developer.nvidia.com/t/notice-cuda-linux-repository-key-rotation/212771/8
+# TODO: Around May29th check if re-downloading the cuda image without this works
+RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub 2
+RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/7fa2af80.pub 2
+
+# install basics & create virtual environment
+
+RUN apt-get update -y \
+ && apt-get install -y apt-utils git curl ca-certificates bzip2 cmake tree htop bmon iotop g++ \
+ && apt-get install -y libglib2.0-0 libsm6 libxext6 libxrender-dev libyaml-dev nano wget graphviz graphviz-dev \
+ && apt-get install -y ffmpeg libgl1-mesa-glx software-properties-common python3 python3-pip python3-venv
+
+RUN pip3 install --upgrade pip 
+RUN apt-get install  -y libjpeg8-dev zlib1g-dev
+RUN pip3 install --ignore-installed pillow
+
+# install requirements
+
+COPY requirements.txt .
+RUN pip3 install -r requirements.txt
+
+# install latest PyTorch 1.7.1
+
+RUN pip3 install torch==1.7.1+cu101 torchvision==0.8.2+cu101 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html 
+
+# install PyTorch Detection
+ARG FORCE_CUDA="1"
+ENV FORCE_CUDA=${FORCE_CUDA}
+
+RUN update-alternatives --install /usr/bin/python python /usr/bin/python3 1

INSTALL.md

@@ -2,54 +2,18 @@
 
 ### Requirements:
 - PyTorch 1.7
-- torchvision
-- cocoapi
+- torchvision 0.8.2
+- pycocotools 2.0.4
 - yacs>=0.1.8
 - numpy>=1.19.5
-- matplotlib
+- matplotlib 3.3.4
 - GCC >= 4.9
-- OpenCV
+- OpenCV 4.5.5.64
 - CUDA >= 10.1
 
+The complete list of requirements is available in the document *requirements.txt*
 
-### Option 1: Step-by-step installation
-
-```bash
-# first, make sure that your conda is setup properly with the right environment
-# for that, check that `which conda`, `which pip` and `which python` points to the
-# right path. From a clean conda env, this is what you need to do
-
-conda create --name sg_benchmark python=3.7 -y
-conda activate sg_benchmark
-
-# this installs the right pip and dependencies for the fresh python
-conda install ipython h5py nltk joblib jupyter pandas scipy
-
-# maskrcnn_benchmark and coco api dependencies
-pip install ninja yacs>=0.1.8 cython matplotlib tqdm opencv-python numpy>=1.19.5
-
-conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=10.1 -c pytorch
-conda install -c conda-forge timm einops
-
-# install pycocotools
-conda install -c conda-forge pycocotools
-
-# install cityscapesScripts
-python -m pip install cityscapesscripts
-
-# install Scene Graph Detection
-git clone https://github.com/microsoft/scene_graph_benchmark
-cd scene_graph_benchmark
-
-# the following will install the lib with
-# symbolic links, so that you can modify
-# the files if you want and won't need to
-# re-build it
-python setup.py build develop
-
-
-```
-### Option 2: Docker Image (Requires CUDA, Linux only)
+### Docker Image (Requires CUDA, Linux only)
 
 Build image with defaults (`CUDA=10.1`, `CUDNN=7`, `FORCE_CUDA=1`):
 

README.md

@@ -1,69 +1,91 @@
 # Scene Graph Benchmark in PyTorch 1.7.1
 
+## 0. Prerequisites
 
-## 0. Requirements
+Docker is required to use the repository
 
-## 1. Setup for first-time use (Docker)
+## 1. Setup for first-time use
 
 - Clone repository: `git clone https://github.com/AU-Nebula/scene_graph_benchmark.git SGB`
 - Change directory: `cd SGB`
 
 If at least one GPU is available, follow section 1a, if not, follow 1b.
 
-### 1a. Setup for first-time use (Docker) - GPUs required
+### 1a. Setup for first-time use - GPUs required
 
 For an automatic setup, use the following command: `sh installation_steps.sh 1`.
 
 Manual setup:
 
-- Download *Visual Genome* metadata: `sh custom_files/download_VG.sh`
-- Download pre-trained model: `sh custom_files/download_pretrained.sh`
+- Download *Visual Genome* metadata: `sh scripts/download_VG.sh`
+- Download pre-trained model: `sh scripts/download_pretrained.sh`
 - Download NVIDIA Docker image: `docker pull nvidia/cuda:10.1-cudnn7-devel-ubuntu18.04`
-- Build Docker image: `docker build -t au/sgb:10.1-cudnn7-devel-ubuntu18.04 ./docker --build-arg FORCE_CUDA=1`
+- Build Docker image: `docker build -t au/sgb:10.1-cudnn7-devel-ubuntu18.04-pip3 . --build-arg FORCE_CUDA=1`
 
-Activation of the environments:
-- Run Docker image: `docker run -it -v $PWD:/sgb --gpus all au/sgb:10.1-cudnn7-devel-ubuntu18.04`
-- Activate *conda* environment: `conda activate sgb`
+Activation of the environment:
+- Run Docker image: `docker run -it --name "sgb-container" -v $PWD:/sgb --gpus all au/sgb:10.1-cudnn7-devel-ubuntu18.04-pip3`
 
-- Test model: `sh custom_files/run_validation.test.sh cuda`
-
-*Output test image can be found in directory:* `custom_images/francesco_example/3_out.jpg`
+Installation of the code *inside Docker*:
+- Install modules: `python setup.py build develop`
 
-### 1b. Setup for first-time use (Docker) - CPU-only case
+### 1b. Setup for first-time use - CPU-only case
 
 For an automatic setup, use the following command: `sh installation_steps.sh 0`.
 
 Manual setup:
 
-- Download *Visual Genome* metadata: `sh custom_files/download_VG.sh`
-- Download pre-trained model: `sh custom_files/download_pretrained.sh`
+- Download *Visual Genome* metadata: `sh scripts/download_VG.sh`
+- Download pre-trained model: `sh scripts/download_pretrained.sh`
 - Download NVIDIA Docker image: `docker pull nvidia/cuda:10.1-cudnn7-devel-ubuntu18.04`
-- Build Docker image: `docker build -t au/sgb:10.1-cudnn7-devel-ubuntu18.04 ./docker --build-arg FORCE_CUDA=0`
+- Build Docker image: `docker build -t au/sgb:10.1-cudnn7-devel-ubuntu18.04-pip3 . --build-arg FORCE_CUDA=0`
 
-Activation of the environments:
-- Run Docker image: `docker run -it -v $PWD:/sgb au/sgb:10.1-cudnn7-devel-ubuntu18.04`
-- Activate *conda* environment: `conda activate sgb`
+Activation of the environment:
+- Run Docker image: `docker run -it --name "sgb-container" -v $PWD:/sgb au/sgb:10.1-cudnn7-devel-ubuntu18.04-pip3`
 
-- Test model: `sh custom_files/run_validation.test.sh cpu`
-
-*Output test image can be found in directory:* `custom_images/francesco_example/3_out.jpg`
+Installation of the code *inside Docker*:
+- Install modules: `python setup.py build develop`
+
+### 2. Generate Scene Graphs from custom images
+
+Once the setup is completed, to test the model use this command inside the running container:
+
+`./scripts/run_validation_image.sh -d `**`DEVICE`**` -i Example/pexels-daniel-frese-574177.jpg`
+
+Substitute **`DEVICE`** with `cuda` if you set up the repository through 1a, otherwise substitute it with `cpu`
+
+If you wish to use a different image, specify the **`PATH`** after the `-i` option:
+
+`./scripts/run_validation_image.sh -d `**`DEVICE`**` -i `**`PATH`**
+
+If you wish to set thresholds for the objects and the relations, use the following command:
+
+`./scripts/run_validation_image.sh -d `**`DEVICE`**` -i `**`PATH`**` -o `**`Obj_th`**` -r `**`Rel_th`**
+
+where **`Obj_th`** and **`Rel_th`** are numbers in the interval [0,1) and are respectively the object and relation thresholds. Objects detected with confidence less than or equal to **`Obj_th`** are filtered out. The same goes with relations and **`Rel_th`**.
+
+An example of input and output content can be found in the directory: `Example`
+
+In particular:
+- `pexels-daniel-frese-574177.jpg`: sample input image
+- `pexels-daniel-frese-574177_annotated.jpg`: output annotated image
+- `pexels-daniel-frese-574177_causal_tde.json`: document containing the graph in json format
+- `pexels-daniel-frese-574177_graph.pdf`: pdf file containing the schematized graph
 
-## 2. Generate Scene Graphs from custom images
+### 3. Starting and stopping the container
 
-### Work in progress
+Once you have finished testing the model, you can stop the container either using *ctrl+d* inside Docker or using this command in another shell:
 
-*Currently studying HOW and WHERE output SGs are generated...*
+`docker stop sgb-container`
 
-Check [mini_tsv](https://github.com/AU-Nebula/scene_graph_benchmark/tree/main/tools/mini_tsv) README 
+To restart the container use this command:
 
-#
+`docker start -a sgb-container`
 
-You can find below the original README
+---
 
-#
 **This project is based on [maskrcnn-benchmark](https://github.com/facebookresearch/maskrcnn-benchmark)**
 
-![alt text](demo/R152FPN_demo.png "from https://storage.googleapis.com/openimages/web/index.html")
+![alt text](tools/webcam/R152FPN_demo.png "from https://storage.googleapis.com/openimages/web/index.html")
 
 
 ## Highlights