Docker and bash support

.gitignore

@@ -0,0 +1,9 @@
+# ignore torch models
+*.pt
+*.pth
+
+# ignore videos
+*.mp4
+*.MP4
+*.MOV
+*.avi

README.md

@@ -30,6 +30,11 @@
     <a href="https://go.vision.in.tum.de/tandem">Project Page</a>
 </p>
 
+## Overview
+This repository is mostly based on [this](https://github.com/rancheng/tandem) repository by rancheng and official [TANDEM](https://github.com/tum-vision/tandem) repository.. Main changes include adding a `custom_dockerfile` to complete the existing `image` and adding support for automatically cloning and building [tandem](https://github.com/tum-vision/tandem) repository.
+
+Instruction can be found inside `/docker`.
+
 ## Code and Data
 - [x] 📣 C++ code released before Christmas! Please check [tandem/](tandem/).
 - [x] 📣 CVA-MVSNet released! Please check [cva_mvsnet/](cva_mvsnet/).
@@ -38,6 +43,7 @@
 - [x] Release of the ScanNet-trained model 
 - [ ] Docker image for TANDEM. **Contributions are highly welcomed!**
 
+
 ### Abstract
 <p align="justify">In this paper, we present TANDEM a real-time monocular tracking and dense mapping framework. For pose estimation, TANDEM performs photometric bundle adjustment based on a sliding window of keyframes. To increase the robustness, we propose a novel tracking front-end that performs dense direct image alignment using depth maps rendered from a global model that is built incrementally from dense depth predictions. To predict the dense depth maps, we propose Cascade View-Aggregation MVSNet (CVA-MVSNet) that utilizes the entire active keyframe window by hierarchically constructing 3D cost volumes with adaptive view aggregation to balance the different stereo baselines between the keyframes. Finally, the predicted depth maps are fused into a consistent global map represented as a truncated signed distance function (TSDF) voxel grid. Our experimental results show that TANDEM outperforms other state-of-the-art traditional and learning-based monocular visual odometry (VO) methods in terms of camera tracking. Moreover, TANDEM shows state-of-the-art real-time 3D reconstruction performance.</p>
 

calib/evo.txt

@@ -0,0 +1,4 @@
+748.3980684335114 744.46389846898 483.1141835180486 268.1011960644272 0
+1920 1080
+crop
+640 480

calib/evo_orig.txt

@@ -0,0 +1,4 @@
+1496.7880684335114 1488.93389846898 966.2141835180486 537.2011960644272 0
+1920 1080
+crop
+640 480

calib/gopro.txt

@@ -0,0 +1,4 @@
+1008.5922377108149 1023.4650230516394 965.6042551747383 546.7564480605071 -0.02769386261552577 0.019314637296003356
+1920 1080
+crop
+640 480

calib/gopro_wide.txt

@@ -0,0 +1,4 @@
+985.0634313458714 999.0480986801257 964.5133546853123 547.5985161076464 -0.34480971439143393 0.2605143210518555
+1920 1080
+crop
+640 480

calib/iphone.txt

@@ -0,0 +1,4 @@
+1678.8298074165925 1674.773088668592 985.013786649560 540.0966152878177 0.14774896028264306 -0.8331999100095582
+1920 1080
+crop
+640 480

calib/tmp.txt

@@ -0,0 +1,4 @@
+1496.7880684335114 1488.93389846898 966.2141835180486 537.2011960644272 0
+1920 1080
+crop
+640 480

calib/tum.txt

@@ -0,0 +1,4 @@
+0.349153000000000	0.436593000000000	0.493140000000000	0.499021000000000	0.933271000000000
+1280 1024
+0.4 0.53 0.5 0.5 0
+640 480

cva_mvsnet/export_model.py

@@ -128,6 +128,7 @@ def main(args: argparse.Namespace):
     selection_index = [inverse_view_index[i] for i in view_index]
 
     model = tandem.cva_mvsnet
+
     del tandem
 
     for s in model.stages:
@@ -193,7 +194,7 @@ def main(args: argparse.Namespace):
         traced_script_module = torch.jit.trace(model, example_inputs=example_inputs, check_trace=True,)
         if args.jit_freeze:
             print("--- jit_freeze")
-            traced_script_module = torch.jit.freeze(traced_script_module)
+            traced_script_module = torch.jit.freeze(traced_script_module.eval())
             assert len(list(traced_script_module.named_parameters())) == 0
 
         if args.jit_run_frozen_optimizations:

docker/README.md

@@ -0,0 +1,141 @@
+# Tandem Docker
+
+This is the docker environment setup tutorial for `TUM Tandem`, it supports remote `OpenGL` display, thus you can render `Pangolin window` with remote ssh.
+
+## Build Image
+
+Run the following commands to build the image:
+
+```shell
+cd docker
+./build-dokcer-image.bash
+```
+
+and run
+
+```shell
+./run-docker-container.bash
+```
+
+to run the container.
+
+Please note that you can customize your own `ssh port` by modifying the `run-docker-container.bash` file:
+
+```shell
+docker run --rm \
+  --name tandem \
+  --ipc=host \
+  --gpus all \
+  --privileged \
+  -p 3751:22 \         <======= please choose your own port
+  -v /tmp/.X11-unix:/tmp/.X11-unix:rw \
+```
+
+Remove the `--rm` flag in case you don't want to erase all the changes and data after you stop the `container`.
+
+You can also pull the built image from `dockerhub.io`:
+
+```shell
+docker pull pytholic/tumtandem:tumtandem
+```
+
+## Run
+
+Then in a new remote or local terminal, run the following command to connect the remote environment:
+
+```shell
+ssh -p 3751 work@xx.xx.xx.xx
+# default password: work123
+```
+
+Or if you already `docker run` run once, you can simple do:
+
+```shell
+docker start <container name>
+```
+
+Add you camera calibration file in the specified [format](https://github.com/pytholic/tandem/blob/master/tandem/README.md).
+
+Run the following comand in the terminal.
+```shell
+build/bin/tandem_dataset \
+      preset=dataset \
+      result_folder=path/to/save/results \
+      files=path/to/scene/images \
+      calib=path/to/scene/camera.txt \
+      mvsnet_folder=exported/tandem \
+      mode=1
+```
+
+### Bash script
+Alternatively, you can use bash scripts insice `/utils.
+
+Put your `input` video and `calib.txt` in the same folder as `run_video.sh`. Then run the following command in the terminal.
+
+Set executable permission on the script.
+```shell
+chmod +x run_video.sh
+```
+
+```shell
+./run_video.sh --input <input video path> 
+```
+
+If your input are `images`, then use `run_image.sh`
+```shell
+./run_image.sh --input <path to images folder> 
+```
+
+
+## Note
+
+### Display
+In case you get some `X11` related error, rememebr that you need to set display inside your container same as your host display.
+```shell
+# In the host shell
+echo $DISPLAY
+:1
+
+# In the container shell
+echo $DISPLAY
+:0
+
+export DISPLAY=:1
+```
+
+If `docker start` is not working, run with `-ai` flag.
+```shell
+docker start -ai tandem
+```
+Run the following if the error is related to `X11` error.
+```shell
+xhost + local:
+```
+Ref -> https://stackoverflow.com/questions/73490184/sudo-nautilus-gives-authorization-required-but-no-authorization-protocol-specif
+
+### Input size
+Currently `cva-mvsnet` model supports `640x480` input size. If you want to input different size, follow instructions [here](https://github.com/pytholic/tandem/tree/master/cva_mvsnet) to export the model with desired dimensions. Update the path while running tandem accordingly.
+
+### Nvidia container toolkit
+In case you face some `GPU` supported related issues, rememebr that you need to install [nvidia-container-toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html) on your host.
+
+```shell
+distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
+      && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
+      && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
+            sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
+            sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
+
+distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
+      && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
+      && curl -s -L https://nvidia.github.io/libnvidia-container/experimental/$distribution/libnvidia-container.list | \
+         sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
+         sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
+
+sudo apt-get update
+sudo apt-get install -y nvidia-docker2
+sudo systemctl restart docker
+
+# Test
+sudo docker run --rm --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi
+```

docker/build-docker-image.bash

@@ -0,0 +1,19 @@
+#!/bin/bash
+
+file_dir=`dirname $0`
+
+# get parameter from system
+user=`id -un`
+group=`id -gn`
+uid=`id -u`
+gid=`id -g`
+
+# --build-arg http_proxy=http://10.141.6.84:7890 \
+# --build-arg https_proxy=http://10.141.6.84:7890 \
+
+# build docker images
+docker build -t pytholic/tandem -f dockerfile . \
+    --build-arg USER=${user} \
+    --build-arg UID=${uid} \
+    --build-arg GROUP=${group} \
+    --build-arg GID=${gid} \

docker/config/terminator/config

@@ -0,0 +1,30 @@
+[global_config]
+  title_receive_bg_color = "#64d8ff"
+  enabled_plugins = TerminalShot, LaunchpadCodeURLHandler, APTURLHandler, Logger, LaunchpadBugURLHandler
+  title_transmit_bg_color = "#15b0dc"
+[keybindings]
+  switch_to_tab_1=<Alt>1
+  switch_to_tab_2=<Alt>2
+  switch_to_tab_3=<Alt>3
+  switch_to_tab_4=<Alt>4
+  switch_to_tab_5=<Alt>5
+  switch_to_tab_6=<Alt>6
+  switch_to_tab_7=<Alt>7
+[profiles]
+  [[default]]
+    use_system_font = False
+    background_darkness = 0.75
+    background_type = image
+    scroll_background = False
+    font = Monospace 12
+    scrollback_infinite = True
+    use_custom_command = False
+[layouts]
+  [[default]]
+    [[[child1]]]
+      type = Terminal
+      parent = window0
+    [[[window0]]]
+      type = Window
+      parent = ""
+[plugins]