Self-supervised Aux Tasks in pointnav and objectnav

[twni2016]

This PR includes:

• Provide logging in one folder option and make it as default
• Provide a base class VisualNavActorCritic to be inherited from different downstream navigation model classes (i.e. code refactoring), and also add the modified ResNet18 from https://github.com/joel99/habitat-pointnav-aux
• Introduce self-supervised aux tasks and fusion models based on the code: https://github.com/joel99/habitat-pointnav-aux , including inverse dynamics, temporal distance, and CPC|A.
• Update corresponding experiment config files in pointnav and objectnav

I tested the change by running commands:

python main.py projects/objectnav_baselines/experiments/robothor/objectnav_robothor_rgb_rawgru_ddppo.py \
    -o experiment_output/objectnav/ --seed 302
python main.py projects/pointnav_baselines/experiments/robothor/pointnav_robothor_rgb_simpleconvgru_ddppo.py\
    -o experiment_output/pointnav/ --seed 302

[lgtm-com]

This pull request introduces 11 alerts when merging 4a4fd5f into 894c2ce - view on LGTM.com

new alerts:

• 9 for Unused import
• 2 for Unused local variable

[Lucaweihs]

Awesome work Tianwei! I'm always impressed when I see the amount of work you put in and the code you produced. I've left a lot of comments but they are mostly about naming and adding doc strings. One thing: there were a few instances where a change broke backwards compatibility/behavior. In general we really try to prevent this as much as possible so it would be great to find solutions that make new behavior opt-in rather than opt-out (if you feel strongly about making a new behavior the new default then please add an argument to this thread).

[lgtm-com]

This pull request introduces 13 alerts when merging fee8003 into eb2c12e - view on LGTM.com

new alerts:

• 11 for Unused import
• 2 for Unused local variable

[twni2016]

Most of comments have been resolved. I've refactored the logging by using save_dir_fmt argument.
Remaining issues:

• Self-supervised Aux Tasks in pointnav and objectnav #319 (comment): where to save test results given a checkpoint
• Self-supervised Aux Tasks in pointnav and objectnav #319 (comment): check if compatible to previous checkpoints

[twni2016]

I use the following script for testing:

python main.py projects/objectnav_baselines/experiments/robothor/objectnav_robothor_rgb_unfrozenresnet_gru_ddppo.py \
    -s 302 --save_dir_fmt NESTED \
    -o experiment_output/objectnav/
python main.py projects/pointnav_baselines/experiments/robothor/pointnav_robothor_depth_simpleconvgru_ddppo.py \
    -s 302 --save_dir_fmt NESTED \
    -o experiment_output/pointnav/ 

Shall we also add these running scripts into somewhere in the doc?

[lgtm-com]

This pull request introduces 13 alerts when merging 8479e03 into eb2c12e - view on LGTM.com

new alerts:

• 11 for Unused import
• 2 for Unused local variable

[Lucaweihs]

Great updates! A few small little things to clean up left but almost there.

[lgtm-com]

This pull request introduces 13 alerts when merging 21bf39d into eb2c12e - view on LGTM.com

new alerts:

• 11 for Unused import
• 2 for Unused local variable

[twni2016]

These LGTM alerts are not coming from my code. Shall I resolve them? @Lucaweihs

[Lucaweihs]

@twni2016 - We might be looking at different LGTM pages but you can find some alerts in your new files here. These are mostly about unused imports but there also appear to be 2 instances of unused local variables.

[twni2016]

I see. I clicked the alert button in LGTM webpage which shows other lines.

[lgtm-com]

This pull request introduces 13 alerts when merging 074e5ac into eb2c12e - view on LGTM.com

new alerts:

• 11 for Unused import
• 2 for Unused local variable

[twni2016]

@Lucaweihs I have fixed the key mismatch issue in objectnav checkpoint. The script

python main.py projects/objectnav_baselines/experiments/robothor/objectnav_robothor_rgbd_resnetgru_ddppo.py \
    -s 1 --save_dir_fmt NESTED --infer_output_dir --eval -i \
    -c pretrained_model_ckpts/robothor-objectnav-challenge-2021/Objectnav-RoboTHOR-RGBD-ResNetGRU-DDPPO/2021-02-09_22-35-15/exp_Objectnav-RoboTHOR-RGBD-ResNetGRU-DDPPO_0.2.0a_300M__stage_00__steps_000170207237.pt

But, I cannot find the experiment config robothor-pointnav-rgb-resnet for pointnav checkpoint, so I cannot run it. By the way, the wget script pretrained_model_ckpts/download_navigation_model_ckpts.sh download the ckpt into pretrained_model_ckpts for objectnav while into the root dir for pointnav. I think they should be consistent.

[lgtm-com]

This pull request introduces 5 alerts when merging 2d8c480 into eb2c12e - view on LGTM.com

new alerts:

• 3 for Unused import
• 2 for Unused local variable

[Lucaweihs]

There are a few straggling LGTM.com alerts. I've highlighted them here. They should be quick fixes, we're good to merge after these are in.

[lgtm-com]

This pull request introduces 2 alerts when merging 387d9d8 into eb2c12e - view on LGTM.com

new alerts:

• 2 for Unused import