[Feature]Support NeRF-Det

[Yanyirong]

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

Support the MultiViewPipeline and ScanNetMultiViewDataset used in NeRF-Det

Modification

1. Modify the function to generate a new format of pkl
2. Support the MultiVIewPipeline
3. Support the ScanNetMultiViewDataset

BC-breaking (Optional)

Does the modification introduce changes that break the back-compatibility of the downstream repos?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.

Checklist

1. Pre-commit or other linting tools are used to fix the potential lint issues.
2. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
3. If the modification has potential influence on downstream projects, this PR should be tested with downstream projects.
4. The documentation has been modified accordingly, like docstring or example tutorials.

[CLAassistant]

All committers have signed the CLA.

[OrangeSodahub]

Good

[Yanyirong]

Here is a version which can inference correctly.
The test result of nerfdet_res50_2x_low_res_depth.py is as follow:

The training module is still under development.

[ZwwWayne]

We can pack the code for nerfdet into the projects directory.

[ZwwWayne]

Should update readme.

[Yanyirong]

Here is the training result of resnet50 and with depth information.

10/20 01:30:07 - mmengine - INFO - Exp name: nerfdet_res50_2x_low_res_depth_20231017_171142
10/20 01:30:07 - mmengine - INFO - Saving checkpoint at 12 epochs
10/20 01:36:28 - mmengine - INFO - Epoch(val) [12][50/78]    eta: 0:03:30  time: 7.5244  data_time: 7.2881  memory: 9993  
10/20 01:40:26 - mmengine - INFO - 
+----------------+---------+---------+---------+---------+
| classes        | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+----------------+---------+---------+---------+---------+
| chair          | 0.7477  | 0.8187  | 0.4315  | 0.5234  |
| table          | 0.5771  | 0.7686  | 0.3805  | 0.5200  |
| sofa           | 0.7369  | 0.8660  | 0.4176  | 0.5361  |
| garbagebin     | 0.3199  | 0.5642  | 0.1266  | 0.2792  |
| bookshelf      | 0.4468  | 0.7792  | 0.1637  | 0.3636  |
| curtain        | 0.2828  | 0.5522  | 0.0534  | 0.1194  |
| cabinet        | 0.3816  | 0.6559  | 0.1480  | 0.3038  |
| picture        | 0.0283  | 0.1532  | 0.0018  | 0.0360  |
| window         | 0.1821  | 0.4184  | 0.0139  | 0.0780  |
| showercurtrain | 0.2924  | 0.6071  | 0.0526  | 0.2143  |
| bed            | 0.8458  | 0.8765  | 0.7403  | 0.7778  |
| counter        | 0.4900  | 0.7692  | 0.1471  | 0.2885  |
| desk           | 0.7027  | 0.9291  | 0.4773  | 0.6850  |
| door           | 0.3267  | 0.5546  | 0.0830  | 0.2163  |
| refrigerator   | 0.5540  | 0.6316  | 0.2123  | 0.3333  |
| sink           | 0.4798  | 0.5510  | 0.2521  | 0.3265  |
| toilet         | 0.9185  | 0.9310  | 0.6504  | 0.7069  |
| bathtub        | 0.7987  | 0.8710  | 0.4073  | 0.4839  |
+----------------+---------+---------+---------+---------+
| Overall        | 0.5062  | 0.6832  | 0.2644  | 0.3773  |
+----------------+---------+---------+---------+---------+
10/20 01:40:26 - mmengine - INFO - Epoch(val) [12][78/78]    table_AP_0.25: 0.5771  sofa_AP_0.25: 0.7369  
garbagebin_AP_0.25: 0.3199  bookshelf_AP_0.25: 0.4468  chair_AP_0.25: 0.7477  curtain_AP_0.25: 0.2828  
window_AP_0.25: 0.1821  desk_AP_0.25: 0.7027  cabinet_AP_0.25: 0.3816  bed_AP_0.25: 0.8458  door_AP_0.25: 0.3267  
showercurtrain_AP_0.25: 0.2924  counter_AP_0.25: 0.4900  picture_AP_0.25: 0.0283  bathtub_AP_0.25: 0.7987  
refrigerator_AP_0.25: 0.5540  sink_AP_0.25: 0.4798  toilet_AP_0.25: 0.9185  mAP_0.25: 0.5062  table_rec_0.25: 0.7686  
sofa_rec_0.25: 0.8660  garbagebin_rec_0.25: 0.5642  bookshelf_rec_0.25: 0.7792  chair_rec_0.25: 0.8187  
curtain_rec_0.25: 0.5522  window_rec_0.25: 0.4184  desk_rec_0.25: 0.9291  cabinet_rec_0.25: 0.6559  
bed_rec_0.25: 0.8765  door_rec_0.25: 0.5546  showercurtrain_rec_0.25: 0.6071  counter_rec_0.25: 0.7692  
picture_rec_0.25: 0.1532  bathtub_rec_0.25: 0.8710  refrigerator_rec_0.25: 0.6316  sink_rec_0.25: 0.5510  
toilet_rec_0.25: 0.9310  mAR_0.25: 0.6832  table_AP_0.50: 0.3805  sofa_AP_0.50: 0.4176  garbagebin_AP_0.50: 0.1266  
bookshelf_AP_0.50: 0.1637  chair_AP_0.50: 0.4315  curtain_AP_0.50: 0.0534  window_AP_0.50: 0.0139  
desk_AP_0.50: 0.4773  cabinet_AP_0.50: 0.1480  bed_AP_0.50: 0.7403  door_AP_0.50: 0.0830  showercurtrain_AP_0.50: 0.0526  
counter_AP_0.50: 0.1471  picture_AP_0.50: 0.0018  bathtub_AP_0.50: 0.4073  refrigerator_AP_0.50: 0.2123  
sink_AP_0.50: 0.2521  toilet_AP_0.50: 0.6504  mAP_0.50: 0.2644  table_rec_0.50: 0.5200  sofa_rec_0.50: 0.5361  
garbagebin_rec_0.50: 0.2792  bookshelf_rec_0.50: 0.3636  chair_rec_0.50: 0.5234  curtain_rec_0.50: 0.1194  
window_rec_0.50: 0.0780  desk_rec_0.50: 0.6850  cabinet_rec_0.50: 0.3038  bed_rec_0.50: 0.7778  
door_rec_0.50: 0.2163  showercurtrain_rec_0.50: 0.2143  counter_rec_0.50: 0.2885  picture_rec_0.50: 0.0360  
bathtub_rec_0.50: 0.4839  refrigerator_rec_0.50: 0.3333  sink_rec_0.50: 0.3265  toilet_rec_0.50: 0.7069  
mAR_0.50: 0.3773  data_time: 7.1716  time: 7.4085

The mAP is lower than the testing result.
Maybe this is the same problem with Depth makes performance worse

[Tai-Wang]

Don't change this script out of projects