Any pre-processed data for training and test

[yangengt123]

Hi,
Thanks for sharing the code of your great work.

I am trying to run the training code, but I found the whole ScanNet dataset is rather than large. May I ask if it is necessary to download all the .sens from the dataset? Is it possible to get access to your per-processed data, or to train the network with part of this dataset?

BTW, a small contribution, if any one uses Cuda 9 + RTX 2080 GPU, the arch should be sm_70 for the nvcc compilement. A link for other gpu types: https://arnon.dk/matching-sm-architectures-arch-and-gencode-for-various-nvidia-cards/

[art-programmer]

We are processing that. I will keep you updated. And thank you for the pointer!

[yangengt123]

Hi, I was trying to run the training code, and I found in

planercnn/datasets/plane_dataset.py

Line 94 in 557583f

with open(self.dataFolder + '/scannetv2-labels.combined.tsv') as info_file:

it asked a .tsv file named scannetv2-labels.combined.tsv is needed, and in

planercnn/datasets/plane_dataset.py

Line 43 in 557583f

if not os.path.exists(scenePath + '/' + scene_id + '.txt') or not os.path.exists(scenePath + '/annotation/planes.npy'):

a .txt file named with the scene_id seems required to be under each scene's directory.

May I ask where I can get access to these files?

[art-programmer]

They are from the original ScanNet dataset. You need to download the data and uncompress the .sens file (maybe delete the .sens file after uncompression to save space).

[yangengt123]

Thanks.
Just for someone else who is not familiar with ScanNet dataset, to get thees files, please use the argument --type .sens .txt and --label_map during download.

By the way, how can I run the training with multiple gpu? I cannot find any line contain the function torch.nn.DataParallel(model).cuda(), which I usually use for multi-gpu training. I tried to uncommment

planercnn/config.py

Line 360 in 41a0f63

# Uncomment to train on 8 GPUs (default is 1)

but it seems does not work as well.

[dreamPoet]

Sorry but is there any way we can run ScanNet original sens reader.py for all sens files scattered in different scene files? It looks like I cannot move all .sens file into one directory as it is important to map to corresponding annotations in later merging operation.

[yangengt123]

Hi @dreamPoet , I just simply set the --filename as the path/to/Scannet_data/scans/, and it seems work well for me.

[dreamPoet]

Hi @yangengt123, do you simply use --filename path/to/scans/ ？ I get the error Is a directory: '../../ScanNet/scans/'. If I use ../../ScanNet/scans/*/*.sens, it gives error unrecognized arguments: scans/scene0000_01/scene0000_01.sens...

[yangengt123]

Yeah, the only difference is I used absolute path, something like /hdd/Scannet_data/scans/, and I did not meet any trouble to parse it. But I think it is always possible to look into the SensorData.py to check which path the load function is using.

[dreamPoet]

Thanks.
Just for someone else who is not familiar with ScanNet dataset, to get thees files, please use the argument --type .sens .txt and --label_map during download.

By the way, how can I run the training with multiple gpu? I cannot find any line contain the function torch.nn.DataParallel(model).cuda(), which I usually use for multi-gpu training. I tried to uncommment

planercnn/config.py

Line 360 in 41a0f63

# Uncomment to train on 8 GPUs (default is 1)

but it seems does not work as well.

Thank you, but have you solve the problem of multi-GPU training now? I meet the same problem.

[dreamPoet]

By the way I use srun with gpu:8 and -n8 as bash command.

[dreamPoet]

And there are three parameters related to batch size, one in config, one in option, and one for dataset loader...Frankly, I am a bit of confused...

[wullish]

@dreamPoet did you solve the .sens files extraction problem using reader.py? I'm in the exactly same problem. Any idea?

Best.

[wullish]

Well, I could solve it using python 2.7 but calling one .sens file per once. And dont forget about the reader.py options arguments such as --export_depth_images, --export_color_images, etc.

[yangengt123]

That is right, I use python2.7. Maybe this is the reason I did not meet any problem.

By the way, may I ask if you can still download the data from ScanNet recently? I tried to download the data to a new server but it seems my download script cannot work.

[wullish]

Ah thanks.
Yes, I've downloaded the whole data last week. Maybe they've changed the urls. I think you can contact scannet@googlegroups.com and ask them about the script.

[yangengt123]

yeah, maybe I should. Thanks.
I will close this issue now.
Please let me know if any pre-processed data become available.