Hi! Could you provide more details on the dataset for training ?

[GabbySuwichaya]

Thanks for your kind support last time and thank you very much for sharing the training script... It is quite interesting for me.
Here, I would like to kindly ask about the data for training.
As I have tried to follow the instruction to download the data from https://github.com/lzx551402/GL3D

• I have a question on which of these three datasets (1) gl3d_imgs, (2) gl3d_raw_imgs (3) gl3d_blended_images from https://github.com/lzx551402/GL3D#downloads ...... are to be downloaded ... or all of them ?

• I have downloaded gl3d_raw_imgs... However, I received the error (below).... Does this mean that I did not download correctly? Or that I have downloaded the wrong dataset?

• My setting for gl3d.yaml file is as follows. Should rawdata_dir be the cloned directory of https://github.com/lzx551402/GL3D ? I am very sorry as this is not what you wrote in the instruction. The reason that I thought that this maybe the GL3D cloned directory is because dump.py also looks for GL3D/data/list/comb/imageset_train.txt... :

data_name: gl3d_train
rawdata_dir: /mnt/HDD4TB2/GL3D   
feature_dump_dir: /mnt/HDD4TB3/SGMNet/gl3d_desc_dir
dataset_dump_dir: /mnt/HDD4TB3/SGMNet/gl3d_dataset_dir

The error:

python dump.py --config_path configs/gl3d.yaml
dump.py:20: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  config = yaml.load(f)
Formatting data...
  0%|                                                                                                                                                     | 0/109 [00:00<?, ?it/s]
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/gabby-suwichaya/anaconda3/envs/sgmnet/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/home/gabby-suwichaya/anaconda3/envs/sgmnet/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/mnt/HDD4TB3/SGMNet/datadump/dumper/gl3d_train.py", line 147, in format_seq
    pair_list=np.loadtxt(os.path.join(seq_dir,'geolabel','common_track.txt'),dtype=float)[:,:2].astype(int)
  File "/home/gabby-suwichaya/anaconda3/envs/sgmnet/lib/python3.7/site-packages/numpy/lib/npyio.py", line 1067, in loadtxt
    fh = np.lib._datasource.open(fname, 'rt', encoding=encoding)
  File "/home/gabby-suwichaya/anaconda3/envs/sgmnet/lib/python3.7/site-packages/numpy/lib/_datasource.py", line 193, in open
    return ds.open(path, mode, encoding=encoding, newline=newline)
  File "/home/gabby-suwichaya/anaconda3/envs/sgmnet/lib/python3.7/site-packages/numpy/lib/_datasource.py", line 533, in open
    raise IOError("%s not found." % path)
OSError: /mnt/HDD4TB2/GL3D/data/586326ad712e276146904571/geolabel/common_track.txt not found.
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "dump.py", line 27, in <module>
    dataset.format_dump_data()
  File "/mnt/HDD4TB3/SGMNet/datadump/dumper/gl3d_train.py", line 244, in format_dump_data
    pool.map(self.format_seq,indices)
  File "/home/gabby-suwichaya/anaconda3/envs/sgmnet/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/home/gabby-suwichaya/anaconda3/envs/sgmnet/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: /mnt/HDD4TB2/GL3D/data/586326ad712e276146904571/geolabel/common_track.txt not found.

[vdvchen]

Hi,

1. For GL3D donwloading, I think these data need to be download:

gl3d_imgs
gl3d_cams
gl3d_depths
gl3d_ct

and these data are organized as

data
└──
├── undist_images/*
├── geolabel/common_track.txt&cameras.txt
├── depths/*.pfm
└── basenamest.txt

2. Yes, the raw_datadir in readme is cloned directory of https://github.com/lzx551402/GL3D, sorry for the confusion. The error you met is due to lack of gl3d_ct and gl3d_ct.

I will add a more detaild instructions later on. Please tell me if you met any other issues in data generation or training,

Thanks.

[GabbySuwichaya]

Thanks.. So, could you please confirm how to download these data?
gl3d_imgs
gl3d_cams
gl3d_depths
gl3d_ct

Is it to use download_data.sh? But could you provide what is the Chunk starts and end ?
For example, is it to download like this ..... ?

bash download_data.sh gl3d_imgs 0 125
bash download_data.sh gl3d_cams 0 125
bash download_data.sh gl3d_depths 0 125
bash download_data.sh gl3d_ct 0 125

[vdvchen]

Yes, these data are downloaded with download_data.sh script, but the chunk size are different (you may find them in GL3D repo).

The download command should be

bash download_data.sh gl3d_imgs 0 125
bash download_data.sh gl3d_cams 0 0
bash download_data.sh gl3d_depths 0 59
bash download_data.sh gl3d_ct 0 0