About the create_lmdb_mp.py

[zrongcheng]

1. I created my dataset, my dataset had 200 folders(included val, total 120GB), but i got error, it's said that i didn't have enough memory; Thus, i want to know what the hardware configuration.
2. Did you test your code with VMAF? The video train with rgb or yuv420 whether get the different result by VMAF?

[zestloveheart]

for 1........do you meet the error when you create lmdb?

[xinntao]

1. The script reads all the images to the memory. If you want to read images one by one, you need to change some codes. Please refer to this.

2. No, we have not. Did you mean this by VMAF?

[zrongcheng]

1. Ok, thanks.
2. Yes, VMAF is a good assessment norm.

[xinntao]

Thanks, I will have a look at VMAF.

[crypticsymmetry]

What changes to create_lmdb_mp.py did you make for it to use less memory? Thanks!

[xinntao]

@crypticsymmetry For fast IO speed during training.

[crypticsymmetry]

sorry, i meant what changes to Create lmdb did he make to not load all images at one time, as i have a max of 32gb of RAM. i checked out the basicSR version but didnt understand how to implement it to EDVR's create lmdb file. thanks for your time

[xinntao]

@crypticsymmetry I will update a version with limited memory consumption tmr.

[xinntao]

@crypticsymmetry I have update the create_lmdb_mp.py. You can use it to generate the lmdb files.