pretreatment.py is causing dealock error on OUMVLP dataset

[gosiqueira]

System information (version)

• Pytorch => 1.10
• Operating System / Platform =>Ubuntu 20.04
• Cuda => Cuda 11.5

Detailed description

pretreatment.py for OUMVLP dataset results in random deadlocks.
Currently you are making asynchronous calls using a thread pool and storing the result e ordinary Python list. Thus, when I execute the code to convert OUMVLP files to pickles it's happening random deadlocks during the code execution.

I forked the directory and I'm trying to solve this myself to make a PR, bit I believe it's relevant to keep you apart of this problem.

Steps to reproduce

Steps:

1. Execute the code for OUMVLP dataset using personal machine processor. The error occurs in a non-deterministic fashion.

Issue submission checklist

• I checked the problem with documentation, FAQ, issues, and have not found solution.

[chuanfushen]

Awesome!

we also find this issue, but haven't pointed it out at docs.

Thanks for letting people know, and feel free to make a PR! Thanks.

[jackhanyuan]

I found that some other format files were generated in the OUMVLP_rearranged folder, so the processing would randomly break.

Just filtering the png files will solve this problem.

jackhanyuan@046083a

[chuanfushen]

I found that some other format files were generated in the OUMVLP_rearranged folder, so the processing would randomly break.

Just filtering the png files will solve this problem.

https://github.com/jackhanyuan/OpenGait/blob/master/misc/pretreatment.py

Okay, I see you filter png, so does it work perfectly now? Currently, I am working on verifying @gosiqueira pull_request . After I finish verification, I will try to merge both your and gosiqueira code.

[jackhanyuan]

@noahshen98 Okay, After filtering png, it worked well.

[chuanfushen]

System information (version)

• Pytorch => 1.10

• Operating System / Platform =>Ubuntu 20.04

• Cuda => Cuda 11.5

Detailed description

pretreatment.py for OUMVLP dataset results in random deadlocks.

Currently you are making asynchronous calls using a thread pool and storing the result e ordinary Python list. Thus, when I execute the code to convert OUMVLP files to pickles it's happening random deadlocks during the code execution.

I forked the directory and I'm trying to solve this myself to make a PR, bit I believe it's relevant to keep you apart of this problem.

Steps to reproduce

Steps:

1. Execute the code for OUMVLP dataset using personal machine processor. The error occurs in a non-deterministic fashion.

Issue submission checklist

• I checked the problem with documentation, FAQ, issues, and have not found solution.

Hi, I have verified the output of your modified pretreatment is identical to previous version. Besides, your modified version is much clear and understandable. Thanks for your contribution.