Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loading data too slow #52

Open
CopyNinja1999 opened this issue Feb 20, 2023 · 1 comment
Open

Loading data too slow #52

CopyNinja1999 opened this issue Feb 20, 2023 · 1 comment

Comments

@CopyNinja1999
Copy link

How long do you usually load the data? I have approximately 3000h data which takes me more than 1 hour to load the data and no idea which part caused this. Any ideas to speed it up? Does it help to apply larger nj?

@cantabile-kwok
Copy link

I'd say I will be very satisfied if it takes only 1+ hour to load such large amount of data. I currently have only 240h hours of data and it already takes more than 1 hour before training starts. I think the main reason is because there are toooooo many small files (.qnt.pt, .phn.txt, ...) Iterating through these files and reading them certainly is expensive particularly for large file systems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants