-
Notifications
You must be signed in to change notification settings - Fork 938
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Simplify CQT tests #1176
Comments
Let's make some sense of this by first mapping out what the current test sweeps are. To keep it simple, I'm not counting all of our specific test cases, only the broadest tests with huge parameter grids. `test_cqt` sweeps 9 parameters (8 non-trivial), 768 tests:
`test_vqt` sweeps 10 (4 non-trivial), 24 tests
`test_hybrid_cqt` sweeps 10 (7 non-trivial), 672 tests
`test_icqt` sweeps 6, 96 tests
`test_griffinlim_cqt` sweeps 12 parameters (9 non-trivial), 768 tests
In most of these cases, the parameters can be broken down into interacting sets that affect either A) the quality of the filters (res_type, filter_scale, sparsity, norm), B) the output frequency range (bins_per_octave, fmin, n_bins), C) frame rate (hop_length, sr). Although there are certainly some interactions here (eg between frequency range and frame rate), I think we can factor the tests to focus on specific aspects.
We should focus on reducing the number of tests for For
For For
With the above modifications, we ought to be able to reduce the runtime for |
* added coverage.xml to gitignore * accelerated some cqt tests * fixes #1176; simplifies CQT test suite * switched default cqt test to res_type=none instead of polyphase to trigger early downsampling * streamlining cqt tests some more * blacked test_constantq.py * fixed early downsampling test coverage for cqt
At present, our test suite takes somewhere in the neighborhood of 15 minutes to run (on Travis). A substantial portion of that is due to the CQT tests, which brute-force through a very large parameter grid.
Rather than trying all combinations of parameters, we should refactor the tests to focus on the behavior of individual parameters (or small grids of parameters where interactions are inevitable).
For reference, here's the runtime on my dev machine for the full test suite:
and for just the cqt module:
The text was updated successfully, but these errors were encountered: