[FEATURE] Optionally process WaveNet conditions with another WaveNet #190
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Updated WaveNet constructor to accept a unique_ptr for a condition DSP, enabling conditional processing. - Implemented checks to ensure compatibility between WaveNet input channels and condition DSP input/output channels. - Added a new method, _process_condition, to handle the processing of condition inputs and outputs. - Adjusted SetMaxBufferSize to accommodate resizing of condition DSP buffers. - Updated tests to include condition processing scenarios, ensuring comprehensive coverage of the new functionality.
- Added clarification comments in the set_weights_ method regarding the handling of condition_dsp weights. - Modified the process method to ensure the first layer array uses the original input instead of the processed condition output, improving clarity and correctness in processing logic. - Updated weight initialization comments in the test_condition_processing file for better understanding of identity mappings and weight matrix dimensions.
- Add ADDASSERT test block in run_tests.cpp to verify assertions work - Add ENABLE_ASSERT_TEST CMake option to enable assertion testing - Allows building with -DENABLE_ASSERT_TEST=ON to test that assertions aren't being optimized away in release builds
- Add friend declaration in DSP class to allow WaveNet to access protected members - Change WaveNet constructor to accept std::unique_ptr<DSP> instead of std::unique_ptr<WaveNet> - Update Factory function to use DSP type for condition_dsp parameter This enables WaveNet to use any DSP subclass as a condition DSP, not just another WaveNet instance, while maintaining encapsulation through friendship.
- Updated the output buffer allocation in the test_with_condition_dsp_multichannel function to support three output channels instead of one. - Enhanced the output verification to assert that all output channels are non-zero and finite, improving the robustness of the test.
- Add condition DSP functionality to get_dsp and wavenet - Add wavenet_condition_dsp.nam example model - Add benchmark and plotting tools for wavenet_a1 models - Update tests for get_dsp functionality
…tead of file dump workaround
- Resolved conflict in NAM/wavenet.cpp: combined explicit variable declarations with head1x1_params parsing - Resolved conflict in tools/run_tests.cpp: included both test_condition_processing.cpp and test_head1x1.cpp - Both features from HEAD and main branches are now integrated
- Add missing Head1x1Params argument to LayerArrayParams in test_condition_processing.cpp - Update all test files to use correct LayerArrayParams constructor signature - Code formatting cleanup across test files
…nam_files function - Changed paths in the test_load_and_process_nam_files function to be relative to the root directory instead of the build directory, ensuring accurate file loading during tests.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TODO
nam:DSPSummary
Instead of using the input audio signal directly as the "condition" input to Layer arrays (and Layers), optionally process it through another WaveNet to produce some "learned" conditioning.
This is somewhat similar in spirit to "context stacks" cf. Sec. 2.6. of van den Oord et al., 2016.
Resolves #188
Developed with support from TONE3000