Support wavetables with more than 1024 samples/frame #461
Comments
|
As requested via this comment on issue #872, Here is an example file of a Serum wavetable. Inside the Zip are 3 things.
|
|
Thank you very much! |
|
Oh now that I have dynamic memory allocation this should be entirely doable with .wt files. Let me tag this for 1.6.2 and I'll do it this summer. |
|
So this is notes to myself more than anything else
Basically it seems Serum uses non-standard RIFF chunks to indicate looping and so on. Those chunks have two flavors
My raw python reader to decode is on my branch So seems this is integrally linked with #198 also - namely once I know how to parse these serum wav files I will end up using basically raw binary readers I write myself, since libsndfile doesn't read other chunks and neither does windows (hence the mmio) even though macOS sndfile api will. So probably just end up writing code which does it all with ::read to be portable. If I can figure out what's in that |
|
The CLM block has content which looks like this
or
Wonder what that string of binary means up front. Probably something about the size of the loop in the wavetable. Notably the Wavetable.wav file here contains no block saying anything about the size of the sub-samples other than the CLM chunk. So you get a standard wav fmt chunk, you get the clm chunk shown above, and then you get 2MB of PCM data. How you break that up must be described by that clm block somehow. |
Addresses surge-synthesizer#461 Addresses surge-synthesizer#835 Addresses surge-synthesizer#198
This diff supports WAV file in several important ways 1. Read WAV file portably so mac and linux get them 2. Allow direct open of a wav file so you don't have to move them to documents/surge 3. Reliably support the information in the smpl block to allow sampler wave files to be wavetables 4. Support the Serum 'clm ' block meaning Serum files work 5. Increase the surge max wavetable size to 4096 and have that interact properly with dynamic memory management. I am going to tag these issues as "addresses" rather than "closes" for some of them since we want some testing, but I think this gets us the path to wav done except for 511 and 509. Addresses surge-synthesizer#461 Addresses surge-synthesizer#835 Addresses surge-synthesizer#198 Addresses surge-synthesizer#509
This diff supports WAV file in several important ways 1. Read WAV file portably so mac and linux get them 2. Allow direct open of a wav file so you don't have to move them to documents/surge 3. Reliably support the information in the smpl block to allow sampler wave files to be wavetables 4. Support the Serum 'clm ' block meaning Serum files work 5. Increase the surge max wavetable size to 4096 and have that interact properly with dynamic memory management. I am going to tag these issues as "addresses" rather than "closes" for some of them since we want some testing, but I think this gets us the path to wav done except for 511 and 509. Addresses surge-synthesizer#461 Addresses surge-synthesizer#835 Addresses surge-synthesizer#198 Addresses surge-synthesizer#509
This diff supports WAV file in several important ways 1. Read WAV file portably so mac and linux get them 2. Allow direct open of a wav file so you don't have to move them to documents/surge 3. Reliably support the information in the smpl block to allow sampler wave files to be wavetables 4. Support the Serum 'clm ' block meaning Serum files work 5. Increase the surge max wavetable size to 4096 and have that interact properly with dynamic memory management. I am going to tag these issues as "addresses" rather than "closes" for some of them since we want some testing, but I think this gets us the path to wav done except for 511 and 509. Addresses #461 Addresses #835 Addresses #198 Addresses #509
|
Just an update: I just merged a change which reads .wav files with serum extensions properly on all platforms, including the one you shared here. The nightly should be available in an hour or so. Any testing greatly appreciated! |
|
Oh that's some exciting news! I'll test the nightly once it's up. |
|
Currently I am getting an error saying "File is not a standard RIFF/WAVE file" with anything I try, and it fails to import. I am currently just testing standard Serum Wavetables, including both Factory ones and ones I've created myself. Attached is a table that is freshly exported from Serum, but does not import. |
|
Yeah sorry! Just saw that on slack too. Have a fix coming in right now. (The windows optimizer was re-ordering an operation validly that made that test fail because my code was wrong-in-a-way-that-worked-on-mac and I tested in windows only on the second to last change). Another nightly in about an hour. Apologize for the noise. |
|
Haha not a problem. I'm happy for the noise, I've been excited for this for a while. |
|
Great. Well I just merged the pull request. Was a simple change. The nightlies usually take 30-45 minutes to get to the website. If you grab a nightly which is today and has hash 721bfa2 in the name you have the new one. Very interested if it works for you generally once this dumb step 0 bug is gone. Thanks! |
|
New nightly up |
|
Been testing the nightly for about half an hour, and on preliminary inspection: it works!! Pretty much anything exported from Serum works flawlessly. You can't believe how excited I am. Having a Free, Open-Source synth with the ability to load custom wavetables is incredible. Thank you! Now, a Couple of cleanup notes: I purposely tried to load things that weren't Serum wavetables, and the results were mixed. Case 1: When loading most .wav files that are NOT Serum wavetables, Surge crashes, and brings down Reaper too. No error message, just hard crash to the desktop. This is the most common case. Case 2: Occasionally I've come across a file that when attempting to load it, brings up the following error message: "Sorry we can only read power of 2 wavetables right now but sample support coming soon!q" Surge's oscillator then shows (Patch Sample) as the displayed wavetable name. The only sound it makes is a click when played. Important to note is that it clears whatever wavetable was there previously. I would expect that an error would keep the current wavetable. Attached is an example of a file that triggers case 2. ^That is an export/rip of one of the original NI Massive wavetables, before it was run through Serum and re-saved. This one and a handful of others trigger case 2 consistently, although other wavetables in the set just crash Surge. Case 2 is also triggered by all of the single-cycle waveforms in the Adventure Kid (AKWF) set (at least the ones I've tested, there's a lot. Those are found here: |
|
Super thanks for the detailed feedback. I will look in the next day or so! |
|
Ahh those adventurekid sample sets are all 600 sample long cycles (he sampled everything at “D2+2” to get to 600). For a couple of reasons, surge assumes wavetables align on power of 2 boundaries. Breaking that is hard but since I’m in this code anyway, no reason I couldn’t do a resampling at load time to the nearest higher power of 2 on load time using a good resampler (probably like https://en.m.wikipedia.org/wiki/Lanczos_resampling). That’s a bit more code but I will keep this issue open to track that. And of course the crashes shouldnt’ crash. I bet I just freak out and abort if I don’t have any loop data in the wav header at all. And the error messages shouldn’t have random “q” characters in them either. LOL! Just saw that. |
|
Alright I just took a look at that AdditiveNative1.wav file. I have a few questions
It looks to me a bit like this wav file has an invalid smpl block (or at least non-standard). Entry one of the smpl is '255' but that is supposedly a manufacturer specific type. So when you convert this properly into serum here's my questions
Thank you! |
|
Oh and (duh me for not trying this) opening the .wav in audacity makes it clear that the intent is 4 tables of 1024 blocks. Let me dig into the 'cue' block which the data format has. Maybe the answer is there... |
|
Yeah sorry to answer my own question. That file comes through the |
1. Support a `cue ` block of metadata as indicating loop points if the loop points are regular. Apparently some NI files have this format 2. Generally clean up error reporting with - cleaner error messages - don't reset the wavetable if we don't have a wavetable - don't crash if we don't have a loop Addresses surge-synthesizer#461
1. Support a `cue ` block of metadata as indicating loop points if the loop points are regular. Apparently some NI files have this format 2. Generally clean up error reporting with cleaner reporting 3. Support .wav files with no loop data by correctly configuring them as 1024 window single shot samples which load properly into the oscillator. Addresses surge-synthesizer#461 Closes surge-synthesizer#511
1. Support a `cue ` block of metadata as indicating loop points if the loop points are regular. Apparently some NI files have this format 2. Generally clean up error reporting with cleaner reporting 3. Support .wav files with no loop data by correctly configuring them as 1024 window single shot samples which load properly into the oscillator. 4. Allow .wav as well as .wt files to come through the drag n drop path (which only works on windows right now) Addresses surge-synthesizer#461 Closes surge-synthesizer#511 sq
1. Support a `cue ` block of metadata as indicating loop points if the loop points are regular. Apparently some NI files have this format 2. Generally clean up error reporting with cleaner reporting 3. Support .wav files with no loop data by correctly configuring them as 1024 window single shot samples which load properly into the oscillator. 4. Allow .wav as well as .wt files to come through the drag n drop path (which only works on windows right now) Addresses #461 Closes #511
|
OK @theofficialajyoung I just pushed a change which fixes all the issues you posted, and reinstates the .wav file as single shot working properly (which hasn't really worked since 1.5.0). New nightly is building now. |
|
New nightly up! |
|
Nice! I'll get to testing. Clarification on that Additive wavetable: I almost guarantee you'll never see that type of file structure/format in actual use, as the Massive Wavetables were converted to Serum-style years ago and that's what everyone uses instead. I just included it as an example of "Oh hey, here's a file that Surge does something weird with". |
|
Ha. Well if I do ever see that particularly weird form of loop marker again, I'll read it fine. Seems to make sense to me anyway. But appreciate the context. Look forward to hearing how testing goes. Appreciate the clear feedback - thank you. Good bug reports make good software! |
|
Thanks! It's looking good so far, no crashes yet! I'll let you know if I encounter anything. The sample import is cool for WAV files that aren't tables. It's unfortunate that Stereo files don't work though, as that's what most sample packs are. It makes sense though, as Surge's audio path isn't true stereo. (EDIT: It feels like it should almost be it's own oscillator type, outside of Wavetable. thoughts?) I've actually been thinking lately of writing up a feature request for a fully stereo signal path. The "Wide" filter routing is nice, but I find myself wishing to be able to use the other filter routings while still having stereo information. Currently there's no way to use dual/parallel filter routing except in mono. |
|
Also, just as a follow up, here's some formats you may actually encounter. Attached is a Wavetable in WAV format from each of the other Wavetable Synths I own: Hive 2, Dune 3, and Rapid. |
|
Thank you. I'll run those through the system and see what happens! Oh don't know if you saw: Drag n Drop is back for wav files also. Just drag a .wav onto the picture of the oscillator and it will load. May be faster than the menu. (Should work on windows but only tested on mac tbh) |
|
It does indeed work! |
|
OK of these three
Super useful. I bet I can get at least some of these working. Thank you. |
|
Good to know! Dune is the only one of the 3 that has a wavetable editor to make custom wavetables, so that would be the only one commonly encountered. The other 2 are mostly "hey, maybe some crazy person would try this". As far as Dune goes, the synth doesn't have sampling abilities, and I assume just goes "don't put anything that isn't the expected format in this folder". Probably doesn't even bother with headers of its wavetables. |
|
Alright literally nothing to be done with the DUNE one; but the HIVe and RAPID ones I was able to load and based on metadata choose a reasonable default guess as to what they mean. Getting a PR in now. Would appreciate more testing for course. If the CI builds probably another hour to the nightly. Yeah the DUNE comment makes sense. I suppose I could add an "Open wav as 2048 wavetable" option which forces the format. Since surge does single shot and wavetables it relies on the file to indicate which is which. If I had a better UI framework than VSTGUI I would write a proper importer of course. But I don't... |
|
OK so I'm going to go ahead a close this issue - I think the current WAV support is pretty robust. I'm sure we'll find bugs but let me add new issues for those as we go as opposed to just splatting this long one which is basically finished. @theofficialajyoung thank you very very much for both the testing and the .wav samples. They were critical in getting this feature put together. Greatly appreciated. Also: I fixed some more wav bugs which are especially annoying - including never closing the wav file (!!) - so you want to upgrade to the latest nightly if you have an older nightly on your system. Thanks again |
Most recent wavetable synths (u-he Hive, Serum, Kontakt 6's wavetable engine) support wavetables with 2048 samples/frame, whereas Surge goes up to 1024. It might be a good idea to support wavetables with more than that. Say up to 8192 samples.
The text was updated successfully, but these errors were encountered: