Replies: 6 comments 9 replies
-
Yes - the quality scaling switches an A2 model from "lite" to "full" when you go past 0.5. When it switches internal models, NAM Core prewarms the new model, and NAM Core prewarm is very expensive (compared to my internal code). So currently, as I've noted in the NeuralAudio README, quality scaling is not realtime-safe. Despite this, in the LV2 plugin I still do it during realtime processing since it currently seems like the best alternative. If/when I do my own A2 implementation (rather than relying on NAM Core), I may choose to handle things differently. |
Beta Was this translation helpful? Give feedback.
-
Yes, but they are very easy to solve. While the vector warnings coming directly from the NAM core library.
So then I'll wait a bit before updating NeuralRack. I don't think it is akzeptable to produce a Xrun while model changing (or scaling) in a plugin. |
Beta Was this translation helpful? Give feedback.
-
I'm just doing the best I can with the situation I've been given... The NeuralAudio library doesn't force you to change the model quality on the runtime thread - that's just the solution I'm currently going with for this plugin. My UI options are limited. You may be able to handle it differently. |
Beta Was this translation helpful? Give feedback.
-
Fwiw, TooB Neural Amp Modeler routes handling of control changes to the Slim control to a background thread. (And it is using the NeuralAudio library). The Neural Audio Library does allow client plugins to do the right thing. And as I read the situation, it is currently the client's responsibility to make sure that Slim/Quality control changes are not done on a realtime thread. In prior versions, you have to do it for model changes already. Now you have to do it for both model changes AND Slim/Quality control changes. It may or may not also be help if you mark the LV2 Silm/Quality control as "expensive", which may or may not make hosts and/or UI frameworks send just final values of a control change when you release the mouse button, , rather than continuous change as the knob moves. (Sending continuous control changes for the Slim control produces a rather hilariously awful effect). And -- as the author of both a plugin and an LV2 host -- you might want to consider doing that work on a thread other than the LV2 scheduler thread, since the current cost of prewarming a NAM model is really significant. (And maybe limit the number of prewarming operations that can take place simultaneuously based on the number of available CPUs). There is a significant risk that you may cause the Ubuntu kernel to kill your process for excessive CPU use if the host you are using runs LV2 Scheduler requests at realtime priority, instead of Nice+ priority. (high priority, non-realtime scheduler). PiPedal now uses Nice+ priority for the Lv2 Scheduler worker threads; other hosts may not). |
Beta Was this translation helpful? Give feedback.
-
I got a contribution from @rominator1983 for the neural-amp-modeler-ui, so it comes with a Quality controller now as well. |
Beta Was this translation helpful? Give feedback.
-
|
👍 |
Beta Was this translation helpful? Give feedback.
-
|
I'm also considering changing the NAM Core container implementation (ie: when you have two or more models in one .nam - like the current A2) to prewarm all models on load, but then not prewarm again when you switch models. It would make loading a bit more expensive, but that is already a background operation. Once a model is prewarmed once, it is at least in a stable state. It isn't clear that prewarming again when you switch models is useful. You will get discontinuities if audio is playing, because the new model won't have the current audio history baked in, but that is pretty much inevitable, and prewarming again doesn't help. |
Beta Was this translation helpful? Give feedback.
-
|
The coolest thing would be, if we could keep the history and only switch the model. Then, you only needs to prewarm once at startup and be done. I've just worked out a convolution engine which allow exactly that (swap the kernel live without audio dropouts). True, I'm far from understand the core of neural amp modelling, so I've no idea if that would be possible at all with neural models, but, I guess that is the way how we could implement controllers for a model, like tonestack or drive. |
Beta Was this translation helpful? Give feedback.
-
Assuming you mean to keep the audio history, it doesn't really help since you still need to re-run the layered convolutions (which is basically what prewarming is doing - it just assumes a history of silence). So it can't be done live, and if you do it in the background then by the time you are finished it is already out of sync again... |
Beta Was this translation helpful? Give feedback.
-
|
I mean the internal filter state. |
Beta Was this translation helpful? Give feedback.
-
|
I transferred this discussion here to the NeuralAudio repo, where it is more relevant. |
Beta Was this translation helpful? Give feedback.
-
|
I just checked in a composite_models branch which uses my own implementation for handling A2 slimmable container models. The current behavior is to prewarm all models on load, and then not re-prewarm again when models switch based on quality scaling. This makes quality scale switching realtime-safe, which simplifies things a lot. The discontinuity you get with this method seems, if anything, a bit less than if you prewarm on model switching. I am considering adding optional "lazy" model loading that doesn't prewarm (and maybe doesn't even create) non-active models on load. This would mean that setting the quality would sometimes be RT safe and sometimes not, so that complexity would have to be handled by the plugin. I could potentially provide a method to check if a quality scale change is RT safe or not. |
Beta Was this translation helpful? Give feedback.
-
I would use that, I think! Tempting as the possibility of real-time quality changes is. In either case, you get a discontinuity if there is active input when the switch takes place. (Yes the newly active submodel has been prewarmed, but the submodel's memory of previous input still needs to catch up to current input). Lower initial prewarm time, at the cost of having the same discontinuity (which only happens when the input control crosses a slimmable value threshold) delayed by 30ms because of a trip to the background thread to prewarm. That would be my preference. Even better if you expose the actual array of model weights, since that would also allow me to display affordances to users on where the model changes take place (ticks where the transitions take place, perhaps). Robin Davies |
Beta Was this translation helpful? Give feedback.
-
|
Ok, this is all now implemented and merged. See the updated docs, particularly the composite model loading and quality scale factor sections. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
The decrease in dsp load is really impressive, when scale a A2 model down to 0.0 quality. Here it goes mostly from 6.1% dsp load to 1.2% dsp load. Maybe my ears are not the best any more, but sound-wise I didn't notice a difference. Very nice.
What I noted is, that it seems to work in most cases like a switch, so below 0.5 quality ~= 1.2% dsp, above 6.1% dsp, there seems to be no transition. Downscale the quality works fine, but upscaling introduce a xrun every time reproducible. Other than that it seems to work stable here, true, I got a couple of warnings during compile, like ```
warning: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
111 | for (auto i = 0; i < outSize; i++)
_Vector_impl_data::_M_end_of_storage’ may be used uninitialized [-Wmaybe-uninitialized]
369 | _M_impl._M_end_of_storage - _M_impl._M_start);
Beta Was this translation helpful? Give feedback.
All reactions