Multithreaded Rendering #1172
Replies: 4 comments 7 replies
|
One more note about my current implementation... I made multi-threading something that can be turned on and off from the preferences | audio panel. You can enable/disable it and you can specify how many worker threads you want. It is nice because I can enable it and disable it while the graph is running and you can see the CPU usage jump up and down but the sound remains the same. |
|
@BuzzBurrowes awesome! how/when do you decide to render a graph again after it is in the background and quiet and gets activated again? i guess we can afford a little silence when changing graphs but still interested. |
|
Not to muddy the waters, but it might be worth thinking about where this could go in the future to make it even better. As it stands, with this change, rendering can only go wide to render each graph in a session in parallel. It can't render the nodes IN a graph in parallel. From a very high level, let me propose how the next step could be made to allow a graph's nodes to be rendered in parallel. Assumptions...
There is already code that "builds" the graph's processing order, assigning work buffers, etc. This would be pretty significantly rewritten as follows... First, you visit every node (order doesn't matter) and assign a member Next, you identify each "terminating node." Terminating nodes are nodes whose outputs don't go anywhere else inside the graph. For example... the audio output node. Now... let's say every node has a So the graph builder walks its list of terminating nodes and calls setNodeDepth(0). Those nodes call their input nodes' setNodeDepth(1), etc. OK... one wrinkle... the output of a node could be the input to BOTH one node at "level 4" of the graph and one node at "level 2" of the graph. So... in Now you have every node with a depth, and we can think of this depth as, "All the nodes at depth x must be processed before all nodes at depth x-1", or "deeper nodes need to get processed before shallower nodes." NOTE: This isn't strictly true, some shallower nodes could be processed sooner, but there is are ways to deal with optimizing this even further I won't go into here. So you sort all the nodes based on their depth. Deepest (biggest number) first. Now you can build your task list... Done. |
|
throwing some more thougths in. in the style of puredata (where you have the pd~ object which creates a separate instance of pd within pd that can use other cores then the main pd process, it might actually be more desirable for users to have only sub-graphs assigned to another core/thread in a graph, not individual nodes. that way the user can somehow control which plugins go where... this might also be easier to implement. or put another way, how would your system distribute node's over threads/cores? would there be any logic to it other then depth? |
Uh oh!
There was an error while loading. Please reload this page.
I have local changes that spawns worker threads and does multithreaded rendering. It saved 0% processing for a session with a single graph... since I am not digging in to the graph rendering to figure out if there are opportunities to go wide in there. BUT... it saves a TON of CPU in any session with multiple graphs where some/all of those have there Rendering Mode set to 'single'. This makes Element useable for my use case, where I thought I might need to move to a commercial application otherwise. I'll explain more about that in a moment, but first...
I wanted to gauge interest and discuss with whoever knows the most about (or owns) 'audioengine.cpp'.
OK... Some additional detail. (Sorry... this is going to be a little long winded.)
I THOUGHT that creating one session with a bunch of graphs, marking each of those graphs as single, and assigning each graph a program number would allow me to quickly switch between graphs during a show AND only pay the CPU price for the one currently active graph. I built a big session like this only to find out it wasn't true.
While MIDI messages are not sent to background graphs (I will use the term 'background graph' to refer to any graph in any rendering mode behind a current, 'single' rendering mode graph), the background graphs' render function is always called. And in my case, even the VSTs not making any sound added up to considerable CPU cost. I wasn't going to be able to have my giant session that had all the graphs I needed for a show. This was a problem.
So my first idea was to solve this problem by modifying the logic that decided whether to call a graph's renderBypassed() or render() function. (This code is in 'audioengine.cpp) Originally renderBypassed() was only called if a graph was suspended (and it seems that is something different than a graph being "pushed behind" a 'single' mode graph.) So I made it call that function if the graph being looked at is not the 'current' graph and the current graph is 'single mode'. In otherwords... a 'backgound graph'.
My solution was a little more complicated than that... I allowed a background graph to have one additional render() call so its output could be faded over the course of one render block before their render() function stopped getting called. This mostly worked. But if you switched away from a graph that was currently 'sounding' (say 'graph A'), to a different 'single' mode graph (say 'graph B'), while graph A would fade nicely without a click, when you switched from graph B back to graph A, and graph A's render call started getting called again, you would hear the tail of whatever was playing on it when you switched away.
And best as I could find there was really no elegant way to fix this. Some (all?) VSTs don't respond to 'allSoundsOff' midi messages. There is no JUCE call that says something like, "hey! silence yourself!". Best I could find with some web searching was someone pointing out that VSTs usually reset themselves when you change their buffer size and/or sample rate. So you could kind of force the issue by walking the graph and telling all VSTs you find that they have a new buffer size and sample rate (say 16 and 22050) and then immediately set them back to correct. That seemed ugly.
OK... That led me to start thinking about multithreading.
And I got it working! Solved all my performance problems (keep the jokes to yourselves).
(BTW: I've done a bunch of multithreaded audio rendering work in my pre-retirement work for Sony, Harmonix, and Epic Games.)
Basically, I created task manager and worker classes. In the RootGraphRender::renderGraphs function I divvy up all the graphs in the graphs member and put them in a task queue and submit them to the task manager. task workers wake up and start processing the graphs. The renderGraphs function then waits for all of the rendering jobs to finish and then mixes them together as it always has.
My huge session with 6 big graphs went from 60% CPU utilization to 10% CPU utilization. (BTW: The CPU usage meter is misleading. I am pretty sure it is more of a 'real-time utilization' number. You see... I have a 32 core machine. When Element is telling me I was using 60% CPU I was only using 60% of ONE CORE. 31 cores where idle. But when I got multithreading working, and it went wide, I was still executing the same amount of instructions, but since it was running wide the render finished MUCH faster 'real-time'. I went from 60% down to 10%. Boom... less CPU, smaller ASIO buffer size, no clicks! Yay!
So... I would be happy to dig in deep on this project and contribute back to the community, but this is a pretty substantial change. I imagine there are some key stakeholders that might want to discuss this kind of thing further before I start making pull requests for massive changes to audioengine.cpp :-P
Happyto discuss with anyone that has thoughts.
All reactions