Feature Request: Allow Multiple Instances #146

Open
texneus opened this Issue Nov 14, 2016 · 3 comments

Projects

None yet

3 participants

@texneus
texneus commented Nov 14, 2016

Handbrake documentation reveals that 6 CPU cores is optimal (see: https://handbrake.fr/docs/en/latest/advanced/video-encoding-performance.html). In other words, running Handbrake on a hypothetical 12 core CPU would not double the encoding speed (as might be expected) as compared Handbrake on an otherwise identical 6 core CPU. At least one study conducted on 6 and 12 core CPUs confirm this effect and concluded there is little benefit to running Handbrake on more than 9 cores: https://macperformanceguide.com/Optimizing-Handbrake.html

The link above suggests a means to use all CPU cores is to run two instances of Handbrake and dividing the video sources equally among the two instances. When this was done on their 12 core machine a 40% increase in total throughput was realized (surprisingly, a boost was also noted in their 6 core hardware).

On my own I7-6700 machine (where half the CPU cores are virtual) the effect is still evident. Running a single instance of Vidcoder results in a CPU load of about 80%, or roughly 6 cores of actual "work". When I run two instances of Vidcoder, a small (5-10%) increase in throughput was indeed noted. So the limit is real and even left over virtual cores are somewhat useful. This will surely become an increasing problem as high core count CPUs become more mainstream (AMD Zen is rumored to come with as many as 32 cores/64 thread variants at launch).

The problem with Vidcoder is it is not easy to run two instances. When a second instance of Vidcoder is opened, the que from the first instance appears in each new copy, and each copy dutifully encodes the full list. The only way I know to run two instances of Vidcoder is to run it under two different user accounts, or run the release and the beta versions side by side. I would like to see the ability to open two or more instances of Vidcoder and have each que be unique to that instance. Alternatively, a single instance of Vidcoder could also call multiple instances of Handbrake as separate "workers", each instance of Handbrake working on a unique video from Vidcoder's que. The later perhaps could be based on CPU load, where if it doesn't see the CPU as being 90-100% busy, new workers are automatically created until the CPU is loaded to full capability.

@RandomEngy
Owner

That's a really interesting idea. If it was controlled by detected CPU load it might run into trouble if the bottlenecked resource was the disk or they're streaming in the input over the network. In those cases the CPU load wouldn't be 100% but additional instances wouldn't help. I could see it as an option to explicitly enable, though. I will think about this.

@texneus
texneus commented Nov 23, 2016

I did some more tests with a 6700k and the average difference in encoding speed across several samples is turning out to be >5%. Still it sounds like if there are more real cores the difference is noticeable. You are of course correct throwing more CPU when read/writes are bottle necked will not help, but I don't think most people work that way, or at least I don't. CPU load does sit at 98-100% (vs. 80% +/-) when I can run two side by side encodes.

@humandxp

isnt that connected to x264 encoding threads not cpu cores.
for my encodes i use additional line for x264 and use 12 threads so my 4 cores are always at 100% and priority at idle, can use pc and play games, where rest of cpu is used for handbrake in vidcoder.
rc-lookahead=50:vbv-maxrate=62500:level=4.1:vbv-bufsize=78125:ref=5:bframes=5:b-adapt=2:direct=auto:me=umh:subme=8:threads=12
and always use constant quality 22 is best for full hd videos.
and i dont think that threads wait for previous frame to encode, i think they split file into 12 parts and encode cos more threads generate different file size. same is with 7zip

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment