Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upGitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Multi-Threaded postproc for playlists #12215
Comments
|
FYI: #12077 is useful in some cases. |
|
Indeed, thanks. N ffmpeg processes working on their own files almost always parallelizes better than 1x ffmpeg process passed -threads N, because not all routines in ffmpeg can-be/are parallelized: it depends on the underlying codec to support multi-threading. but you already knew this ;-P |
|
Using parallel to utilize CPU cores, run N postproc at same time. e.g. |
|
Nice thanks Jethro that looks like itll do exactly what I want (except
that the downloads are parallel too, slightly increasing my chance of
getting banned lol).
Maybe there could be a flag "--external-jobrunner parallel" or some
such that simply wraps that command (and gracefully defaults back to
internal/regular/single-threaded if the external jobrunner dne).
sounds ez enough, but Im terrible at python :(
…On 5/23/17, Jethro Yu ***@***.***> wrote:
Using parallel to utilize CPU cores, run N postproc in same time.
parallel's default is 100% which will run one job per CPU core on each
machine.
e.g.
```
youtube-dl --get-id PLAYLIST_ID -i | tee ids
parallel --linebuffer -- youtube-dl -x -k --embed-thumbnail --add-metadata
--audio-format mp3 <ids
```
--
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
#12215 (comment)
|
|
As a side note: #3746 is related |
and others
Would be nice if postproc was done in separate thread(s) (up to N threads, N being the number of CPU cores). As soon as a download is finished, the postproc job should be dispatched and the next download should start immediately. But if all N cores are doing postproc, wait for one to finish before downloading another playlist item. If the machine has only 1 core then the the behavior is exactly the same as it currently is.