Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upGitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
SSD friendly file downloading #23606
Comments
|
Download files to RAM disk. |
That requires a lot of RAM since some files are huge, I'm talking about dozens of gigabytes per file. Wouldn't it be possible to download current fragment to memory and then write it to file instead of abusing hardware with unnecessary writes? Or at least add an option to specify directory where youtube-dl downloads fragments to. |
|
I've been revisiting this issue, it turns out |
Description
Downloading a simple video file eg: youtube-dl -f "hls-480p" xvideos_site_56MB_vid_mp4
Creates many temp files on HDD during download:
x.mp4.part-Frag55.part
x.mp4.part-Frag56.part files of 0-1KB
x.mp4.ytdl
x.mp4.part
SSDs have limited lifespans when writing to them, so please add a feature to use more RAM instead of temporary HDD/SSD writes. Eg. --memory 128MB, would wait for 128MB incoming data then write to the final file, without unnecessary additional writes.
Imagine an SSD in a download server, or SD in RaspberryPi, or even in HDD it causes fragmentation.
See Transmission's file cache handling as an example.