Download speed isn't fast as it could be while downloading consecutive files

[bibek22]

Say, I run a command to download a video. It downloads on full speed till the first file(.mp4 w/o audio) is downloaded. But when it proceeds to download the audio file(.m4a), the speed lowers significantly (from ~250KBps, my full bandwidth, to ~10KBps).

But if I cancel the download and run the command again, it resumes the download with full speed.
It's not just for .m4a files. For ex.it happens also when proceeding to .mp4 file of next video after finishing .m4a files of prev. video from the same playlist.

It's a minor annoyance really to having to enter commands each mins or so. But I am automating it as I run shell from within emacs.

Any help would be nice.
It happens on both of my ubuntu and linux mint machine. I'm using the version of 2015.08.09.
(Also I don't know if it's a right place to report this.)

[dstftw]

There is nothing youtube-dl can do about that - that's how it's served by YouTube.