[instagram] fix highlights extraction

[vsyx]

Currently the highlight extractor fails when there are more than 30 reel_ids supplied to the "/v1/feed/reels_media/" endpoint.

For example, fetching highlights from "https://www.instagram.com/f1/" (87 highlights at the time I'm writing this)

gallery-dl -v --no-download -o include="highlights" "https://www.instagram.com/f1/"

will respond with status code 400.

Initially I tried splitting reel_ids into chunks of length 30, however, I quickly found that I consistently get 5xx responses when the total number of reel_ids is above 30 (I waited an entire week just to make sure that I wasn't being rate limited). When the total number of reel_ids is <= 30 (example account "https://www.instagram.com/bemytravelmuse/", 27 highlights at the time I'm writing this), you can fetch everything with a single request, however, in my case 5xx responses can still be frequently observed.

I haven't tested everything thoroughly, but it looks like chunks of 15 seem to work most of the time. That being said, it might be a good idea to start at 30 and just reduce the chunk size (e.g. halve it) until you stop getting 5xx responses.

[mikf]

Thank you.

I reduced the chunk_size even further to a save 5 (same as on instagram itself) since I immediately got a 560 error when testing your solution.
I've also changed it so it doesn't have to wait for all highlight reels to load before starting to download the first file.