fix(attachment): convoy separation

[KenLSM]

Problem

Closes FRM-1757

Admins are not able to receive all the download responses when clicking download with CSV.

Most modern browser blocks sequential downloads that run in quick successions.

Symptoms

• results are mainly without attachments
• responses > 10

Solution

Implementation of convoy separation.

Downloads are grouped into convoys where the front of the convoy must be x ms away from the next convoy.

[ o . o . . o o o ] . . [ o o o . . o . o] [ o o o o ]
  ^                       ^                  ^
  1st convoy              2nd convoy         3rd convoy                  

CONVOY_SIZE = 5
The time between each convoy should X amount of distance away to avoid a flood
3rd Convoy is too close and thus will be forced to wait

[ o . o . . o o o ] . . [ o o o . . o . o] . . . . [ o o o o ]
  ^                       ^                          ^
  1st convoy              2nd convoy                 3rd convoy

Additional Context

Total responses = 44
Experiments:

Control Group

• no delay -> get 20 zips

Batch delay

• 1000ms of delay every 7 files -> get all 44 zips (increased completion time of 6 seconds)
• 200ms of delay every 7 files -> get 22 zips

Convoy delay (CONVOY_SIZE = 7)

• 1000ms between first of next convoy, -> get 39 zips

Convoy delay (CONVOY_SIZE = 5)

• 1000ms between first of next convoy, -> get all 44 zips

Breaking Changes

• No - this PR is backwards compatible

Tests

Regression on large submissions >10 crossing two convoys

• Create 40 submissions with 20 attachments
• Ensure that Download CSV (without attachment) returns 1 .csv file
• Ensure that Download CSV with attachment returns 40 .zip and 1 .csv file

Regression on small submissions < 10

• Create 7 submissions with 3 attachments only 1 convoy
• Ensure that Download CSV (without attachment) returns 1 .csv file
• Ensure that Download CSV with attachment returns 7 .zip and 1 .csv file

[g-tejas]

LGTM! The algo looks good to me

[linear]

FRM-1757 Browser blocks admin response download when too many downloads triggered too quickly