Broadcast gossip announcements in sub batches

[johng]

In this PR we split a list of announcements in the gossiper into sub batches. These are broadcasted at regular intervals instead of all at once.

See #2979

Pull Request Checklist

• If this is your first time contributing, we recommend you read the Code
  Contribution Guidelines
• All changes are Go version 1.12 compliant
• The code being submitted is commented according to Code Documentation and Commenting
• For new code: Code is accompanied by tests which exercise both
  the positive and negative (error paths) conditions (if applicable)
• For bug fixes: Code is accompanied by new tests which trigger
  the bug being fixed to prevent regressions
• Any new logging statements use an appropriate subsystem and
  logging level
• Code has been formatted with go fmt
• For code and documentation: lines are wrapped at 80 characters
  (the tab character should be counted as 8 characters, not 4, as some IDEs do
  per default)
• Running make check does not fail any tests
• Running go vet does not report any issues
• Running make lint does not report any new issues that did not
  already exist
• All commits build properly and pass tests. Only in exceptional
  cases it can be justifiable to violate this condition. In that case, the
  reason should be stated in the commit message.
• Commits have a logical structure according to Ideal Git Commit Structure

[cfromknecht]

@johng thanks for the PR, great work!! i've completed an initial first pass, and the functionality seems to be correct. apart from one or two reliability comments, the rest are mostly around simplifications and adhering to the style guidelines :)

[cfromknecht]

@johng thanks for the quick turn around. completed another pass, mostly nits and bringing the PR in line with the style guidelines. appreciate the thorough testing of the batch splits 👍

[cfromknecht]

Thanks @johng! This looks pretty good to me, would be good to get a second review from @wpaulino or @halseth

[cfromknecht]

Gonna start running this patch on my node and see how it fairs

[wpaulino]

@johng could you squash the fixup commits and rebase?

[wpaulino]

Mostly just nits from me on commit structure.

[wpaulino]

Nit: s/batches/batch/

[wpaulino]

Nit: renaming these to annBatch and splitAnnBatch can help gain some more line space.

[johng]

I think I thought the more read able variable name is more important here

[wpaulino]

I think ann is a reasonable abbreviation for announcement but fine with me either way.

[wpaulino]

Nit: looks like the TestSplitAnnouncementCorrectSubBatches changes are a fixup of f4a9edc?

[wpaulino]

Nit: this fixup should be applied to f4a9edca7e107e9af31ad8b44c8ed9c5f78c2792 since that's where the changes were originally introduced.

[johng]

@wpaulino following your comments, and since this was a small change for the gossiper I've combined the two testing commits into one so that all commits compile.

[cfromknecht]

@johng been testing this for a bit on my node, everything appears to be working smoothly! one final comment from me, see inline

[cfromknecht]

i think it'd be preferable to do this computation entirely using integer arithmetic, instead of using floating points and math lib functions. current calculation does ceil(batchSize * subBatchDelay / totalDelay), which is equivalent to using ceiling integer division:

subBatchSize := (batchSize*subBatchDelay + totalDelay - 1) / totalDelay

typically we try to avoid using floats when possible, as behavior the is less consistent across architectures

[cfromknecht]

thanks @johng latest changes lgtm! ready to be squashed

[cfromknecht]

Awesome work @johng! Really happy with how this turned out. LGTM 🍻

Will need one more rebase (there were some issues with our build that have now been fixed), if you pull in master it should get a clean run on travis and be merge-ready. Thanks!

[wpaulino]

LGTM 🎱

A few non-blocking nits left inline -- would prefer if the commits were squashed into one as we generally favor each commit being able to build on its own.

[wpaulino]

I think ann is a reasonable abbreviation for announcement but fine with me either way.

[wpaulino]

Nit: unnecessary cast on batchSize.