Backfill gossip without buffering directly in LDK

[TheBlueMatt]

This is intended as a pre-req for #1604 with a few of the changes from #1604 pulled in. The commit description of the "main" commit follows, with a few other minor tweaks present as well.

Instead of backfilling gossip by buffering (up to) ten messages at
a time, only buffer one message at a time, as the peers' outbound
socket buffer drains. This moves the outbound backfill messages out
of PeerHandler and into the operating system buffer, where it
arguably belongs.

Not buffering causes us to walk the gossip B-Trees somewhat more
often, but avoids allocating vecs for the responses. While its
probably (without having benchmarked it) a net performance loss, it
simplifies buffer tracking and leaves us with more room to play
with the buffer sizing constants as we add onion message forwarding
which is an important win.

[valentinewallace]

Looks good, basically :)

[valentinewallace]

Looks good! I'm happy with this after we get a second reviewer. Feel free to squash

[TheBlueMatt]

Squashed without diff.

[valentinewallace]

ACK after squash

[TheBlueMatt]

Squashed without further changes.

[valentinewallace]

Ah, I think a warning may have been introduced in stable actually?

warning: variable does not need to be mutable
   --> lightning/src/routing/gossip.rs:345:7
    |
345 |         let mut iter = if let Some(pubkey) = starting_point {
    |             ----^^^^
    |             |
    |             help: remove this `mut`
    |
    = note: `#[warn(unused_mut)]` on by default

[jkczyz]

Ah, I think a warning may have been introduced in stable actually?

warning: variable does not need to be mutable
   --> lightning/src/routing/gossip.rs:345:7
    |
345 |         let mut iter = if let Some(pubkey) = starting_point {
    |             ----^^^^
    |             |
    |             help: remove this `mut`
    |
    = note: `#[warn(unused_mut)]` on by default

Yeah, that's from addressing one of my comments. TIL you can avoid the explicit next call with this syntax:

diff --git a/lightning/src/routing/gossip.rs b/lightning/src/routing/gossip.rs
index cb49ab81..e91d251f 100644
--- a/lightning/src/routing/gossip.rs
+++ b/lightning/src/routing/gossip.rs
@@ -38,6 +38,7 @@ use io_extras::{copy, sink};
 use prelude::*;
 use alloc::collections::{BTreeMap, btree_map::Entry as BtreeEntry};
 use core::{cmp, fmt};
+use core::ops::Bound;
 use sync::{RwLock, RwLockReadGuard};
 use core::sync::atomic::{AtomicUsize, Ordering};
 use sync::Mutex;
@@ -342,14 +343,9 @@ where C::Target: chain::Access, L::Target: Logger
 
        fn get_next_node_announcement(&self, starting_point: Option<&PublicKey>) -> Option<NodeAnnouncement> {
                let nodes = self.network_graph.nodes.read().unwrap();
-               let mut iter = if let Some(pubkey) = starting_point {
-                               let mut iter = nodes.range(NodeId::from_pubkey(pubkey)..);
-                               iter.next();
-                               iter
-                       } else {
-                               nodes.range::<NodeId, _>(..)
-                       };
-               for (_, ref node) in iter {
+               let starting_bound = starting_point.map_or(
+                       Bound::Unbounded, |pubkey| Bound::Excluded(NodeId::from_pubkey(pubkey)));
+               for (_, ref node) in nodes.range((starting_bound, Bound::Unbounded)) {
                        if let Some(node_info) = node.announcement_info.as_ref() {
                                if let Some(msg) = node_info.announcement_message.clone() {
                                        return Some(msg);

[TheBlueMatt]

Squashed without further changes.