feat(quick-protobuf-codec): reduce allocations during encoding

[thomaseizinger]

Description

We can reduce the number of allocations during the encoding of messages by not depending on the Encoder implementation of unsigned-varint but doing the encoding ourselves.

This allows us to directly plug the BytesMut into the Writer and directly encode into the target buffer. Previously, we were allocating each message as an additional buffer that got immediately thrown-away again.

Related: #4781.

Notes & open questions

Change checklist

• I have performed a self-review of my own code
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• A changelog entry has been made in the appropriate crates

[thomaseizinger]

Draft because it is stacked on top of another PR, but ready for review.

[joshuef]

@thomaseizinger I'm trying to have a bash at this for heaptracking, but getting compile errors atm.

Is there a stablish commit here worth testing from?

eg
error[E0277]: the trait bound `quick_protobuf_codec::Codec<proto::structs::Message>: Decoder` is not satisfied --> /Users/josh/.cargo/git/checkouts/rust-libp2p-98135dbcf5b63918/aa08f5a63e91/protocols/autonat/src/protocol.rs:48:43 | 48 | let message = FramedRead::new(io, codec()) | --------------- ^^^^^^^ the trait `Decoder` is not implemented for `quick_protobuf_codec::Codec<proto::structs::Message>` | | | required by a bound introduced by this call | = help: the following other types implement trait `Decoder`: BytesCodec LengthCodec LinesCodec asynchronous_codec::framed_write::FramedWrite2<T>

[thomaseizinger]

Try the one before the last merge commit! Sorry for that, I didn't expect there to be compile errors with merging master 😅

[joshuef]

Ran into issues today and didnt have time to get deeper. Probs will be next week when i come back with heaps i'm afraid as I'm off tomorrow

[thomaseizinger]

Perhaps @jxs has time to check whether this reduces allocations for lighthouse?

[jxs]

@thomaseizinger I'm trying to have a bash at this for heaptracking, but getting compile errors atm.

Is there a stablish commit here worth testing from?

eg error[E0277]: the trait bound `quick_protobuf_codec::Codec<proto::structs::Message>: Decoder` is not satisfied --> /Users/josh/.cargo/git/checkouts/rust-libp2p-98135dbcf5b63918/aa08f5a63e91/protocols/autonat/src/protocol.rs:48:43 | 48 | let message = FramedRead::new(io, codec()) | --------------- ^^^^^^^ the trait `Decoder` is not implemented for `quick_protobuf_codec::Codec<proto::structs::Message>` | | | required by a bound introduced by this call | = help: the following other types implement trait `Decoder`: BytesCodec LengthCodec LinesCodec asynchronous_codec::framed_write::FramedWrite2<T>

Hi @joshuef this seems fixable by running cargo clean and cargo build again.

Perhaps @jxs has time to check whether this reduces allocations for lighthouse?

Sorry Thomas, this fix runs on the latest rust-libp2p master which I could not make compatible with Lighthouse's current transport stack, either because SelectUpgrade doesn't implement InboundConnectionUpgrade or because I couldn't add with_bandwidth_logging to the the new SwarmBuilder with tcp quic and dns, my changes are here if you want to take a look.
Thanks!

[thomaseizinger]

Oh, thanks for trying @jxs. I might try and backport it for v0.52 then.

[joshuef]

Oookay, reupped main and the branch. The difference is only in the libp2p v. Main is on 0.52, the libp2p-branch one is running this (so w/0.53 changes in). All sorted by allocations here.

baseline (main):

main w/ this PR:

Focussing on gossip specifically (main is the top, this PR bottom)

So the main allocating functions seem to change. This PR seems like it's allocating a fair bit less...

Though the overall stats main is leaking/allocating less overall 🤔 (same order).

I have the heaps for this if that's helpful (largest is ~12mb i think).

I'm not sure how to account for that difference on the overall. But it's not complete apples-apples as main is on 0.52x as I say...

i'm going to run our main w/ the 0.53 in libp2p/master to get more apples/apples with this PR.

[joshuef]

Okay, here is our main w/ master libp2p 0.53.

[thomaseizinger]

I've done some more benchmarking of this code. Here are the results:

• Less allocations. As intended with this refactoring, we are essentially no longer allocating (assuming the target buffer is big enough):

  Baseline	This PR

  Pay attention to peak heap memory consumption. This is 3MB for the baseline and 2MB for the this version. With a message of 1 MB size, the large_message test needs to allocate the message itself plus the target buffer. In the baseline version, we allocate an additional 1MB for encoding. This is now gone. I've generated this data but running heaptrack against the large_message test binary.

• Slight performance improvement for encoding. This is expected as we perform less allocations now:
  

[thomaseizinger]

cc @altonen It might be of interest to you as maintainer of unsigned-varint that we are no longer using its Encoder and Decoder impl because it results in lots of temporary allocations. I think the only way this could be avoided is if the codec of unsigned-varint holds another inner codec that directly encoder item to the destination buffer.

In our case, that would be a protobuf-aware codec. With the current impl (prior to this PR), we have to encode the message into a temporary buffer because the codec in unsigned-varint only accepts Buf. After playing around with these impls, I came to the conclusion that is likely better to not use a codec for length-prefixing in the first place but rather provide a single, more elaborate codec that can efficiently encode typed messages.

This is quite significant as it affects every protobuf message sent over libp2p, e.g. kademlia, gossipsub, identify, ...

[altonen]

Thanks for the ping. I've noticed the same thing as well which is why litep2p is not directly using unsigned-varint's Encode/Decode because it had quite a negative effect on performance in some load testing.

If I ever find time, I'll take a look if the issue can be fixed in unsigned-varint because right now the situation is not good.

[mxinden]

As mentioned out-of-band, great catch!

[thomaseizinger]

@mxinden I incorporated #4911.

[mxinden]

Thanks for the find and the fix!

[mxinden]

@mergify refresh

[mergify]

refresh

✅ Pull request refreshed

[mxinden]

Reran webrtc job. Seems like it didn't get a runner.