Add buildMessageDelimited: size-delimited streams of Messages

[shlevy]

No description provided.

[googlebot]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.

---

• If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
• If your company signed a CLA, they designated a Point of Contact who decides which employees are authorized to participate. You may need to contact the Point of Contact for your company and ask to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the project maintainer to go/cla#troubleshoot.
• In order to pass this check, please resolve this problem and have the pull request author add another comment and the bot will run again.

[shlevy]

Note: This currently typechecks but is not tested, testing locally next.

[judah]

Note that #61 has some prior discussion of this feature.

[shlevy]

Tested to work via https://github.com/LukeHoersten/prometheus/pull/7

[shlevy]

I signed it

[judah]

Copy this from Data.ProtoLens.Encoding.Bytes?

[judah]

Would it be simpler to just call toLazyByteString (or toStrict . toLazyByteString) on a builder and then get its size?

It's hard for me to reason about the performance tradeoffs of this change. We have some benchmarks in this repo in proto-lens-benchmarks; I'd be reluctant to refactor how the building code works without seeing a noticeable improvement in those benchmarks (or a new one). I could anticipate slowdowns caused by a space leak in the _size parameter (it's hard for me to reason about off the top of my head), or due to the redundant additions/bookkeeping, or due to confusing GHC's optimizer.

[meiersi-da]

I would wary of the lazy _size as well. If we want to maximize encoding speed, I would expect that the approach used by http://hackage.haskell.org/package/cborg with an intermediate custom tailored token stream and a hot encoder loop would perform best. See: https://youtu.be/1WrSPSYhE6o?t=28m23s

[meiersi-da]

@judah @shlevy note that you could use http://hackage.haskell.org/package/bytestring-0.10.8.2/docs/Data-ByteString-Builder-Prim.html to avoid the recursion completely and perform the check whether the buffer is large enough in a single step. See the encodeCharUtf8HtmlEscaped example for inspiration.

[meiersi-da]

Recalling the times when I was working on the bytestring builder, I'd expect about an order-of-magnitude speedup for encoding varInts with this change.

[shlevy]

@judah removed premature optimization