Variable length encoding for enum discriminants

[dtolnay]

I think the majority of enum discriminants are less than 128. A very simple variable length encoding like encode <128 as one byte and >=128 as four bytes would address most concerns from people who are astonished that Bincode encodes their 2-variant enum with [0x00, 0x00, 0x00, 0x00] and [0x00, 0x00, 0x00, 0x01] tags.

[dtolnay]

Regarding backward compatibility - I think it would be sufficient to provide an external bincode_upgrade crate that takes a &[u8] of old data and a type T and gives you a Vec<u8> of new data.

[arthurprs]

IMHO there's little to no benefit in creating yet another varint variant. Most will fit 0-127 in a single byte and they could be just reused for sequences/map/string lengths.

[dtolnay]

Fair enough, let's do that.

[TyOverby]

LEB128 is a pretty nice format (used in the DWARF format I think) that has a solid Rust crate.

[dtolnay]

I filed #182 to provide this in an opt-in way for integer types as well.

[rnewman]

Is there consensus on making this change? I just ran into this, and I'd rather have bincode do all the work than manually serialize my enum's tag to save 3 bytes per value.

[arthurprs]

It might be nice to consider (benchmark) PrefixVarint in addition to LEB128.

[cynecx]

Has there been any plans about implementing this? I am really interested in this, as it could even further optimize my use-cases.

Actually I've been playing around replacing the variant-index with a le-prefix-varint: master...cynecx:prefix-varint (The code is really just hacky, I was confused by all the serde api stuff which I am not familiar with).

[maciejhirsz]

@dtolnay @TyOverby what about adding a varint (or some such) feature to the bincode crate that, when turned on, will treat enum discriminants and sequence lengths as varints? That should be pretty easy to do I reckon, and wouldn't break any existing code.

I'd be happy to pen a PR for this.

[jdm]

@maciejhirsz That sounds like a useful path forward. We would be willing to merge that pull request.

[tkaitchuck]

I think #294 could serve as a reasonable template for this.
Similar to how it adds support for configuring lengths of arrays and strings, a value could be added for enums.