Support serializing to Vec<u8> with unknown seq/map length

[dtolnay]

When the sequence is done it should go back and write the length in the correct spot.

Though variably encoding the length may make this less feasible - cc #75.

[arthurprs]

Is that a problem in practice? I think the majority of binary serialization formats have problems supporting unknown lengths.

[TyOverby]

I'm not sure how useful this. How many people encode using different types than they decode?

[JoshMcguigan]

If I'm understanding this suggestion correctly, it would resolve #245. I thought this could be implemented by tracking the index into our underlying vec when creating a Compound, then going back and updating the appropriate size at that starting index in the vec during the SerializeMap::end method.

The trouble is all of the serialization methods in bincode are generic over the std::Write trait, which doesn't allow the type of access we need into the underlying vec. I'm not sure if there is a better way to handle this? Or if perhaps there are additional trait bounds that would help here without limiting the API too much?

Edit - Actually I think there is a way to do this. The Compound struct could store the bytes to write in a vec of it's own, as opposed to writing them immediately to the underlying writer. Then only at the end the Compound struct could write the length, plus the data. There may be a performance trade-off for this though? If there is interest, I'd be happy to start a PR for this.

[MTRNord]

Any Idea when this will get any kind of fix? :(

[ZoeyR]

@MTRNord so far there has not been a proposed solution that does not come with significant drawbacks. Implementing this is held up until we can identify an implementation that doesn't come with those drawbacks.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[mitsuhiko]

This issue should be reopened. It's still a big problem, particularly when working with structs involving serde's flatten feature.

[nmccarty]

We have decided to declare this issue out of scope.

Bincode only officially supports types that serialize to a known length, and the initial proposed solution of going back and rewriting the correct length after serializing the sequence both causes issues with varint encoded lengths, and can not be made general as it can not be assumed that any given writer supports Seek, and specializing for Vec<u8> would incur unacceptable performance penalties for common usecases.