Enforce Message Size Limits

[sarvalabs-manish]

Enforcing Size Limits

• Protocol Buffers messages have a hard limit of 2 GB because most implementations use signed 32-bit signed arithmetic (determines the max encodable tag value). Google's official implementation enforce a hard limit of 64 MB because the entire message needs to be loaded into RAM to be decoded. Read More
• Currently go-polo does not enforce any size limit and uses 64-bit unsigned arithmetic and hence has a much higher potential size. This needs to be artificially limited to 64 MB to make the implementation crash tolerant while loading large messages into memory. There has been no instance of such crashes (yet) but better safe than sorry. (This must probably also be enforced by the specification across implementation)
• Consideration for size limiting are only for free sized wire types:
  • Integers and Floats use under 8 bytes, Booleans use space and Varint Tags are a maximum of 10 bytes.
  • This leaves BigInt and Word as the atomics that can have arbitrary sized data. (need their own size limit enforcement)
  • Pack and Doc encoded data must implicitly ensure that the size of sub-elements collectively does not exceed the size limit. This check will filter down to arbitrary sized encoding as their encoded output will be rejected by the pack data.

Implementation Approach

• As mentioned in Allow Polorizer & Depolorizer Option Flags #29, we will use an EncodingOption to determine the allowed message size and default the wire config value to 64MB and disallow values over 2GB in the encoding option