Maximum string length changes depending on linked components

[alexcrichton]

Currently the canonical ABI defines a maximum string length where the upper bit can be reserved for a tag in the utf16+latin1 encoding, but the way this is currently specified to be enforced means that if a component tries to pass a string of length MAX_STRING_BYTE_LENGTH-1 that may work depending on the linked component. If a component using the utf-8 encoding of strings attempts to pass this string to a component using utf-16 encoding then the doubling of the string byte length will exceed MAX_STRING_BYTE_LENGTH which will cause a trap. If linked to a utf-8 component this would end up working (assuming the components can actually transfer gigabyte-scale strings which albeit is a bit out-there).

Should the maximum byte length for strings be further limited to ensure that no matter what component you're linked to the same maximum byte limit is always in effect?

[lukewagner]

That's a great question and one I wrestled with a bit too. It's clearly nicer to try to spec something encoding-independent, like number-of-USVs, but we only know that after traversing (read: allocating and copying) a string, so we can't do that (w/o forcing a pre-pass). One thing that made me feel a bit better about this is that it's basically a special-case of resource exhaustion, which is internal-representation-dependent anyway. Moreover, MAX_STRING_BYTE_LENGTH is big enough that on many wasm platforms, you'll have hit the platform-defined memory size limit (which, again, will be based on memory.size and thus encoding) well before, so as a general rule, one should avoid sending "big" strings (and lists, for that matter) by value. (For big strings/lists, a stream is your better option.)

If we really wanted to, we could spec checking the USV-count after performing the realloc+copy (thereby requiring the impl to maintain a USV count as it copies). So you'd end up with an early byte-size overflow check (rejecting definitely-too-big cases) followed by a late USV-count check (tightening up the requirement) and this would force more-regular behavior. My impression was that this regularity wasn't worth the cost (again, considering resource exhaustion), but I'm certainly willing to discuss the option more or consider alternatives.

[alexcrichton]

That sounds reasonable to me. This struck me as odd initially but the parallels to memory.size and the choice of encoding makes sense to me.