@vmihailenco msgpack organization doesn't have golang implementation, and msgpack.org introduces
When de-serializing these new timestamps, does it make sense to consider an extension of type -1 with unrecognized length an error?
I'm currently implementing this in MPack and I'm trying to decide how to handle this. The spec doesn't explicitly say that such data should be considered invalid, but the pseudocode in the spec does say "error" if the length is not 4, 8 or 12. So I'm guessing raising an error is the expected behaviour, and judging by the implementors linking to this issue so far, this seems to be the consensus.
It seems to me that this is a bit of a problem though because data that was previously considered valid, like say an extension of type -1 and length 7, is now considered invalid and will flag errors in the newest parsers. Although negative extension types were reserved, this still feels like a backwards-incompatible tightening of the rules. It seems to contradict the new description of extensions which is that it is the application (not the library) that gets to decide whether to treat it as opaque or reject it as invalid.
I had considered reporting an extension of type -1 as a timestamp if it's parseable (i.e. the length is 4, 8 or 12 and the nanoseconds are in bounds), and otherwise just reporting it as an opaque extension object like any other instead of raising an error. Extension functions (like accessing the raw data) would still be available for extensions of type -1 regardless of length and regardless of whether it's recognized as a timestamp. This way if someone for whatever reason was using -1, this change would not break their code or data.
I suppose we can just say that nobody should have been using an extension of type -1 in the first place. So maybe it doesn't matter and I'm worrying over nothing. Still, as far as I know most libraries do not differentiate between reserved extensions and application-defined extensions (at least not through anything more than documentation.) So there's nothing stopping a user from accidentally choosing a reserved type and having it break later when libraries start rejecting their data.
For this reason I'm starting to think that maybe libraries and the spec should completely differentiate between reserved extensions and application-defined extensions. I may decide to handle reserved extensions as an entirely separate type within MPack, that way if users use them, it will be obvious that they may break in the future and they're on their own.
Something else to keep in mind with raising errors on unknown lengths is that this prevents us from inserting new lengths for timestamp in the future. For example lots of people wanted timezones, so I had previously suggested that we could extend this later to add one or two bytes to each length to specify it. This is actually not possible because it won't be backwards compatible. Existing libraries will report a parsing error (probably discarding the whole message) when they encounter a timestamp with the new lengths. So this means that this definition of timestamps is frozen forever: no other lengths can be added, and any change or addition will have to be introduced as a different extension type.