time: enforce strict parsing for RFC3339 for Time.Unmarshal

[dsnet]

When calling:

time.Parse(time.RFC3339, s)

one would expect that this strictly parses s according to RFC 3339 as the constant suggests.
This parses all valid RFC 3339 timestamps (except those with leap-seconds),
however it fails to reject certain inputs that are not valid RFC 3339.

Parse differs from the RFC 3339 syntax in the following ways:

1. Parse permits the hour fields to only have one digit, while RFC 3339 requires that it always be two digits.

   • See time: Parse of RFC3339 is incorrect for hour field #54568.
   • For example, 0000-01-01T0:00:00Z is considered valid, when it should not be.
   • There is an open issue (time: one-digit hour is accepted for 15:04:05 template #29911)
     requesting that "15" in the format be always parsed as a two-digit field.
     However, it is unlikely that this behavior will be changed due to it being a breaking change.

2. Parse permits a comma separator between seconds and sub-seconds, while RFC 3339 only permits a period.

   • See time: Parse of RFC3339 is incorrect for fractional second seperator #54571.
   • For example, 0000-01-01T00:00:00,000Z is considered valid, when it should not be.
   • This behavior changed recently in time: support Decimal Comma for fractional seconds #43823,
     which modified the behavior of "." in the format to parse either "." or ",".
     While this expands the set of ISO 8601 timestamps we can handle,
     it further breaks compliance with RFC 3339.

3. Parse permits time zone offsets with any range of values, while RFC 3339 limits hours to be less than 24 and minutes to be less than 60.

   • For example, 0000-01-01T00:00:00+24:60 is considered valid, when it should not be.

In all situations, the time format template provides no way to strictly parse RFC 3339.

Since we can't change the text template meaning of "15" and ".",
nor can we change the constant value of RFC3339 or RFC3339Nano,
I propose that we special-case Parse to use strict parsing when handling a format template
identical to RFC3339 and RFC3339Nano.

Justification for this special-casing:

• RFC 3339 is by far the most popular time representation.
• RFC 3339 exists because ISO 8601 had too many alternate representations.
  Being loose in what Go considers "valid" RFC 3339 is going against the goal of RFC 3339.

\cc @robpike

[apparentlymart]

I like the goal of having a strict parser for this format, but it feels surprising to me for Parse to retroactively treat specific format strings as special.

I would find it more intuitive to have a separate function specifically for parsing RFC 3339 per its specification, without any implication that it's "just another format string". Perhaps that could be accompanied by a deprecation of the constant RFC3339 to discourage its use in favor of the function, so that the existing constant will be less of an attractive nuisance for those new to the package looking for an RFC3339 parser. 🤔

This would also avoid changing behavior for any existing users of the current API, allowing existing callers to adopt the new function when they are ready.

[dsnet]

The UnmarshalText and UnmarshalJSON methods are already documented as operating according to RFC 3339, what do we want to do with those? We either:

1. Say that the Unmarshal methods are stuck on old incorrect behavior, OR
2. We accept the fact that we have to change existing behavior.

I'm sympathetic to not changing the Parse function since special-cases are not pretty. I feel more strongly that we change the behavior of the Unmarshal methods, otherwise the unmarshaling of time.Time will perpetually be non-compliant with RFC 3339.

As for whether we are allowed to make this change, the Go 1 compatibility document says:

Specification errors. If it becomes necessary to address an inconsistency or incompleteness in the specification, resolving the issue could affect the meaning or legality of existing programs. We reserve the right to address such issues, including updating the implementations. Except for security issues, no incompatible changes to the specification would be made.

Since functionality that claims to be RFC 3339 do not operate according to RFC 3339, this is an issue of "legality" and the document gives the package the "right to address such issues".

Also, this could be an interesting use of the GODEBUG flags discussed in #55090. You could imagine having a GODEBUG=timestrictrfc3339=0 override that opts into the older behavior.

[apparentlymart]

I don't typically like to get into the business of rules-lawyering based on exactly what the docstrings say, but this seems like a fiddly enough case that using the existing documentation as guidance might be relevant.

The time.Parse function is documented as parsing a timestamp in a general way based on a format string with a particular syntax. The specific thing that would "feel surprising" to me is for that to retroactively gain an exception like "...unless the format string is exactly what's in time.RFC3339, in which case it uses a different parser".

On the other hand, the UnmarshalText method is explicit in its documentation that it expects RFC 3339 format and says nothing about how it achieves that. The fact that it currently uses time.Parse with time.RFC3339 is an implementation detail rather than an explicit contract.

The time.RFC3339 constant has no direct documentation of its own because all of the time format constants are documented together in one large documentation comment. The name of the symbol certainly implies that it's intending to be a description of the RFC3339 format though.

Based purely on the contract described in the docs then, it seems reasonable to me that there be a new time.ParseRFC3339 function which implements exactly the grammar in the RFC, and then UnmarshalText's implementation changes to call ParseRFC3339 instead. The documented contract for both Parse and UnmarshalText is preserved, and anyone who wants to strictly accept RFC 3339 could either use MarshalText (to rely on that documented contract) or could call time.ParseRFC3339 directly.

Any existing callers using time.Parse will still get the behavior as originally documented, which seems to be the closest superset of RFC3339 that the documented format syntax is capable of describing.

I would agree that the existence of the time.RFC3339 constant itself seems to be a "specification error" in the sense that the format string syntax used by time.Parse is not actually capable of representing the RFC 3339 requirements. That seems like reasonable justification for deprecating it in favor of a function that does correctly implement RFC 3339, even though there is no possible format string (per the currently-documented syntax) that can accurately describe those requirements. (I suppose this strategy would also require adding a Time.FormatRFC3339 method, because deprecating the constant would also deprecate using time.Format(time.RFC3339) even though that format string is technically capable of producing valid RFC 3339 strings.)

While I do appreciate that it would be possible to use GODEBUG to temporarily dynamically recover the existing implementation, that doesn't feel quite right to me in this case because:

• Different parts of a program are likely to have differing requirements.

  For example, HashiCorp Terraform has a public-facing function which is documented to accept RFC 3339 strings but uses time.Parse(time.RFC3339, ...) and so is accidentally currently accepting the superset described by that format string. Terraform is also subject to compatibility promises for its own functions and so Terraform may need to, at least for some time, retain the more liberal parser and potentially generate deprecation warnings for anyone relying on the old implementation. However, other parts of Terraform consume wire formats with different tradeoffs where stricter parsing would not be problematic in the same way and would be better to switch to stricter parsing earlier.

• The time.Parse function is specified not as an RFC 3339 parser but as a general parser for various time formats that can be described by a special microsyntax. Therefore it feels a bit dishonest to say that its failure to handle parsing rules outside of the bounds of its documented syntax is a "specification error": it's behaving as specified when given the particular format string that is specified in time.RFC3339.

  I think the time.RFC3339 constant is reasonable to describe as a specification error, because its name implies a promise it's unable to keep. However, a constant doesn't have any inherent behavior of its own so a GODEBUG-based approach wouldn't work for that, whereas deprecation in favor of a new explicit function could potentially work.

[dsnet]

It seems we're in agreement that the Unmarshal methods should change behavior. I'm not quite fond of adding new API like ParseRFC3339. If we change the Unmarshal methods, that may be sufficient. The way to call it will be differ:

• ParseRFC3339 would return a time.Time, while UnmarshalText mutates the receiver,
• ParseRFC3339 would accept a string, while UnmarshalText accepts a []byte.

Different parts of a program are likely to have differing requirements.

That's true of all the GODEBUG flags. It is a global change of behavior that may be inconsistent in what a particular library wants. The use of GODEBUG will not fix all cases of changed behavior in the standard library but it provides users a knob that is better than nothing.

[apparentlymart]

Indeed, I think it's defensible that the current implementation of UnmarshalText is buggy in that it claims to implement RFC 3339 but it doesn't. It may still have impact on existing users, but in a way that can be clearly justified in terms of its existing specification.

It is indeed true that all GODEBUG flags are global in nature, but that's a more appropriate solution for some situations than others. For example, using GODEBUG to choose between resolver implementations seems reasonable because looking up hostnames is typically (though certainly not always) a global concern that the entire program should agree on. I hesitate to consider it an appropriate solution for all possible changes to the standard library, since there are various situations where the decision is inherently more localized.

My sense is that GODEBUG has traditionally been a last resort for situations where the new behavior is "obviously better" but where some small number of programs may need to retain the old behavior. I guess I'm not convinced that a special case in Parse to handle something it's never claimed to support is "obviously better", and that some specialized API that directly addresses the evident use-case (strictly parsing RFC 3339 exactly as it's specified) is both better for backward compatibly and, by my estimation, easier to learn and understand than a special exception buried inside the Parse docstring. (This "easier to learn and understand" is assuming that time.RFC3339 would also be deprecated so that authors will see feedback if they notice it and try to use it. Otherwise it's likely to be an attractive nuisance.)

[gopherbot]

Change https://go.dev/cl/444277 mentions this issue: time: implement strict RFC 3339 during marshal and unmarshal

[dsnet]

I sent out https://go.dev/cl/444277 which only special-cases strict RFC 3339 for the Marshal and Unmarshal methods since they explicitly say that they operate according to RFC 3339.

There is also existing precedence for strict checking since the Marshal methods already check that the year is within the right range.

In my opinion, modifying just Marshal and Unmarshal is a justifiable bug fix that doesn't require a proposal anymore.

[dsnet]

I'm withdrawing this proposal. I think fixing the marshal/unmarshal methods is sufficient.