date::parse() doesn't have a formatting character for ISO 8601 strings without 'T' separator

[kiplingw]

The ISO 8601 standard mandates as of 2019 the T separator character between the date and the start of the time. Before that it was optional.

Despite this, there is a lot of code out there which doesn't properly emit the T separator. Part of the reason is probably because people think it's optional, and by the time they find out its not there's so much code and strings already out there it's too late.

It's confusing for a lot of people. PostgreSQL output is wrong. Even ISO itself gets it wrong in its own example.

I am a fan of the robustness principle, namely that code should generate correctly formatted ISO 8601 strings. But the principle also encourages the library to still be able to consume the sometimes broken strings too.

As of write now I'm not aware of any way of elegantly handling this with the library, other than to call the library's date::parse() twice:

// Convert ISO 8601 string to a C++ time point...

    // Not all ISO 8601 strings really are correctly formatted with a T
    //  separator, despite the 2019 standard mandating it. In the event
    //  there appears to be a time separator...
    if(ISO8601String.find_first_of('T') != string::npos)
        InputStringStream >> date::parse("%FT%T %z", TimePoint);

    // Otherwise if no 'T' separator present....
    else
        InputStringStream >> date::parse("%F %T %z", TimePoint);        

    // Failed...
    if(!InputStringStream)
        throw runtime_error(
            string(_("Failed to parse last training date in ISO 8601 format: ")) +
            "\""s + ISO8601String + "\"");

One suggestion might be to add a formatting character to the library's existing list, perhaps a %q ("quirk") which consumes only a T or a single white space character. That way the above call to date::parse() could be written only once as:

InputStringStream >> date::parse("%F%q%T %z", TimePoint);

Feedback welcome.

[jleffler]

ISO 9075 (aka SQL) mandates a space where ISO 8601 expects a T. Both are standards. Both are correct. They simply different. Note that ISO 8601 allows parties to agree on different formats. It is used for information exchange. As long as the parties agree what a particular format means, ISO 8601 is content to allow (even more) alternative formats. Jonathan Leffler ***@***.***> #include <disclaimer.h> Guardian of DBD::Informix - v2018.1031 - http://dbi.perl.org "Blessed are we who can laugh at ourselves, for we shall never cease to be amused."

…

On Wed, Apr 17, 2024 at 17:05 Kip ***@***.***> wrote: The ISO 8601 standard mandates as of 2019 the T separator character between the date and the start of the time. Before that it was optional. Despite this, there is a lot of code out there which doesn't properly emit the T separator. Part of the reason is probably because people think it's optional, and by the time they find out its not there's so much code and strings already out there it's too late. It's confusing for a lot of people. PostgreSQL output is wrong <https://www.postgresql.org/docs/current/datatype-datetime.html>. Even ISO itself gets it wrong <https://www.iso.org/iso-8601-date-and-time-format.html> in its own example. I am a fan of the robustness principle, namely that code should generate correctly formatted ISO 8601 strings. But the principle also encourages the library to still be able to consume the sometimes broken strings too. As of write now I'm not aware of any way of elegantly handling this with the library, other than to call the library's date::parse() twice: // Convert ISO 8601 string to a C++ time point... // Not all ISO 8601 strings really are correctly formatted with a T // separator, despite the 2019 standard mandating it. In the event // there appears to be a time separator... if(ISO8601String.find_first_of('T') != string::npos) InputStringStream >> date::parse("%FT%T %z", TimePoint); // Otherwise if no 'T' separator present.... else InputStringStream >> date::parse("%F %T %z", TimePoint); // Failed... if(!InputStringStream) throw runtime_error( string(_("Failed to parse last training date in ISO 8601 format: ")) + "\""s + ISO8601String + "\""); One suggestion might be to add a formatting character to the library's existing list <https://howardhinnant.github.io/date/date.html#from_stream_formatting>, perhaps a %q ("quirk") which consumes only a T or a single white space character. That way the above call to date::parse() could be written only once as: InputStringStream >> date::parse("%F%q%T %z", TimePoint); Feedback welcome. — Reply to this email directly, view it on GitHub <#824>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACCAHBTHPOT2EDCVQVOBFTTY5355LAVCNFSM6AAAAABGMF3ZSKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGI2DSNBRGI2TEOI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[HowardHinnant]

That's an interesting suggestion. If we were to do it, there's two more difficult things that would need to be done:

1. Generalize it to accept any character, and provide a way to parse the character so you could inspect it.
2. Standardize it. This library is now an example implementation of C++20 <chrono>.

The second part we could do after. This library would be the field experience for the proposal. But I wouldn't want to do the field experience without being serious about standardizing it if the field experience is positive.

Here's another way to to parse this pattern. It still requires two calls to parse, but it can be done by streaming consecutively through the stream:

using TimePoint = date::sys_seconds;

TimePoint
parseISO8601(std::istream& is)
{
    date::local_days td;
    TimePoint::duration tod;
    std::chrono::minutes offset;
    is >> date::parse("%F", td);
    auto T = static_cast<char>(is.get());
    if (T != 'T' && T != ' ')
        is.setstate(std::ios::failbit);
    is >> date::parse("%T %z", tod, offset);
    if (is.fail())
        throw std::runtime_error(" ... ");
    return TimePoint{(td + tod).time_since_epoch() - offset};
}

My current feeling is that this is clean enough that I'm not motivated to extend the standard (which is a heavy lift).