0.1.0 datetime handling does not conform to iso 8601

[korobkov-mindbox]

In JSON serialization it is generally considered standard to use string-like datetime format. For example, .NET Newtonsoft.JSON serializes datetime values as strings: {"Now":"2018-11-14T15:47:11.881469+03:00"}, same as javascript's JSON library.

Prior to 0.1.0 release it was possible to write our own post_init method to decode such datetime fields:

@dataclass_json
@dataclass(frozen=True)
class ExampleObject:
    SomeDate: datetime

    def __post_init__(self):
        current_value = self.__dict__['SomeDate']
        if isinstance(current_value, str):
            self.__dict__['SomeDate'] = dateutil.parser.parse(current_value)

Release 0.1.0 breaks this behaviour and generally makes it impossible to work with commonly-accepted string-like rfc/iso datetime formats by always forcing datetime representation to be int-like due to this code.

TypeError : an integer is required (got type str)
at
...
File "api.py", line 65, in from_json
return _decode_dataclass(cls, init_kwargs, infer_missing)
File "core.py", line 69, in _decode_dataclass
infer_missing)
File "core.py", line 84, in _decode_dataclass
dt = datetime.fromtimestamp(field_value, tz=tz)

As of now we are forced to use 0.0.25 release, however it would be of great help if we could rely on dataclasses-json to natively handle datetime representations without our own hacks.

Btw, using string-like representation also has inherent benefit of allowing you to store timezone info in the serialized value, ensuring that encoding->decoding->encoding operation is strictly identity (note that d->e->d still will not be strict identity because serialization format may change, but that is okay as you still keep all timezone-related information)

[lidatong]

Hi, thanks for raising this use case!

I agree that ISO is a standard format but would say UNIX time is also a standard format when working with datetime. There were tradeoffs I considered when picking one or the other, and I decided to go with UNIX time via timestamp primarily because I think it's simpler (especially in that it doesn't require the client of the JSON to have an ISO parser).

However I definitely think you should be able to serialize/deserialize from ISO if you pick that as your format, given it is standard! This section of the docs show how you can do it in the latest release.

By the way, there is currently a bug in marshmallow that causes datetime to serialize. So if you're using .schema(), I recommend using your own custom field for now instead of fields.DateTime(format='iso').

You can import the one I made in from dataclasses_json.mm import _IsoField (note _IsoField is intentionally private and could be removed unexpectedly as soon as the marshmallow datetime bug is fixed).

[korobkov-mindbox]

Thanks, but I think we'll stick with 0.0.25 then. As is, it enforces dependency on marshmallow and requires metadata to be configured per field rather than solution-wide, same as our current workaround with post_init.

ISO parser is hardly a problem, given that it is available in dateutil for all pythons between 2.7 and 3.7 except 3.0, 3.1 and 3.2 and is part of the datetime package as of 3.7

Also I fail to see how requiring additional steps to parse datetime from JSON emitted by browser, .NET, JAVA and Haskell keeps it simple. One of the points of JSON is that it is supposed human-readable, which timestamps are obviously not.

Also, per RFC 7493 / Interoperable JSON:

Protocols often contain data items that are designed to contain timestamps or time durations. It is RECOMMENDED that all such data items be expressed as string values in ISO 8601 format, as specified in RFC 3339

Additionally, for obvious reasons timestamps both lack timezones and do not work for any date prior to 1970. There's still a lot of people born all over the world before 1970, and quite a number of historical events all over the world transpired before that.

Even Haskell's Aeson documentation implies that ISO format is the default one:

UTCTime and ZonedTime are represented as strings according to the ISO 8601 standard (specifically, the ECMA-262 subset), which every other JSON parser should understand as well.

From the memory the only library that uses timestamp format in JSON is ASP.Net, and even they use string representation '/Date(1224043200000)/' rather than plain number. And, because of that, nobody likes them and uses Newtonsoft.JSON instead (there was even discussion about including it in .NET Standard).

Could you please explain what platforms you target for compatibility when configuring default behaviour of your library? Also why do you believe it is bad practice to depend on clients having dateutil (post-3.7 even this is not required), but depending on marshmallow is okay?

edit: added a reference in text, fixed formatting