jsonlog: Use as compatible as possible ISO-8601 date format

[gsstark]

If log_timezone is GMT use the maximally compatible format
1975-08-19T23:15:30.000Z as generated by JavaScript (see the demo at
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toJSON)
and which is accepted by jq.

Otherwise respect log_timezone (even though it seems like a bad idea
to use non-GMT in a machine-readable format) but use ISO-8601 numeric
timezone offset such as 1975-08-19T23:15:30.000+05:00 which at least
will be readable by Joda-based parsers such as ElasticSearch.

This should guarantee that the logs are usable on ElasticSearch and if
you use log_timezone=GMT then they should be readable by any json
parsing tool.

(Use pg_get_timezone_offset so it doesn't matter how you spell GMT but
the timestamp format doesn't just change back and force spontaneously
if you use a timezone that has offset 0 for part of the year)

[michaelpq]

So, if I switch use %Z then I get that:
timestamp":"2018-03-09T10:29:02.629 JST
If I use %z then I get that:
"timestamp":"2018-03-09T01:32:10.223 +0000"

Now if I use your patch with log_timezone = 'GMT', then I get that:
"timestamp":"2018-03-09T01:36:52.117Z"
And without GMT I get that:
2018-03-09T10:37:37.016+0900

As I am not much willing to break the parsing logic that any consumers of this plugin use, what about doing the following instead: let's introduce a GUC parameter able to control the timestamp format, which can take the following values.

• timezone_name, which is the default, and uses the timezone name unconditionally.
• timezone_offset, which uses the timezone offset all the time.
• javascript, which is what you are looking for here, and more or less a modified version of the second one.

Note: please be careful about the indentation of your patch, and running git diff --check would remove any noise diffs.

[gsstark]

On 9 March 2018 at 01:40, Michael Paquier ***@***.***> wrote: As I am not much willing to break the parsing logic that any consumers of this plugin use

The timestamps were entirely broken for over three years.... And the whole point of using JSON is to not have to write any parsing logic. I expect I want to use it for the same reason others want to use it, so I can use standard tools that already exist like jq and ElasticSearch. I don't know of any tool that will accept these custom almost-iso8601 dates.

, what about doing the following instead: let's introduce a GUC parameter able to control the timestamp format, which can take the following values. - timezone_name, which is the default, and uses the timezone name unconditionally. - timezone_offset, which uses the timezone offset all the time. - javascript, which is what you are looking for here, and more or less a modified version of the second one. I still think that's a lot less useful than having a standard format. It

means you can't take the json from any install and process it without first knowing the configuration setting for the database it was generated on.

…

-- greg

[gsstark]

In fact. The more I think about it....

I think jsonlog should just ignore the log_timezone entirely. It should print the timestamp in UTC using the iso standard "Z" for the timezone.

log_timezone makes sense for human-readable text logs. However for machine-readable logs in a standard format there's no use case for it at all. Users will view these json logs in some UI which will be capable of converting to whatever timezone they wish.

The current situation is that a user could get logs unreadable in jq or other standard tools and even with my patch the only story for them is "oh, you have to set this parameter to UTC to get json logs that standard json tools can read".

So I'm going to update this pull request to ignore log_timezone entirely. I really think that's the only sensible approach.

[michaelpq]

So I'm going to update this pull request to ignore log_timezone entirely. I really think that's the only sensible approach.

OK. If you have a new patch I'll try to look at it asap. On my side, I have been thinking a bit about it and at the end I don't want to make you unhappy as you seem to be using this plugin more than I do ;)

[michaelpq]

Just pushed a version close to #22 so this one is out of the game.