parse timestamps (number) to Dates

[rcnavas]

It would be nice if jq can parse timestamps (in milliseconds since Epoch) to Dates.
for example:

input:
{ "created": 1405663200000 }

command:
jq '.created | @Date("yyyy-mm-dd")'

output:
"2014-07-18"

[ghost]

It could probably be implemented as a native date filter that could convert between date formats, like .created | date("original format string";"desired format string")

[perbergland]

Yes please

[nicowilliams]

Yeah, strftime()/strptime()-based utility functions for jq would be nice. I'll add this to the todo list. Thanks for the idea.

[logankoester]

Would like to have this also.

[nicowilliams]

And today I happen to need this badly too.

[esamattis]

Would it also parse ISO timestamps like 2015-03-05T19:40:53.324Z? Eg. what you get from javascript new Date().toISOString().

[ghost]

Well, it's not pretty, but you can now do this with regex to some extent. A nice thing to do is to parse the date to an array, as in [2015, 3, 5, 19, 40, 53, 324], so that the lexicographic order sorts it correctly.

[esamattis]

@nicowilliams awesome!

[nicowilliams]

I need to add strftime() and others too, but this is a start.

[pkoppstein]

@nicowilliams -- Please note:

(1) On a Mac (Yosemite with Xcode), HAVE_STRPTIME is not set. Here is the relevant section of config.log:

configure:13891: checking for strptime
configure:13919: gcc -o conftest -g -O2   conftest.c     -lonig >&5
conftest.c:32:11: error: too few arguments to function call, expected 3, have 1
strptime(0)
~~~~~~~~  ^
/usr/include/time.h:117:1: note: 'strptime' declared here
char *strptime(const char * __restrict, const char * __restrict, struct tm * __restrict) __DARWIN_ALIAS(strptime);
^

(2) The error message (strptime/2 not implemented on this platform) can be very misleading, as a jq user might think it is referring to the jq filter named strptime.

[nicowilliams]

@pkoppstein Can you check the date branch of my github clone of jq? It should fix that and add more builtins.

[pkoppstein]

@nicowilliams - I only had time to run your manual.yml test cases as detailed below.

The good news is that there were no failures; the bad news is that in all three cases, the results differ from the advertized output values.

./jq -n '"2015-03-05T23:51:47Z" | fromdate'
1425599507               # i.e. not an array

./jq -c 'strptime("%Y-%m-%dT%H:%M:%SZ")'
"2015-03-05T23:51:47Z"
[2015,2,5,23,51,47,0,0]  # vs output: ['[2015,2,5,23,51,47,4,63]']

./jq 'strptime("%Y-%m-%dT%H:%M:%SZ")|mktime'
"2015-03-05T23:51:47Z"
[2015,2,5,23,51,47,0,0]  # vs output: ['1425621107']

[joelpurra]

I can see this being a package rather than in core.

[nicowilliams]

@pkoppstein Oops, fixed.

@joelpurra Yes, indeed, but having C-coded functions in third-party modules is quite a ways off, and these are super convenient.

I was really annoyed at POSIX though. I'd completely forgotten about the sheer awfulness of mktime(), which conflates two completely different purposes and has fairly shocking side-effects, plus no standard and thread-safe way to correct for the timezone correction that it wants to apply. Plus WIN32 and WIN64 lack a strptime().

On the other hand, to deal just with ISO 8601, one could just parse and format dates in entirely jq-coded functions. But then there are the pesky leap years and leap seconds that one would have to very carefully code around. Calling the C library to handle these is very convenient, not least because any bugs can then be blamed on someone else!

[nicowilliams]

I suppose the docs should say something about localization, given the use of strp/ftime format strings... Up until now jq has completely avoided the subject of localization. If we ever add something like a printf() (which has been requested, and with good reasons), we may have to say something about localization there as well (e.g., for number formatting). Mostly I want to say that jq is agnostic as to localization, and explain where the host OS might do some that is visible to the jq user.