incorrect time returned from yyyymmddhhmmss

[SymbolixAU]

I often use a timestamp created from Sys.time() as a human-readable key (either as a char or int to show when scripts were run). Getting this back into POSIXct with anytime would be a nice addition, however, it doesn't quite get the time stamp correct:

t <- gsub("-|:| ", "", Sys.time())
> t
[1] "20160913122601"
> anytime(t, tz = "Australia/Melbourne")
[1] "2016-09-13 22:01:00 AEST"  ## the time appears incorrect

Is this something you think should be added?

[eddelbuettel]

Bug report for an 'unreleased' package. Wow :)

Now, if you look at the current list you will notice that '%Y%m%d%H%M%S%f" is not a current format. In essence, you were brave and just tried. But the format you require simply isn't there as I assembled a list of common and (to me) sane formats. Other things fail too -- for example two-digit years or non-24 hour clocks with AM/PM. Not having a separator between date and time is, well, not sane.

I plan to refactor this so you can add additional formats at run-time. I will probably do that once we have 0.0.1 on CRAN (if it gets there ;-) ). Then you can add your own local "insane" format.

What to do with with silently wrong parsing is harder. Buyer beware may be the best we can do.

This is meant to be a heuristic aid making it easier on standard formats. Yours, I think, simply is not that standard.

[SymbolixAU]

I saw your SO answer and had to try it out!

I'm aware I was being 'optimistic' with my suggestion - and because the date aspect was correct I thought I'd ask about the time component too :)

[eddelbuettel]

Yeah, I had similar issues when testing. It's tricky. But then ... so are dates, and timezones.

This is supposed to make simple things simpler. For complex stuff we still have strptime() and friends.

[eddelbuettel]

Don't you get truncation when you use it as an int?

R> f <- "20160913122601"
R> f
[1] "20160913122601"
R> fi <- 20160913122601L
Warning message:
non-integer value 20160913122601L qualified with L; using numeric value 
R> fn <- 20160913122601
R> fn
[1] 2.01609e+13
R> 

For this you must keep it as a char, else all bets are off. That way, keeping 'date' and 'time' in two words is safer...

[SymbolixAU]

you're right: I keep it as char - not int

[eddelbuettel]

Or two ints.

[eddelbuettel]

This now works (in the unreleased master branch):

R> library(anytime)
R> anytime:::addFormat("%Y%m%d%H%M%S%f") 
R> anytime("20160913202122")
[1] "2016-09-13 20:21:22 CDT"
R> options(digits.secs=6)
R> anytime("20160913202122.123456")
[1] "2016-09-13 20:21:22.123456 CDT"
R> 

Function names, status not being exported etc pp to change but in general this works. I may however not be likely to make this a permanent format for the reasons discussed above. It may well be too ambiguous.

[SymbolixAU]

Amazing!

One suggestion you may think more appropriate is the DTG used by the military: https://en.wikipedia.org/wiki/Date-time_group
https://www.refactortactical.com/blog/military-date-time-group/

[eddelbuettel]

Can you post an example of what you had in mind?

If it is just DDHHMM(Z)MONYY that already worked/works now with added formats:

R> anytime:::addFormat("%d%H%M%b%y") 
R> anytime("132021Sep16")
[1] "2016-09-13 20:21:00 CDT"
R> 

I am pretty sure you can (and always could) do the same with strptime() ....

[SymbolixAU]

sorry, yes it works with addFormat - I meant is it worth considering as a standard format?

[eddelbuettel]

Maybe a little too fringe for my taste.

[SymbolixAU]

Fair enough - it's no hassle to use addFormat each time :)

[eddelbuettel]

There are side effects when a 'wrong' format comes too early, shadowing later formats.
My money is on https://en.wikipedia.org/wiki/ISO_8601 and whatever unambiguous additions we can add. Ie adding %b is good because it helps with the nasty dd/mm or mm/dd problem. In the example above the day before %H%M is a little dicey. Maybe with more whitespace.

[eddelbuettel]

it's no hassle to use addFormat each time :)

Only once for each R session, actually.

[eddelbuettel]

It became addFormats() but this is now taken care of.

[SymbolixAU]

perfect. Thanks - I'll be making a lot of use of this package!

[eddelbuettel]

Yay!

[Kabouik]

I am late to the party, but I can't seem to display time on a variable formatted as YYYYMMDDHHMMSS and converted using anytime() and the additional format:

anytime:::addFormats("%Y%m%d%H%M%S%f") 
anytime("19960101111220")
[1] "1996-01-01 CET"

Am I missing something? Package anytime installed from CRAN R, not git, and unfortunately I know this date variable is badly formatted, but I'm using long term data bases built this way. I know I can use strptime() or even split the variable based on the number of characters, but it seemed like a good opportunity to try anytime() in my new script.

[eddelbuettel]

I can confirm that that (awful) input does not parse.

Now, anytime does not claim to parse anything for anybody anytime. Sanity in input still helps, this one we cannot accomodate. If you really know that that is the format coming in you can help yourself with the standard tools (ie strptime) for it.

Sorry.

[Kabouik]

Sounds reasonable, but I had to ask because from the messages above, it seemed that an earlier version of anytime did parse YYYYMMDDHHMMSS if provided with the given format.

I'll go with strptime() here, since I am not the author of these data sets and have to comply with the way they are formatted.

Thanks for the anytime package though, it looks very convenient for other situations.

[eddelbuettel]

Acknowledged -- but have a look at the ChangeLog / NEWS. At times we have to change / update / improve, which we try to do carefully.

And yes -- it is "most of the time" a real help. This is a corner case, as is the (documented) inability of the Boost parser to deal with single digit day and month.