Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upanydate transforms to previous day #5
Comments
|
That seems to be a timezone error on your end. What does FWIW, this does not reproduce: > anydate(20150101)
[1] "2015-01-01"
> anydate("2015/01/01")
[1] "2015-01-01"
> anytime(20150101)
[1] "2015-01-01 CST"
> anytime("2015/01/01")
[1] "2015-01-01 CST"
> anytime:::getTZ()
[1] "SystemV/CST6CDT"
> Please try with explicit |
|
Also: R> anytime("2015/01/01", tz="CET")
[1] "2015-01-01 07:00:00 CET"
R> anydate("2015/01/01", tz="CET")
[1] "2015-01-01"
R> |
|
Previous example was from Windows machine (will check tomorrow), but the same is on Linux: and |
|
Thanks for the follow-up. Maybe we check some more tomorrow. Here R> tstr <- "2016-01-01 01:02:03"
R> a <- unclass(as.POSIXlt(as.POSIXct(strptime(tstr, "%Y-%m-%d %H:%M:%S"))))
R> b <- unclass(as.POSIXlt(anytime(tstr)))
R> all.equal(a, b)
[1] TRUE
R> Do that locally and see if |
|
I also get the same issue, running on a windows machine and RStudio in Sydney Australia |
|
Debugging help welcome. "Works here" as the saying goes. Below is from R 3.3.1 in a win7 VM. So works across OSs in my TZ. R> library(anytime)
R> anydate(20150101)
[1] "2015-01-01"
R> anydate("2015/01/01")
[1] "2015-01-01"
R> anytime(20150101)
[1] "2015-01-01 CST"
R> anytime("2015/01/01")
[1] "2015-01-01 CST"
R> anydate("2015/01/01", tz="CET")
[1] "2015-01-01"
R> anydate("2015/01/01", tz="Australia/Sydney")
[1] "2015-01-01"
R> Sys.info()[1:3]
sysname release version
"Windows" "7 x64" "build 7601, Service Pack 1"
R> |
|
I can however replicate it so that is a first step in the right direction: edd@max:~/git/anytime(master)$ R --slave -e 'source("tests/simple.R", echo=TRUE)'
R> Sys.setenv(TZ = "Australia/Sydney")
R> library(anytime)
R> anydate(20150101)
[1] "2014-12-31"
R> anydate("2015/01/01")
[1] "2014-12-31"
R> anytime(20150101)
[1] "2015-01-01 AEDT"
R> anytime("2015/01/01")
[1] "2015-01-01 AEDT"
R> anydate("2015/01/01", tz = "CET")
[1] "2014-12-31"
R> anydate("2015/01/01", tz = "Australia/Sydney")
[1] "2014-12-31"
edd@max:~/git/anytime(master)$ |
|
And I may have a fix. If we first pass to edd@max:~/git/anytime(master)$ R --slave -e 'source("tests/simple.R", echo=TRUE)'
R> Sys.setenv(TZ = "Australia/Sydney")
R> library(anytime)
R> anydate(20150101)
[1] "2015-01-01"
R> anydate("2015/01/01")
[1] "2015-01-01"
R> anytime(20150101)
[1] "2015-01-01 AEDT"
R> anytime("2015/01/01")
[1] "2015-01-01 AEDT"
R> anydate("2015/01/01", tz = "CET")
[1] "2014-12-31"
R> anydate("2015/01/01", tz = "Australia/Sydney")
[1] "2015-01-01"Try version 0.0.1.2 now in GitHub, or else wait for 0.0.2. More testing would be welcome. And thanks a ton for this heads-up. |
|
I've updated to the latest version 0.0.2. The problem still persists on my machine. Is there another TZ setting I need to set? |
|
I have no idea; this does reproduce here: R> Sys.setenv(TZ = "Australia/Sydney") # important: set this before loading anytime
R> library(anytime) # will memoize TZ
R> packageVersion("anytime") # 0.0.2 on CRAN vs 0.0.2.1 on GH shouldn't matter
[1] ‘0.0.2.1’
R> anydate(20150101)
[1] "2015-01-01"
R> I fear that you are on your own until you have something I can reproduce. |
|
Maybe it's a Windows issue, maybe it isn't. I wrote anytime mostly for |
|
One way to help would be to examine all the subcomponents of |
|
I just tried on windows7 and there too I am unable to replicate this: R> Sys.setenv("TZ"="Australia/Sydney")
R> library(anytime)
R> packageVersion("anytime")
[1] �0.0.2�
R> anydate(20150101)
[1] "2015-01-01"
R> Sys.info()[1:3]
sysname release version
"Windows" "7 x64" "build 7601, Service Pack 1"
R> Sys.time()
[1] "2016-09-20 03:27:04.65765 AEST"
R> |
|
Hi, I just had a thought, looks like anytime() is returning a date/time that is 1 hour too early. I'm guessing it could have something to do with daylight saving time? I wonder if it will fixed when we put clocks forward by 1 hour on 2-Oct-2016. |
|
Yes -- please try to look at |
|
I get: |
|
Ok, thanks, that shows it as 'too late'. The day and hour have already shifted. |
|
But I just noticed something. On my Windows box I got (see above) whereas you have |
|
I'm seeing: |
|
Never mind, false alert. AEDT was for Jan 01. In short, I have no relevant idea and cannot reproduce. Sorry. |
|
Why is your whereas mine returns a date and time: |
|
Standard behaviour of |
|
Here in Sydney, Dec 31 happens during DST. If someone in Brisbane could test, that would be helpful too (no DST in Brisbane). The result is a bit odd: "2014-12-31 23:00 AEST" is the same time as "2015-01-01 0:00 AEDT". In the example below, it looks like it's taken the numbers for AEST but stuck the "AEDT" symbol on the end (but we all know that looks can be deceiving). $gmtoff is 11h, which is correct for AEDT. Using anytime v 0.0.3, on R 3.3.1, x86_64, mingw32, on Windows 10: |
|
If only I knew where that hour offset creeps in. |
|
It gets a little weirder on Linux (Debian 7, R.version 3.1.1, anytime 0.0.3): Checking the time and zone: Which is correct (at time of typing). |
|
anytime use gettz to help it get a timezone as a fallback. What does it do for you? R> library(anytime)
R> anytime(Sys.time(), tz="SystemV/CST6CDT")
[1] "2016-10-16 20:51:01.303017 CDT"
R> anytime(Sys.time(), tz="America/Chicago")
[1] "2016-10-16 20:51:10.63891 CDT"
R> anytime:::getTZ()
[1] "SystemV/CST6CDT"
R> I get the correct (nerdy) representation of Chicago-time as "CST6CDT". Oh, and for Jan 1: R> anytime(20150101)
[1] "2015-01-01 CST"
R> anydate(20150101)
[1] "2015-01-01"
R> |
|
I never spotted that: Every day I learn that time zones are even messier than I believed yesterday. :-/ |
|
Ditto, yet it ain't working: R> Sys.timezone()
[1] "SystemV/CST6CDT"
R> Sys.timezone(location=TRUE)
[1] "SystemV/CST6CDT"
R> |
|
g++, yes. Boost: I only have libbost-iostreams. If I install the full libboost, should I re-install anytime? |
|
Try installing anytime from source on Windows, and look at the actual commands. It will point |
|
The Windows box has BH installed (probably from binary), but no Boost I'm aware of apart from that. I was able to successfully install anytime from source. Next step? |
|
The bug is still there: |
|
Yes, sure we haven't drilled yet. Your Boost-via-header-only-package BH is here: -I"C:/Program Files/R/R-3.3.1/library/BH/include" You should now be able to do the equivalent of this (by adding $ g++ -o /tmp/boost_date_parser boost_date_parser_cmdline.cpp
$ /tmp/boost_date_parser "2016-01-01 20:21:22"
ptime is 2016-Jan-01 20:21:22 -- seconds from epoch are 1451679682 from 2016-01-01 20:21:22
$ Code below. // -*- mode: C++; c-indent-level: 4; c-basic-offset: 4; tab-width: 4 -*-
// cf http://stackoverflow.com/a/3787188/143305 and extended
#include <iostream>
#include <boost/date_time.hpp>
namespace bt = boost::posix_time;
const std::locale formats[] = {
// std::locale(std::locale::classic(),new bt::time_input_facet("%x")),
std::locale(std::locale::classic(),new bt::time_input_facet("%Y-%m-%d %H:%M:%S")),
std::locale(std::locale::classic(),new bt::time_input_facet("%Y%m%d %H%M%S")),
std::locale(std::locale::classic(),new bt::time_input_facet("%Y/%m/%d %H:%M:%S")),
std::locale(std::locale::classic(),new bt::time_input_facet("%m/%d/%Y %H:%M:%S")),
std::locale(std::locale::classic(),new bt::time_input_facet("%m-%d-%Y %H:%M:%S")),
std::locale(std::locale::classic(),new bt::time_input_facet("%d.%m.%Y %H:%M:%S")),
std::locale(std::locale::classic(),new bt::time_input_facet("%Y-%m-%d")),
std::locale(std::locale::classic(),new bt::time_input_facet("%Y%m%d"))
};
const size_t formats_n = sizeof(formats)/sizeof(formats[0]);
std::time_t pt_to_time_t(const bt::ptime& pt) {
bt::ptime timet_start(boost::gregorian::date(1970,1,1));
bt::time_duration diff = pt - timet_start;
return diff.ticks()/bt::time_duration::rep_type::ticks_per_second;
}
void seconds_from_epoch(const std::string& s) {
bt::ptime pt, ptbase;
for (size_t i=0; pt == ptbase && i < formats_n; ++i) {
std::istringstream is(s);
is.imbue(formats[i]);
is >> pt;
}
if (pt == bt::ptime()) {
std::cerr << "Parse error for " << s << '\n';
exit(-1);
}
std::cout << "ptime is " << pt << " -- seconds from epoch are "
<< pt_to_time_t(pt) << " from " << s << '\n';
}
int main(int argc, char *argv[]) {
if (argc < 2) {
std::cerr << "Usage: " << argv[0] << " arg1 [arg2...]\n";
exit(-1);
}
for (int i=1; i<argc; i++) {
seconds_from_epoch(argv[i]);
}
exit(0);
} |
|
It'lll either be at this layer, or one step down in anytime when we convert to numeric representation to feed into You are being extremely helpful. Let's see if we can nail this thing over the next few days. Boost Date_time documentation is not for the faint of heart, but there are examples.... |
|
I'll be around for the next few days if you need some sleep. :) |
|
Ok, here we go go go... |
|
Very very good. Now if you poke into anytime's sources you will find the In it, I compute a |
|
Before we chase that: does boost_date_parser_cmdline.exe know about the local time zone, or is it assuming UTC? I ask because looking at that integer: Or is none of that relevant, because it's not yet relevant? :) |
|
It is set to work on localtime -- it is at the beginning of |
|
I commented out in here: And that seems to remove one bug and add another. So now on to the $64,000 question: when and how to apply the DST correction under Windows Downunda. :) |
|
Right. Boost and R seem to disagree about how to deal with Australia. Between them, one will be wrong. |
|
When an idea hits, I'll be very happy to help where I can. Updates on this thread go to my Inbox. Good luck! |
|
If you have any ideas.... |
|
No ideas, but a question: is the BH package identical on Windows and Linux? i.e. nothing compiled. The Boost docs suggest it probably is. Just working out where to look. |
|
No, you just misunderstand how Boost and header-only libraries work -- see eg here on Wikipedia for quick take. There are LOTS of them with just headers. The parts of Boost which require linking are typically i/o related (eg for Boost Date_time you must link if you want to format strings rather than parse them). All the OS-dependent stuff is dealt with via |
|
I wrote a little script that extracts all possible value for When I run that, all 611 values come out equal. I can email you the script--drop me a line at the usual email of edd@debian.org. |
|
The fact that it works for you on Linux is encouraging. This may all come down to a bug in the entry used on Windows. Now to find whether it is R, or Boost. I'll follow-up with a little R/C++ hybrid making use of the 611 TZ values. |
|
I thought I had something but I don't. Boost would need its own timezone db to do what I planned to do --> over kill. |
|
@jason-turner2 Any chance you could give the current master branch a spin, particularly the UTC parsing? That should give us a clue vis-a-vis the time offset as we can not pin things down to UTC-to-localtime-by-R-only. |
|
@eddelbuettel , happy to do it. Might not be able today; will definitely get to it this week. |
|
Sounds good! We should keep each other honest and try to chase this one down. Having |
|
@jason-turner2 : I think i found something crazy. First off, I had though the Rcpp::Rcout << "FYI, isdst is " << localAsTm->tm_isdst << std::endl;as the penultimate line in Then some experiments: Brisbane only every gets AEST, never AEDT ?$ Rscript -e 'library(anytime); anytime:::setTZ("Australia/Brisbane"); anytime("2016-07-11"); \ print(anytime:::getTZ())'
FYI, isdst is 1
[1] "2016-07-11 15:00:00 AEST"
[1] "Australia/Brisbane"
$ Rscript -e 'library(anytime); anytime:::setTZ("Australia/Brisbane"); anytime("2016-01-11"); \print(anytime:::getTZ())'
FYI, isdst is 0
[1] "2016-01-11 16:00:00 AEST"
[1] "Australia/Brisbane"
$Sydney and Canberra get a two-hour difference?$ Rscript -e 'library(anytime); anytime:::setTZ("Australia/Sydney"); anytime("2016-08-11 12:00:00"); print(anytime:::getTZ())'
FYI, isdst is 1
[1] "2016-08-12 03:00:00 AEST"
[1] "Australia/Sydney"
$ Rscript -e 'library(anytime); anytime:::setTZ("Australia/Sydney"); anytime("2016-02-11 12:00:00"); print(anytime:::getTZ())'
FYI, isdst is 0
[1] "2016-02-12 05:00:00 AEDT"
[1] "Australia/Sydney"
$ Rscript -e 'library(anytime); anytime:::setTZ("Australia/Canberra"); anytime("2016-02-11 12:00:00"); print(anytime:::getTZ())'
FYI, isdst is 0
[1] "2016-02-12 05:00:00 AEDT"
[1] "Australia/Canberra"
$ Rscript -e 'library(anytime); anytime:::setTZ("Australia/Canberra"); anytime("2016-08-11 12:00:00"); print(anytime:::getTZ())'
FYI, isdst is 1
[1] "2016-08-12 03:00:00 AEST"
[1] "Australia/Canberra"
$ All this was on Ubuntu 16.04.1; might be interesting on Windows too. CCing @bobjansen who wants to help with testing. Six eyes probably beat four. |
|
I did this:
There is certainly some weirdness. I added a line to print each result with the GMT offset, to keep from confusing myself. It didn't work; I'm confused. Queensland (the state that contains Brisbane) never does DST, but the isdst flag shown during the test is correct for the states that do (e.g. NSW, Victoria, etc). Brisbane seems to subtract one hour or two. Very strange. As you probably know, when Sydney has DST (like now), Brisbane is one hour behind Sydney (some less-than charitable Sydney residents say Brisbane is in fact fifty years plus one hour behind Sydney, but I'm not convinced this is true). During Standard Time hours, the clocks are the same in both cities. In the results below, Sydney and Canberra both subtract an hour, DST or not. Also weird. On Windows 10, R-3.3.1 (Bug in your hair), 64-bit. |
|
Thanks for running these. |
anydatetransforms date to previous day, whileanytimecorrectly transforms the dates: