Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TZ setting under Windows #96

Closed
christophsax opened this issue Jun 26, 2019 · 74 comments

Comments

@christophsax
Copy link
Contributor

commented Jun 26, 2019

Many of the Windows failures are probably due to something like the following. It seems TZ is not set correctly, unless you use useR = TRUE. Perhaps related to the code piece in r_stringToTime()?

If this were resolved, more tests could pass under Windows.

library(anytime)
Sys.setenv("TZ"="UTC")
anytime:::setTZ("UTC")
input <- "2016-09-01 10:11:12"

# this does the right thing
as.POSIXct(input)
#> [1] "2016-09-01 10:11:12 UTC"

# wrong (this uses my local time)
anytime(input)
#> [1] "2016-09-01 08:11:12 UTC"
# still wrong
anytime(input)
#> [1] "2016-09-01 08:11:12 UTC"

# correct
anytime(input, useR = TRUE)
#> [1] "2016-09-01 10:11:12 UTC"
# and now misteriously correct, too
anytime(input)
#> [1] "2016-09-01 10:11:12 UTC"

Created on 2019-06-26 by the reprex package (v0.3.0)

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jun 26, 2019

Try what I do in the unit tests: set the environment variable first, this does the equivalent of setTZ() upon package load. After that, the environment variable is no longer consulted. We use a microoptimisation to get the value from the package environment. All that works fine on the other OSs so ...

And useR=TRUE simply uses a completely different code paths.

So I guess for now the status quo remains, and I will get a bug fix release out as is with fewer tests on Windows. If you have ideas about how to improve this I would be delighted to look at PRs. Next release or thereafter...

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jun 26, 2019

Also, with the example, the devil is in the detail. I think one needs to separate TZ use for parsing, then look at the (numeric) value and then also look at the rendered character string from format().

R> pt <- utctime("1970-01-01 00:00:00")
R> as.numeric(pt)
[1] 0
R> 
R> format(pt)
[1] "1969-12-31 18:00:00"
R> format(pt, tz="UTC", usetz=TRUE)
[1] "1970-01-01 UTC"
R> 

In the above, parsing with anytime() will get a local offset, as will formatting with the local TZ.

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jun 26, 2019

Also, in your example, anytime() parses to local time as documented so maybe you wanted utctime()?

@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jun 26, 2019

But when I set the environment variable first, anytime() parses UTC? Also, that is what it does on normal systems.

Sys.setenv("TZ"="UTC")
library(anytime)
anytime:::setTZ("UTC")
input <- "2016-09-01 10:11:12"

as.POSIXct(input)
#> [1] "2016-09-01 10:11:12 UTC"
anytime(input)
#> [1] "2016-09-01 10:11:12 UTC"

Created on 2019-06-26 by the reprex package (v0.3.0)

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jun 26, 2019

Look at R/inii.R and its .onLoad(). Setting TZ later is a non-op. Not anything that should be different on Windows.

Let's step and design a test script to run, say, in Docker where know no TZ is set.

@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jun 26, 2019

I am still confused by the TZ behavior, it is doing something else on Windows than on other platforms. Am I still setting it the wrong way?

Mac

tz <-"Australia/Sydney"
Sys.setenv("TZ"=tz)

library(anytime)
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"

as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 AEST"
anytime("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 AEST"

Created on 2019-06-27 by the reprex package (v0.2.1)

Win

tz <-"Australia/Sydney"
Sys.setenv("TZ"=tz)

library(anytime)
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"

as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 AEST"
anytime("2016-09-01 10:11:12")
#> [1] "2016-09-01 20:11:12 AEST"

Created on 2019-06-27 by the reprex package (v0.3.0)

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jun 26, 2019

I confuse myself about this too. I would say that the last line is indicative of something being wrong on Windows withe the format() / print() method invoked for the POSIXct object. When I run your script on Linux, and add an additional as.numeric() around the last parse, I get

[1] "Australia/Sydney"
[1] "2016-09-01 10:11:12 AEST"
[1] "2016-09-01 10:11:12 AEST"
[1] 1472688672

just like you. There are only two small segments in src/anytime.cpp that are #ifdef'ed to Windows, and I don't think we do anything wrong.

One last thing: can you compare what unclass(as.POSIXlt(...)) on the two parses given in Windows?

And one more :) Does the difference persist for other TZ value? We did have issue tickets on some OZ timezones in the past.

@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jun 26, 2019

First Question - POSIXlt

Mac

tz <- "Australia/Sydney"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "Australia/Sydney"
library(anytime)
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"

unclass(as.POSIXlt(anytime("2016-09-01 10:11:12")))
#> $sec
#> [1] 12
#> 
#> $min
#> [1] 11
#> 
#> $hour
#> [1] 10
#> 
#> $mday
#> [1] 1
#> 
#> $mon
#> [1] 8
#> 
#> $year
#> [1] 116
#> 
#> $wday
#> [1] 4
#> 
#> $yday
#> [1] 244
#> 
#> $isdst
#> [1] 0
#> 
#> $zone
#> [1] "AEST"
#> 
#> $gmtoff
#> [1] 36000
#> 
#> attr(,"tzone")
#> [1] "Australia/Sydney" "AEST"             "AEDT"
unclass(as.POSIXlt(as.POSIXct("2016-09-01 10:11:12")))
#> $sec
#> [1] 12
#> 
#> $min
#> [1] 11
#> 
#> $hour
#> [1] 10
#> 
#> $mday
#> [1] 1
#> 
#> $mon
#> [1] 8
#> 
#> $year
#> [1] 116
#> 
#> $wday
#> [1] 4
#> 
#> $yday
#> [1] 244
#> 
#> $isdst
#> [1] 0
#> 
#> $zone
#> [1] "AEST"
#> 
#> $gmtoff
#> [1] 36000
#> 
#> attr(,"tzone")
#> [1] "Australia/Sydney" "AEST"             "AEDT"

Created on 2019-06-27 by the reprex package (v0.2.1)

Win

tz <- "Australia/Sydney"
Sys.timezone()
#> [1] "UTC"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "Australia/Sydney"
library(anytime)
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"

unclass(as.POSIXlt(anytime("2016-09-01 10:11:12")))
#> $sec
#> [1] 12
#> 
#> $min
#> [1] 11
#> 
#> $hour
#> [1] 20
#> 
#> $mday
#> [1] 1
#> 
#> $mon
#> [1] 8
#> 
#> $year
#> [1] 116
#> 
#> $wday
#> [1] 4
#> 
#> $yday
#> [1] 244
#> 
#> $isdst
#> [1] 0
#> 
#> $zone
#> [1] "AEST"
#> 
#> $gmtoff
#> [1] 36000
#> 
#> attr(,"tzone")
#> [1] "Australia/Sydney" "AEST"             "AEDT"
unclass(as.POSIXlt(as.POSIXct("2016-09-01 10:11:12")))
#> $sec
#> [1] 12
#> 
#> $min
#> [1] 11
#> 
#> $hour
#> [1] 10
#> 
#> $mday
#> [1] 1
#> 
#> $mon
#> [1] 8
#> 
#> $year
#> [1] 116
#> 
#> $wday
#> [1] 4
#> 
#> $yday
#> [1] 244
#> 
#> $isdst
#> [1] 0
#> 
#> $zone
#> [1] "AEST"
#> 
#> $gmtoff
#> [1] 36000
#> 
#> attr(,"tzone")
#> [1] "Australia/Sydney" "AEST"             "AEDT"

Created on 2019-06-27 by the reprex package (v0.3.0)

@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jun 26, 2019

Second Question - Other time zones

Mac

tz <- "America/Atikokan"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "America/Atikokan"
library(anytime)
anytime:::.pkgenv$tz
#> [1] "America/Atikokan"

anytime("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 EST"
as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 EST"

Created on 2019-06-26 by the reprex package (v0.2.1)

Win

tz <- "America/Atikokan"
Sys.timezone()
#> [1] "UTC"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "America/Atikokan"
library(anytime)
anytime:::.pkgenv$tz
#> [1] "America/Atikokan"

anytime("2016-09-01 10:11:12")
#> [1] "2016-09-01 05:11:12 EST"
as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 EST"

Created on 2019-06-26 by the reprex package (v0.3.0)

It seems that on Win it just uses the original time zone (which is UTC here).

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jun 26, 2019

Not really sure what to make of that.

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jun 26, 2019

Wait. Is that a rebuilt version with your PR to correct the TZ unset or not?

@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jun 26, 2019

Yes, including these lines:

#ifdef _WIN32
    Rcpp::Function f("Sys.setenv");
    f(Rcpp::Named("TZ") = (oldtz == nullptr ? "" : oldtz));
@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jun 26, 2019

But same with CRAN version. It depends on the initial time zone, which is now "Europe/Berlin" but was "UTC" in my earlier example.

tz <- "Australia/Sydney"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "Australia/Sydney"
library(anytime)
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"
anytime("2016-09-01 10:11:12")
#> [1] "2016-09-01 18:11:12 AEST"

Created on 2019-06-27 by the reprex package (v0.3.0)

Session info
devtools::session_info()
#> - Session info ----------------------------------------------------------
#>  setting  value                       
#>  version  R version 3.5.3 (2019-03-11)
#>  os       Windows Server x64          
#>  system   x86_64, mingw32             
#>  ui       RTerm                       
#>  language (EN)                        
#>  collate  English_United States.1252  
#>  ctype    English_United States.1252  
#>  tz       Australia/Sydney            
#>  date     2019-06-27                  
#> 
#> - Packages --------------------------------------------------------------
#>  package      * version date       lib source        
#>  anytime      * 0.3.4   2019-06-18 [1] CRAN (R 3.5.3)
#>  assertthat     0.2.1   2019-03-21 [1] CRAN (R 3.5.3)
#>  backports      1.1.4   2019-04-10 [1] CRAN (R 3.5.3)
#>  callr          3.2.0   2019-03-15 [1] CRAN (R 3.5.3)
#>  cli            1.1.0   2019-03-19 [1] CRAN (R 3.5.3)
#>  crayon         1.3.4   2017-09-16 [1] CRAN (R 3.5.3)
#>  desc           1.2.0   2018-05-01 [1] CRAN (R 3.5.3)
#>  devtools       2.0.2   2019-04-08 [1] CRAN (R 3.5.3)
#>  digest         0.6.19  2019-05-20 [1] CRAN (R 3.5.3)
#>  evaluate       0.14    2019-05-28 [1] CRAN (R 3.5.3)
#>  fs             1.3.1   2019-05-06 [1] CRAN (R 3.5.3)
#>  glue           1.3.1   2019-03-12 [1] CRAN (R 3.5.3)
#>  highr          0.8     2019-03-20 [1] CRAN (R 3.5.3)
#>  htmltools      0.3.6   2017-04-28 [1] CRAN (R 3.5.3)
#>  knitr          1.23    2019-05-18 [1] CRAN (R 3.5.3)
#>  magrittr       1.5     2014-11-22 [1] CRAN (R 3.5.3)
#>  memoise        1.1.0   2017-04-21 [1] CRAN (R 3.5.3)
#>  pkgbuild       1.0.3   2019-03-20 [1] CRAN (R 3.5.3)
#>  pkgload        1.0.2   2018-10-29 [1] CRAN (R 3.5.3)
#>  prettyunits    1.0.2   2015-07-13 [1] CRAN (R 3.5.3)
#>  processx       3.3.1   2019-05-08 [1] CRAN (R 3.5.3)
#>  ps             1.3.0   2018-12-21 [1] CRAN (R 3.5.3)
#>  R6             2.4.0   2019-02-14 [1] CRAN (R 3.5.3)
#>  RApiDatetime   0.0.4   2018-10-21 [1] CRAN (R 3.5.2)
#>  Rcpp           1.0.1   2019-03-17 [1] CRAN (R 3.5.3)
#>  remotes        2.1.0   2019-06-24 [1] CRAN (R 3.5.3)
#>  rlang          0.3.4   2019-04-07 [1] CRAN (R 3.5.3)
#>  rmarkdown      1.13    2019-05-22 [1] CRAN (R 3.5.3)
#>  rprojroot      1.3-2   2018-01-03 [1] CRAN (R 3.5.3)
#>  sessioninfo    1.1.1   2018-11-05 [1] CRAN (R 3.5.3)
#>  stringi        1.4.3   2019-03-12 [1] CRAN (R 3.5.3)
#>  stringr        1.4.0   2019-02-10 [1] CRAN (R 3.5.3)
#>  testthat       2.1.1   2019-04-23 [1] CRAN (R 3.5.3)
#>  usethis        1.5.0   2019-04-07 [1] CRAN (R 3.5.3)
#>  withr          2.1.2   2018-03-15 [1] CRAN (R 3.5.3)
#>  xfun           0.8     2019-06-25 [1] CRAN (R 3.5.3)
#>  yaml           2.2.0   2018-07-25 [1] CRAN (R 3.5.2)
#> 
#> [1] D:/Users/christoph/Documents/R/win-library/3.5
#> [2] C:/Program Files/R/R-3.5.3/library
@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jun 26, 2019

I am puzzled. I just reached out to @joshuaulrich to get another set of eyes on; he is current hamstrung by what you fixed in #97.

(A few mins later.) He can replicate. We'll get to this. Maybe ... it is just Boost. We'll see.

@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jun 27, 2019

Here's my other (perhaps related) observation copied from #98: TZ is striped by 3 characters when using useR = TRUE, at least with long TZ names. Not when TZ is set, e.g., to UTC.

Update: copied this wrongly in the initial version.

tz <- "America/Atikokan"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "America/Atikokan"
library(anytime)
anytime:::.pkgenv$tz
#> [1] "America/Atikokan"
anytime("2016-09-01 10:11:12", useR = TRUE)
#> Warning in as.POSIXlt.POSIXct(x, tz): unknown timezone 'rica/Atikokan'
#> [1] "2016-09-01 04:11:12 EST"
Sys.timezone()
#> [1] "rica/Atikokan"

Created on 2019-06-26 by the reprex package (v0.3.0)

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jun 27, 2019

That's good, and useful. At some point, though, we need to disentangle the code paths. useR=TRUE fundamentally uses a different one (from base R via the RApiDatetime package). So we have an issue on Windows with the normal parsing, and a (unrelated?) issue about the parsing. Let's get to that first.

The package has a hook anytime:::setDebug(TRUE). You can turn that on by call from R or even in the code (line 41 in src/anytime.cpp for me here). It mostly gives output related to the string parsing and issues before it gets to the Boost code. But we could add something to the Boost parse. Maybe something like in the stringToTime() function:

--- a/src/anytime.cpp
+++ b/src/anytime.cpp
@@ -225,6 +225,8 @@ double stringToTime(const std::string s, const bool asUTC=false, const bool asDa
 
     if (pt == ptbase) return NA_REAL; // NA for non-parsed dates
 
+    if (debug) Rcpp::Rcout << "Boost parsed as " << pt << std::endl;
+
     if (asUTC) {
         return ptToDoubleUTC(pt, asDate);
     } else {
edd@rob:~/git/anytime(master)$ 

yielding

edd@rob:~/git/anytime(master)$ Rscript -e 'library(anytime); anytime:::setDebug(TRUE); anytime("2019-06-27 05:57:00")'
[1] TRUE
before tests: 2019-06-27 05:57:00
In: 2019-06-27 05:57:00 out: 2019-06-27 and 05:57:00
s: 2019-06-27 len: 10 res: 0
s: 05:57:00 len: 8 res: 0
One: 2019-06-27 two: 05:57:00
before parse: 2019-06-27 05:57:00
Boost parsed as 2019-Jun-27 05:57:00
[1] "2019-06-27 05:57:00 CDT"
edd@rob:~/git/anytime(master)$ 

We need to figure out where the Australia parse goes wrong.

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jun 27, 2019

And I am now one commit (9361acf) ahead after I merged so please pull / sync before next PR, if any.

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jun 27, 2019

I just realized that one possible issue ... is that we set TZ but to a value Boost does not recognise. Then we would get 'no value' aka UTC. The bulk test tries to work on that, and that had long been turned off for Windows methinks...

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jun 27, 2019

Here is a better version of that diff:

diff --git a/src/anytime.cpp b/src/anytime.cpp
index b1dd19f..25df6a0 100644
--- a/src/anytime.cpp
+++ b/src/anytime.cpp
@@ -225,6 +225,11 @@ double stringToTime(const std::string s, const bool asUTC=false, const bool asDa

     if (pt == ptbase) return NA_REAL; // NA for non-parsed dates

+    if (debug) Rcpp::Rcout << "Boost parsed as " << pt
+                           << " which is UTC " << static_cast<int>(ptToDoubleUTC(pt))
+                           << " local " << static_cast<int>(ptToDouble(pt))
+                           << " diff " << ptToDoubleUTC(pt)-ptToDouble(pt) << std::endl;
+
     if (asUTC) {
         return ptToDoubleUTC(pt, asDate);
     } else {

With that I get

edd@rob:~/git/anytime(master)$  Rscript -e 'library(anytime); anytime:::setDebug(TRUE); anytime("2019-06-27 05:57:00")'
[1] TRUE
before tests: 2019-06-27 05:57:00
In: 2019-06-27 05:57:00 out: 2019-06-27 and 05:57:00
s: 2019-06-27 len: 10 res: 0
s: 05:57:00 len: 8 res: 0
One: 2019-06-27 two: 05:57:00
before parse: 2019-06-27 05:57:00
Boost parsed as 2019-Jun-27 05:57:00 which is UTC 1561615020 local 1561633020 diff -18000
[1] "2019-06-27 05:57:00 CDT"
edd@rob:~/git/anytime(master)$ 

That should help shed some light on where Windows goes off the rails. (And we could make the debug switch local and in the code to reduce output and avoid the one call...)

@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jun 27, 2019

Australia/Sydney example with debug info. Seems the problem is after Boost?

Mac

tz <-"Australia/Sydney"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "Australia/Sydney"
library(anytime)
anytime:::setDebug(TRUE)
#> [1] TRUE
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"
as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 AEST"
anytime("2016-09-01 10:11:12")
#> before tests: 2016-09-01 10:11:12
#> In: 2016-09-01 10:11:12 out: 2016-09-01 and 10:11:12
#> s: 2016-09-01 len: 10 res: 0
#> s: 10:11:12 len: 8 res: 0
#> One: 2016-09-01 two: 10:11:12
#> before parse: 2016-09-01 10:11:12
#> Boost parsed as 2016-Sep-01 10:11:12 which is UTC 1472724672 local 1472688672 diff 36000
#> [1] "2016-09-01 10:11:12 AEST"

Created on 2019-06-28 by the reprex package (v0.2.1)

Win

tz <-"Australia/Sydney"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "Australia/Sydney"
library(anytime)
anytime:::setDebug(TRUE)
#> [1] TRUE
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"
as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 AEST"
anytime("2016-09-01 10:11:12")
#> before tests: 2016-09-01 10:11:12
#> In: 2016-09-01 10:11:12 out: 2016-09-01 and 10:11:12
#> s: 2016-09-01 len: 10 res: 0
#> s: 10:11:12 len: 8 res: 0
#> One: 2016-09-01 two: 10:11:12
#> before parse: 2016-09-01 10:11:12
#> Boost parsed as 2016-Sep-01 10:11:12 which is UTC 1472724672 local 1472717472 diff 7200
#> [1] "2016-09-01 18:11:12 AEST"

Created on 2019-06-28 by the reprex package (v0.3.0)

@joshuaulrich

This comment has been minimized.

Copy link

commented Jun 27, 2019

@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jun 27, 2019

Mac

tz <-"Australia/Sydney"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "Australia/Sydney"
library(anytime)
anytime:::setDebug(TRUE)
#> [1] TRUE
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"
as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 AEST"
anytime("2016-09-01 10:11:12")
#> before tests: 2016-09-01 10:11:12
#> In: 2016-09-01 10:11:12 out: 2016-09-01 and 10:11:12
#> s: 2016-09-01 len: 10 res: 0
#> s: 10:11:12 len: 8 res: 0
#> One: 2016-09-01 two: 10:11:12
#> before parse: 2016-09-01 10:11:12
#> Boost parsed as 2016-Sep-01 10:11:12 which is UTC 1472724672 local tdiff is 409080:11:12
#> pt is 2016-Sep-01 10:11:12
#> 1472688672 diff tdiff is 409080:11:12
#> pt is 2016-Sep-01 10:11:12
#> 36000
#> tdiff is 409080:11:12
#> pt is 2016-Sep-01 10:11:12
#> [1] "2016-09-01 10:11:12 AEST"

Created on 2019-06-28 by the reprex package (v0.3.0)

Mac (updated)

tz <-"Australia/Sydney"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "Australia/Sydney"
library(anytime)
anytime:::setDebug(TRUE)
#> [1] TRUE
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"
as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 AEST"
anytime("2016-09-01 10:11:12")
#> before tests: 2016-09-01 10:11:12
#> In: 2016-09-01 10:11:12 out: 2016-09-01 and 10:11:12
#> s: 2016-09-01 len: 10 res: 0
#> s: 10:11:12 len: 8 res: 0
#> One: 2016-09-01 two: 10:11:12
#> before parse: 2016-09-01 10:11:12
#> Boost parsed as 2016-Sep-01 10:11:12 which is UTC 1472724672 local tdiff is 409080:11:12
#> pt is 2016-Sep-01 10:11:12
#> 1472688672 diff tdiff is 409080:11:12
#> pt is 2016-Sep-01 10:11:12
#> 36000
#> tdiff is 409080:11:12
#> pt is 2016-Sep-01 10:11:12
#> [1] "2016-09-01 10:11:12 AEST"

Created on 2019-07-01 by the reprex package (v0.3.0)

Win

tz <-"Australia/Sydney"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "Australia/Sydney"
library(anytime)
anytime:::setDebug(TRUE)
#> [1] TRUE
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"
as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 AEST"
anytime("2016-09-01 10:11:12")
#> before tests: 2016-09-01 10:11:12
#> In: 2016-09-01 10:11:12 out: 2016-09-01 and 10:11:12
#> s: 2016-09-01 len: 10 res: 0
#> s: 10:11:12 len: 8 res: 0
#> One: 2016-09-01 two: 10:11:12
#> before parse: 2016-09-01 10:11:12
#> tdiff is 409089:11:12
#> pt is 2016-Sep-01 10:11:12
#> tdiff is 409089:11:12
#> pt is 2016-Sep-01 10:11:12
#> Boost parsed as 2016-Sep-01 10:11:12 which is UTC 1472724672 local 1472717472 diff 7200
#> tdiff is 409089:11:12
#> pt is 2016-Sep-01 10:11:12
#> [1] "2016-09-01 18:11:12 AEST"

Created on 2019-06-28 by the reprex package (v0.3.0)

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jun 27, 2019

Nicer diff:

diff --git a/src/anytime.cpp b/src/anytime.cpp
index b1dd19f..57146e1 100644
--- a/src/anytime.cpp
+++ b/src/anytime.cpp
@@ -157,7 +157,7 @@ static std::string setupTZ;
 // account for localtime, and also account for dst
 double ptToDouble(const bt::ptime & pt, const bool asDate=false) {

-    //This local adjustor depends on the machine TZ settings-- highly dangerous!
+    // This local adjustor depends on the machine TZ settings-- highly dangerous!
     typedef boost::date_time::c_local_adjustor<bt::ptime> local_adj;

     const bt::ptime timet_start(boost::gregorian::date(1970,1,1));
@@ -165,8 +165,10 @@ double ptToDouble(const bt::ptime & pt, const bool asDate=false) {

     // seconds sincr epoch (in local time) -- misses DST adjustment
     bt::time_duration tdiff = pt - local_timet_start;
-    //Rcpp::Rcout << "tdiff is " << tdiff << std::endl;
-    //Rcpp::Rcout << "pt is " << pt << std::endl;
+    if (debug) {
+        Rcpp::Rcout << "tdiff in sec is " << tdiff.total_seconds()
+                    << " and pt is " << pt << std::endl;
+    }

     if (asDate) {
         if (debug) Rcpp::Rcout << "days " << pt.date().day_number() - timet_start.date().day_number() << std::endl;
@@ -225,6 +227,14 @@ double stringToTime(const std::string s, const bool asUTC=false, const bool asDa

     if (pt == ptbase) return NA_REAL; // NA for non-parsed dates

+    if (debug) {
+        int utc = static_cast<int>(ptToDoubleUTC(pt));
+        int loc = static_cast<int>(ptToDouble(pt));
+        Rcpp::Rcout << "Boost parsed as " << pt
+                    << " which is UTC " << utc
+                    << " local " << loc
+                    << " diff " << utc-loc << std::endl;
+    }
     if (asUTC) {
         return ptToDoubleUTC(pt, asDate);
     } else {

I think what we are coming up against may be local adjustment to machine time being less solid with Boost than it is with R.

@joshuaulrich

This comment has been minimized.

Copy link

commented Jun 28, 2019

@christophsax it looks like the Mac output in #96 (comment) was mangled a bit. Can you please re-run and update the Mac results?

After looking at this a bit more, I agree with Dirk. The issue looks like it's somewhere in Boost. My current guess is that the issue happens on anytime.c:223. It would be useful if the debugging output would include the timezone. I'm not sure how to force that output format in C++ though. @dirk, do you know?

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jun 28, 2019

Yes indeed.

If either one of you could try build a Boost example such as this one on local/UCT changes I would appreciate it. Should not need more than g++ with an -I..... into the BH package you already have.

@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jun 28, 2019

I am away from my laptop for 2 days. Can only re run on Sunday.

@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jul 1, 2019

@joshuaulrich what do you mean with mangled? I updated re-run the example and get the same results (updated the comment you mentioned).

@eddelbuettel

build a Boost example such as this one on local/UCT changes

Unless I got instructions designed for dummies, this is over my head.

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 1, 2019

Unless I got instructions designed for dummies, this is over my head.

It looks intimidating, but isn't. Save the file as .e.g. boostex.cpp somewhere. Then "all" it should take is

g++ -o boostex boost.cpp -IC:/R/librart/BH/include

In other words, tell g++ to create a binary named boostex via the -o name flag, use the given C++ file and point to the Boost headers you have on your machine via its path with the -I flag. I guestimated a location here, you almost certainly will have to adjust that. For BH use via g++ you have to point to BH/include, not just the package name as we do for R.

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 1, 2019

Ah shucks, never mind. It needs linking for string formatting (which we get from R in anytime) so you need a full boost.

Using my (system-level) Boost libraries

edd@rob:/tmp$ g++ -o boostex boostex.cpp -lboost_date_time
edd@rob:/tmp$ ./boostex 
UTC <--> Zone base on TZ setting
2002-Jan-01 01:00:00 in your TZ is 2002-Jan-01 07:00:00 UTC time 
A difference of: -06:00:00

UTC <--> New York while DST is NOT active (5 hours)
2001-Dec-31 19:00:00 in New York is 2002-Jan-01 00:00:00 UTC time 
2002-Jan-01 00:00:00 UTC is 2001-Dec-31 19:00:00 New York time 

UTC <--> New York while DST is active (4 hours)
2002-May-31 20:00:00 in New York is 2002-Jun-01 00:00:00 UTC time 
2002-Jun-01 00:00:00 UTC is 2002-May-31 20:00:00 New York time 

UTC <--> Arizona (7 hours)
2002-May-31 17:00:00 in Arizona is 2002-Jun-01 00:00:00 UTC time 
edd@rob:/tmp$ 

I could cook this up as an Rcpp-using example. We still need to test string conversion from Boost.

@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jul 5, 2019

It seems not Australia specific:

Win

> input <- "2019-07-05 09:37:58.541235"
> 
> Sys.setenv("TZ"="Japan")
> as.POSIXct(input)
[1] "2019-07-05 09:37:58 JST"
> 
> 
> anytime::anytime(input, useR = TRUE)
[1] "2019-07-05 17:37:58 JST"
> anytime::anytime(input)
[1] "2019-07-05 17:37:58 JST"

Mac

> input <- "2019-07-05 09:37:58.541235"
> 
> Sys.setenv("TZ"="Japan")
> as.POSIXct(input)
[1] "2019-07-05 09:37:58 JST"
> 
> 
> anytime::anytime(input, useR = TRUE)
[1] "2019-07-05 09:37:58 JST"
> anytime::anytime(input)
[1] "2019-07-05 09:37:58 JST"
@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 5, 2019

Running the script

input <- "2019-07-05 09:37:58.541235"

Sys.setenv("TZ"="Japan")

library(anytime)
anytime(input, useR = TRUE)
anytime(input)

I get what you get for the mac, ie

$ Rscript tmp/issue96.R
[1] "2019-07-05 09:37:58.541234 JST"
[1] "2019-07-05 09:37:58.541234 JST"
$

Otherwise I am out of steam. Someone who a) has a windows box and b) cares enough needs to c) dive into the code.

I can't run on Windows. Maybe @joshuaulrich or someone else can corrobate what happens there. Can we exclude any influence from your VM?

@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jul 5, 2019

For now, it seems sufficient to say that TZ setting does not work on Windows, as you did above. Just don't offer useR = TRUE as a solution.

Since I am using anytime with useR = TRUE in tsbox, I am interested in a solution. I can look into it more carefully when I have more time.

I use a virtual desktop on https://aws.amazon.com/workspaces/ which generally works well for my debugging needs. I also have a physical Windows Laptop but without RTools, so I cannot check there.

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 5, 2019

Something is clearly off. There is close to no code difference. There is one block in src/anytime.cpp that is #ifdef'ed for WIN32. You could add a printf or two there for local debugging. Otherwise I am stumped.

@joshuaulrich

This comment has been minimized.

Copy link

commented Jul 5, 2019

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 5, 2019

Thanks

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 5, 2019

I love working with timezones as it is the surest way to loose my sanity quickly and reliably.

We were conflating parsing, and formatting. I am now suspecting that there is another error in formatting. But the following now works for me on Linux and via rhub:

input <- "2019-07-05 09:37:58"

Sys.setenv("TZ"="Japan")

library(anytime)

n1 <- as.numeric(anytime(input, useR = TRUE))
n2 <- as.numeric(anytime(input))
n3 <- as.numeric(as.POSIXct("2019-07-05 09:37:58"))

expect_equal(n1, n2, tol=1.0e-4)
expect_equal(n1, n3, tol=1.0e-4)
expect_equal(n2, n3, tol=1.0e-4)

Of course, I started with format() wrapped around, and there things went astray. So there may still be work remaining on whether the tzone attribute is present, or not -- but the parsing seems to work.

@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jul 5, 2019

It is confusing. On my AWS machine, it won't pass:

> input <- "2019-07-05 09:37:58"
> 
> Sys.setenv("TZ"="Japan")
> 
> library(anytime)
> 
> n1 <- as.numeric(anytime(input, useR = TRUE))
> n2 <- as.numeric(anytime(input))
> n3 <- as.numeric(as.POSIXct("2019-07-05 09:37:58"))
> 
> library(tinytest)
> expect_equal(n1, n2, tol=1.0e-4)
----- PASSED      : <-->
 call| expect_equal(n1, n2, tol = 1e-04) 
> expect_equal(n1, n3, tol=1.0e-4)
----- FAILED[attr]: <-->
 call| expect_equal(n1, n3, tol = 1e-04)
 diff| Mean relative difference: 1.843417e-05 
> expect_equal(n2, n3, tol=1.0e-4)
----- FAILED[attr]: <-->
 call| expect_equal(n2, n3, tol = 1e-04)
 diff| Mean relative difference: 1.843417e-05 

But in the CET time zone, I have to get out now, loose my sanity otherwise as well.

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 5, 2019

I needed a few iterations with the tol= argument as well, and there was a tinytest version that didn't propagate it so make sure you are (reasonably) current). We could also convert to int or do it for date if need be...

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 5, 2019

@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jul 6, 2019

I don't think a problem with tol. 28800 are 8 hours, just the kind of differences we should expect with TZ issues. Otherwise, no clue.

> input <- "2019-07-05 09:37:58"
> 
> Sys.setenv("TZ"="Japan")
> 
> library(anytime)
> 
> n1 <- as.numeric(anytime(input, useR = TRUE))
> n2 <- as.numeric(anytime(input))
> n3 <- as.numeric(as.POSIXct("2019-07-05 09:37:58"))
> 
> n1 - n2
[1] 0
> n1 - n3
[1] 28800
> n2 - n3
[1] 28800
@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 6, 2019

Yes, I also realized later that the numbers being large enough, changed the tolerance washes the result away.

I do not know why n1 and n2 are different from n3 on Windows.

One thing I realized, though, is that anytime does one thing differently and says so. I just don't understand why it is different on Windows. From the help page about the tz argument:

  tz: A string with the timezone, defaults to the result of the
      (internal) ‘getTZ’ function if unset. The ‘getTZ’ function
      returns the timezone values stored in local package
      environment, and set at package load time. Also note that
      this argument applies to the _output_: the returned object
      will have this timezone set. The timezone is _not_ used for
      the parsing which will always be to localtime, or to UTC is
      the ‘asUTC’ variable is set (as it is in the related
      functions ‘utctime’ and ‘utcdate’). So one can think of the
      argument as ‘shift parsed time object to this timezone’. This
      is similar to what ‘format()’ in base R does, but our return
      value is still a ‘POSIXt’ object instead of a character
      value.

Still. I have no idea why Windows differs.

When you set no TZ, the results remain the same though, correct?

@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jul 6, 2019

Here is without setting it, staying with CET:

> input <- "2019-07-05 09:37:58"
> 
> library(anytime)
> 
> n1 <- as.numeric(anytime(input, useR = TRUE))
> n2 <- as.numeric(anytime(input))
> n3 <- as.numeric(as.POSIXct("2019-07-05 09:37:58"))
> 
> n1 - n2
[1] 0
> n1 - n3
[1] 3600
> n2 - n3
[1] 3600
@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 6, 2019

So even the DST field makes it differ. Weird.

There is nothing I can do from here. POSIXct and POSIXlt can be complicated. You can try to poke around interactively, maybe do a unclass(as.POSIXlt())) to see if you can come up with something. Otherwise I can only repeat that in src/anytime.cpp the only change is

#if defined(_WIN32)
    if (totsec > 0  &&                  // on Windows before 1970-01-01: segfault
       localAsTm->tm_year < 1100) {     // tm_year is year-1900, so year 3000 is it
        dstadj = localAsTm->tm_isdst*60*60;
    }
#else
    dstadj = localAsTm->tm_isdst*60*60;
#endif

but as we see it sets the DST adjustment for all OSs so I have no idea anymore. I could just turn Windows builds at CRAN off to have some peace and quiet.

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 6, 2019

I had one more idea about Sys.timezone() but that seems read-only as well. 😞

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 7, 2019

I had one more thought while on the airplane: maybe the way we return the Datetime object by relying on Rcpp slips up on Windows.

We could backtrack and see if the parsed text, when represented "still inside anytime.cpp as a double" has the correct value on Windows. Can you insert a printf? I can probably post a diff here too.

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 7, 2019

I added the line

``return totsec - dstadj;`c++
Rcpp::Rcout << "totsec secs is " << std::setprecision(16) << totsec << " and dstadj is " << dstadj << std::endl;


right before the `return totsec - dstadj;`.  Then with minor mods the example:

```r
input <- "2019-07-05 09:37:58.541235"

Sys.setenv("TZ"="Japan")
#Sys.setenv("TZ"="Europe/Berlin")
library(anytime)
anytime(input, useR = TRUE)
anytime(input)
as.POSIXct(input)

n1 <- as.numeric(anytime(input, useR = TRUE))
n2 <- as.numeric(anytime(input))
n3 <- as.numeric(as.POSIXct(input))
print(n3,digits=16)

cat("n1 - n2: ", n1 - n2, "\n")
cat("n1 - n3: ", n1 - n3, "\n")
cat("n2 - n3: ", n2 - n3, "\n")

I get

edd@rob:~/git/anytime(master)$ Rscript local/gh_issue_96.R 
[1] "2019-07-05 09:37:58.541234 CEST"
totsec secs is 1562315878.541235 and dstadj is 3600
[1] "2019-07-05 09:37:58.541234 CEST"
[1] "2019-07-05 09:37:58.541234 CEST"
totsec secs is 1562315878.541235 and dstadj is 3600
[1] 1562312278.541235
n1 - n2:  0
n1 - n3:  0
n2 - n3:  0
edd@rob:~/git/anytime(master)$

showing the agreement for Japan. For Berlin we see a difference as there is a DST:

[1] "2019-07-05 09:37:58.541234 CEST"
totsec secs is 1562315878.541235 and dstadj is 3600
[1] "2019-07-05 09:37:58.541234 CEST"
[1] "2019-07-05 09:37:58.541234 CEST"
totsec secs is 1562315878.541235 and dstadj is 3600
[1] 1562312278.541235
n1 - n2:  0 
n1 - n3:  0 
n2 - n3:  0 
@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jul 7, 2019

Same here (Windows):

Japan

> input <- "2019-07-05 09:37:58"
> Sys.setenv("TZ"="Japan")
> library(anytime)
> 
> as.numeric(anytime(input, useR = TRUE))
[1] 1562315878
> as.numeric(anytime(input))
totsec secs is 1562319478 and dstadj is 3600
[1] 1562315878

CEST

> input <- "2019-07-05 09:37:58"
> Sys.setenv("TZ"="CEST")
> library(anytime)
> 
> as.numeric(anytime(input, useR = TRUE))
[1] 1562315878
> as.numeric(anytime(input))
totsec secs is 1562319478 and dstadj is 3600
[1] 1562315878
@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 7, 2019

That is the first bit of encouraging news, I think. I think we can trace from here. Maybe it merely is a bug in Datetime object creation by Rcpp on Windows. To be continued ....

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 8, 2019

That wasn't the full test on your side though. Can you rebuild anytime with the last two lines of function ptToDouble() in src/anytime.cpp being

    Rcpp::Rcout << "totsec secs is " << std::setprecision(16) << totsec << " and dstadj is " << dstadj << std::endl;
    return totsec - dstadj;

and then run this script for timezones with/without dst:

input <- "2019-07-05 09:37:58.541235"

#Sys.setenv("TZ"="Japan")
Sys.setenv("TZ"="Europe/Berlin")
library(anytime)
anytime(input, useR = TRUE)
anytime(input)
as.POSIXct(input)

n1 <- as.numeric(anytime(input, useR = TRUE))
n2 <- as.numeric(anytime(input))
n3 <- as.numeric(as.POSIXct(input))
print(n3,digits=16)

cat("n1 - n2: ", n1 - n2, "\n")
cat("n1 - n3: ", n1 - n3, "\n")
cat("n2 - n3: ", n2 - n3, "\n")
@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 8, 2019

And I also committed the change and helper script so if you pull from the repo you should get it.

@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jul 8, 2019

Berlin

> input <- "2019-07-05 09:37:58.541235"
> 
> #Sys.setenv("TZ"="Japan")
> Sys.setenv("TZ"="Europe/Berlin")
> library(anytime)
> anytime:::setDebug(TRUE)
[1] TRUE
> 
> anytime(input, useR = TRUE)
before tests: 2019-07-05 09:37:58.541235
In: 2019-07-05 09:37:58.541235 out: 2019-07-05 and 09:37:58.541235
s: 2019-07-05 len: 10 res: 0
s: 09:37:58.541235 len: 15 res: 0
One: 2019-07-05 two: 09:37:58.541235
before parse: 2019-07-05 09:37:58.541235
[1] "2019-07-05 10:37:58 CEST"
> anytime(input)
before tests: 2019-07-05 09:37:58.541235
In: 2019-07-05 09:37:58.541235 out: 2019-07-05 and 09:37:58.541235
s: 2019-07-05 len: 10 res: 0
s: 09:37:58.541235 len: 15 res: 0
One: 2019-07-05 two: 09:37:58.541235
before parse: 2019-07-05 09:37:58.541235
tdiff in sec is 1562319478 and pt is 2019-Jul-05 09:37:58.541235
totsec secs is 1562319478.541235 and dstadj is 3600
Boost parsed as 2019-Jul-05 09:37:58.541235 which is UTC 1562319478 local 1562315878 diff 3600
tdiff in sec is 1562319478 and pt is 2019-Jul-05 09:37:58.541235
totsec secs is 1562319478.541235 and dstadj is 3600
[1] "2019-07-05 10:37:58 CEST"

Japan

> input <- "2019-07-05 09:37:58.541235"
> 
> Sys.setenv("TZ"="Japan")
> # Sys.setenv("TZ"="Europe/Berlin")
> library(anytime)
> anytime:::setDebug(TRUE)
[1] TRUE
> 
> anytime(input, useR = TRUE)
before tests: 2019-07-05 09:37:58.541235
In: 2019-07-05 09:37:58.541235 out: 2019-07-05 and 09:37:58.541235
s: 2019-07-05 len: 10 res: 0
s: 09:37:58.541235 len: 15 res: 0
One: 2019-07-05 two: 09:37:58.541235
before parse: 2019-07-05 09:37:58.541235
[1] "2019-07-05 17:37:58 JST"
> anytime(input)
before tests: 2019-07-05 09:37:58.541235
In: 2019-07-05 09:37:58.541235 out: 2019-07-05 and 09:37:58.541235
s: 2019-07-05 len: 10 res: 0
s: 09:37:58.541235 len: 15 res: 0
One: 2019-07-05 two: 09:37:58.541235
before parse: 2019-07-05 09:37:58.541235
tdiff in sec is 1562319478 and pt is 2019-Jul-05 09:37:58.541235
totsec secs is 1562319478.541235 and dstadj is 3600
Boost parsed as 2019-Jul-05 09:37:58.541235 which is UTC 1562319478 local 1562315878 diff 3600
tdiff in sec is 1562319478 and pt is 2019-Jul-05 09:37:58.541235
totsec secs is 1562319478.541235 and dstadj is 3600
[1] "2019-07-05 17:37:58 JST"
@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 14, 2019

So I realized belatedly that I could test via rhub by putting code in a script below tests/ and could recover the output -- no more relying on poor @christophsax . That said, and despite maybe a dozen uploads trying some things, I am not making headway. Windows appears broken.

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 21, 2019

@christophsax @joshuaulrich : I completely rewrote r_stringToTime so if you set useR=TRUE it should behave just like R itself ... as it really farms out to R (directly via Rcpp::Function rather than via the RApiDatetime package.

This is currently in a branch feature/no_rapidatetime I made yesterday (but only pushed just now, my bad). There are rhub artifacts if you want a zipfile built for Windows.

Please take a look, I think this may be a better version. In short, Windows (default) mode remains complicated because how Boost does this, and we may now have a better fallback for Windows. Let me know what you think. I am even thinking we could make useR=TRUE the default on that platform.

Edit: A fresher set of build artifacts with the updated documentation etc pp:

anytime 0.3.4.5: OK

Build ID: anytime_0.3.4.5.tar.gz-489c96ab840f412fa5134d674708fe26
Platform: Windows Server 2008 R2 SP1, R-release, 32/64 bit
Submitted: 3 minutes 18.3 seconds ago
Build time: 3 minutes 16.7 seconds

See the full build log:
HTML: https://builder.r-hub.io/status/anytime_0.3.4.5.tar.gz-489c96ab840f412fa5134d674708fe26
Text: https://builder.r-hub.io/status/original/anytime_0.3.4.5.tar.gz-489c96ab840f412fa5134d674708fe26
Artifacts: https://artifacts.r-hub.io/anytime_0.3.4.5.tar.gz-489c96ab840f412fa5134d674708fe26

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 22, 2019

And more with the comparison above across four times zone. No difference between "R code" and anytime with useR=TRUE as the latter now is using the former via Rcpp::Function. Details at
https://artifacts.r-hub.io/anytime_0.3.4.5.tar.gz-bc836ddefd714b44acded71ece8c0349 in file anytime.Rcheck/tests_x64/quickDebug2.Rout which has:

> #Sys.setenv("TZ"="Europe/Berlin")
> library(anytime)
> 
> #input <- "2019-07-05 09:37:58.541235"
> input <- format( utctime( 1.5e9 ) )
> print(input)
[1] "2017-07-14 02:40:00"
> options(scipen=999)
> 
> for (tz in c("UTC", "Europe/Berlin", "Europe/Athens", "Japan", "America/Chicago")) {
+     cat("Setting tz to", tz, "\n")
+     Sys.setenv("TZ"=tz)
+     anytime:::setTZ(tz)
+     Sys.sleep(0.25)
+ 
+     #print(anytime(input, tz=tz))
+     #print(anytime(input, useR = TRUE, tz=tz))
+     #print(format(as.POSIXct(input), tz=tz))
+ 
+     n0 <- as.numeric(as.POSIXct(input))
+     n1 <- as.numeric(anytime(input, tz=tz))
+     n2 <- as.numeric(anytime(input, useR = TRUE, tz=tz))
+     print(c(n0,n1,n2),digits=16)
+ 
+     #cat("n0 - n1: ", n0 - n1, "\n")
+     #cat("n0 - n2: ", n0 - n2, "\n")
+     #cat("n1 - n2: ", n1 - n2, "\n")
+     print(c(n0-n1, n0-n2, n1-n2))
+ }
Setting tz to UTC 
[1] 1500000000 1500000000 1500000000
[1] 0 0 0
Setting tz to Europe/Berlin 
[1] 1499992800 1500000000 1499992800
[1] -7200     0  7200
Setting tz to Europe/Athens 
[1] 1499989200 1500000000 1499989200
[1] -10800      0  10800
Setting tz to Japan 
[1] 1499967600 1500000000 1499967600
[1] -32400      0  32400
Setting tz to America/Chicago 
[1] 1500018000 1500000000 1500018000
[1]  18000      0 -18000

This demonstrates the difficulty of setting TZ on Windows, but that the now-added workaround is effective.

Do you concur?

@christophsax

This comment has been minimized.

Copy link
Contributor Author

commented Jul 23, 2019

That looks fantastic!

My system's time zone is set to Berlin, so useR=TRUE and useR=FALSE are identical for that zone. Contrary to your results, so it seems rhub is at UTC.

Otherwise, no differences between useR=TRUE and what R does, nice!

I have no opinion on what the default of useR should be on Windows. I am using useR=TRUE in tsbox, because I had another issue with Boost, and it is probably the safer option. But then I also don't care about performance.

library(anytime)
> 
> input <- format( utctime( 1.5e9 ) )
> options(scipen=999)
> 
> for (tz in c("UTC", "Europe/Berlin", "Europe/Athens", "Japan", "America/Chicago")) {
+     cat("Setting tz to", tz, "\n")
+     Sys.setenv("TZ"=tz)
+     anytime:::setTZ(tz)
+     Sys.sleep(0.25)
+     
+     #print(anytime(input, tz=tz))
+     #print(anytime(input, useR = TRUE, tz=tz))
+     #print(format(as.POSIXct(input), tz=tz))
+     
+     n0 <- as.numeric(as.POSIXct(input))
+     n1 <- as.numeric(anytime(input, tz=tz))
+     n2 <- as.numeric(anytime(input, useR = TRUE, tz=tz))
+     print(c(n0,n1,n2),digits=16)
+     
+     #cat("n0 - n1: ", n0 - n1, "\n")
+     #cat("n0 - n2: ", n0 - n2, "\n")
+     #cat("n1 - n2: ", n1 - n2, "\n")
+     print(c(n0-n1, n0-n2, n1-n2))
+ }
Setting tz to UTC 
[1] 1500000000 1499992800 1500000000
[1]  7200     0 -7200
Setting tz to Europe/Berlin 
[1] 1499992800 1499992800 1499992800
[1] 0 0 0
Setting tz to Europe/Athens 
[1] 1499989200 1499992800 1499989200
[1] -3600     0  3600
Setting tz to Japan 
[1] 1499967600 1499992800 1499967600
[1] -25200      0  25200
Setting tz to America/Chicago 
[1] 1500018000 1499992800 1500018000
[1]  25200      0 -25200
@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 23, 2019

Yay! This issue was really bugging me. Looks like we have it squashed. I'll make the PR and merge, and maybe work a little on documentation.

I am still thinking about what to do with the useR=... default value. Maybe I just allow an option people can set to default to TRUE to keep the interface the same.

@eddelbuettel

This comment has been minimized.

Copy link
Owner

commented Jul 23, 2019

Also, the PR, once I merge it, will close this bug report. We may open a new wishlist one to see if any of the currently "paused" tests can be relaxed on Windows when setting useR=TRUE. But that is not urgent I feel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.