Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upTZ setting under Windows #96
Comments
|
Try what I do in the unit tests: set the environment variable first, this does the equivalent of And So I guess for now the status quo remains, and I will get a bug fix release out as is with fewer tests on Windows. If you have ideas about how to improve this I would be delighted to look at PRs. Next release or thereafter... |
|
Also, with the example, the devil is in the detail. I think one needs to separate TZ use for parsing, then look at the (numeric) value and then also look at the rendered character string from In the above, parsing with |
|
Also, in your example, |
|
But when I set the environment variable first, Sys.setenv("TZ"="UTC")
library(anytime)
anytime:::setTZ("UTC")
input <- "2016-09-01 10:11:12"
as.POSIXct(input)
#> [1] "2016-09-01 10:11:12 UTC"
anytime(input)
#> [1] "2016-09-01 10:11:12 UTC"Created on 2019-06-26 by the reprex package (v0.3.0) |
|
Look at Let's step and design a test script to run, say, in Docker where know no TZ is set. |
|
I am still confused by the TZ behavior, it is doing something else on Windows than on other platforms. Am I still setting it the wrong way? Mactz <-"Australia/Sydney"
Sys.setenv("TZ"=tz)
library(anytime)
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"
as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 AEST"
anytime("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 AEST"Created on 2019-06-27 by the reprex package (v0.2.1) Wintz <-"Australia/Sydney"
Sys.setenv("TZ"=tz)
library(anytime)
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"
as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 AEST"
anytime("2016-09-01 10:11:12")
#> [1] "2016-09-01 20:11:12 AEST"Created on 2019-06-27 by the reprex package (v0.3.0) |
|
I confuse myself about this too. I would say that the last line is indicative of something being wrong on Windows withe the [1] "Australia/Sydney"
[1] "2016-09-01 10:11:12 AEST"
[1] "2016-09-01 10:11:12 AEST"
[1] 1472688672just like you. There are only two small segments in One last thing: can you compare what And one more :) Does the difference persist for other TZ value? We did have issue tickets on some OZ timezones in the past. |
First Question - POSIXltMactz <- "Australia/Sydney"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "Australia/Sydney"
library(anytime)
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"
unclass(as.POSIXlt(anytime("2016-09-01 10:11:12")))
#> $sec
#> [1] 12
#>
#> $min
#> [1] 11
#>
#> $hour
#> [1] 10
#>
#> $mday
#> [1] 1
#>
#> $mon
#> [1] 8
#>
#> $year
#> [1] 116
#>
#> $wday
#> [1] 4
#>
#> $yday
#> [1] 244
#>
#> $isdst
#> [1] 0
#>
#> $zone
#> [1] "AEST"
#>
#> $gmtoff
#> [1] 36000
#>
#> attr(,"tzone")
#> [1] "Australia/Sydney" "AEST" "AEDT"
unclass(as.POSIXlt(as.POSIXct("2016-09-01 10:11:12")))
#> $sec
#> [1] 12
#>
#> $min
#> [1] 11
#>
#> $hour
#> [1] 10
#>
#> $mday
#> [1] 1
#>
#> $mon
#> [1] 8
#>
#> $year
#> [1] 116
#>
#> $wday
#> [1] 4
#>
#> $yday
#> [1] 244
#>
#> $isdst
#> [1] 0
#>
#> $zone
#> [1] "AEST"
#>
#> $gmtoff
#> [1] 36000
#>
#> attr(,"tzone")
#> [1] "Australia/Sydney" "AEST" "AEDT"Created on 2019-06-27 by the reprex package (v0.2.1) Wintz <- "Australia/Sydney"
Sys.timezone()
#> [1] "UTC"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "Australia/Sydney"
library(anytime)
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"
unclass(as.POSIXlt(anytime("2016-09-01 10:11:12")))
#> $sec
#> [1] 12
#>
#> $min
#> [1] 11
#>
#> $hour
#> [1] 20
#>
#> $mday
#> [1] 1
#>
#> $mon
#> [1] 8
#>
#> $year
#> [1] 116
#>
#> $wday
#> [1] 4
#>
#> $yday
#> [1] 244
#>
#> $isdst
#> [1] 0
#>
#> $zone
#> [1] "AEST"
#>
#> $gmtoff
#> [1] 36000
#>
#> attr(,"tzone")
#> [1] "Australia/Sydney" "AEST" "AEDT"
unclass(as.POSIXlt(as.POSIXct("2016-09-01 10:11:12")))
#> $sec
#> [1] 12
#>
#> $min
#> [1] 11
#>
#> $hour
#> [1] 10
#>
#> $mday
#> [1] 1
#>
#> $mon
#> [1] 8
#>
#> $year
#> [1] 116
#>
#> $wday
#> [1] 4
#>
#> $yday
#> [1] 244
#>
#> $isdst
#> [1] 0
#>
#> $zone
#> [1] "AEST"
#>
#> $gmtoff
#> [1] 36000
#>
#> attr(,"tzone")
#> [1] "Australia/Sydney" "AEST" "AEDT"Created on 2019-06-27 by the reprex package (v0.3.0) |
Second Question - Other time zonesMactz <- "America/Atikokan"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "America/Atikokan"
library(anytime)
anytime:::.pkgenv$tz
#> [1] "America/Atikokan"
anytime("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 EST"
as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 EST"Created on 2019-06-26 by the reprex package (v0.2.1) Wintz <- "America/Atikokan"
Sys.timezone()
#> [1] "UTC"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "America/Atikokan"
library(anytime)
anytime:::.pkgenv$tz
#> [1] "America/Atikokan"
anytime("2016-09-01 10:11:12")
#> [1] "2016-09-01 05:11:12 EST"
as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 EST"Created on 2019-06-26 by the reprex package (v0.3.0) It seems that on Win it just uses the original time zone (which is UTC here). |
|
Not really sure what to make of that. |
|
Wait. Is that a rebuilt version with your PR to correct the TZ unset or not? |
|
Yes, including these lines: |
|
But same with CRAN version. It depends on the initial time zone, which is now tz <- "Australia/Sydney"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "Australia/Sydney"
library(anytime)
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"
anytime("2016-09-01 10:11:12")
#> [1] "2016-09-01 18:11:12 AEST"Created on 2019-06-27 by the reprex package (v0.3.0) Session infodevtools::session_info()
#> - Session info ----------------------------------------------------------
#> setting value
#> version R version 3.5.3 (2019-03-11)
#> os Windows Server x64
#> system x86_64, mingw32
#> ui RTerm
#> language (EN)
#> collate English_United States.1252
#> ctype English_United States.1252
#> tz Australia/Sydney
#> date 2019-06-27
#>
#> - Packages --------------------------------------------------------------
#> package * version date lib source
#> anytime * 0.3.4 2019-06-18 [1] CRAN (R 3.5.3)
#> assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.5.3)
#> backports 1.1.4 2019-04-10 [1] CRAN (R 3.5.3)
#> callr 3.2.0 2019-03-15 [1] CRAN (R 3.5.3)
#> cli 1.1.0 2019-03-19 [1] CRAN (R 3.5.3)
#> crayon 1.3.4 2017-09-16 [1] CRAN (R 3.5.3)
#> desc 1.2.0 2018-05-01 [1] CRAN (R 3.5.3)
#> devtools 2.0.2 2019-04-08 [1] CRAN (R 3.5.3)
#> digest 0.6.19 2019-05-20 [1] CRAN (R 3.5.3)
#> evaluate 0.14 2019-05-28 [1] CRAN (R 3.5.3)
#> fs 1.3.1 2019-05-06 [1] CRAN (R 3.5.3)
#> glue 1.3.1 2019-03-12 [1] CRAN (R 3.5.3)
#> highr 0.8 2019-03-20 [1] CRAN (R 3.5.3)
#> htmltools 0.3.6 2017-04-28 [1] CRAN (R 3.5.3)
#> knitr 1.23 2019-05-18 [1] CRAN (R 3.5.3)
#> magrittr 1.5 2014-11-22 [1] CRAN (R 3.5.3)
#> memoise 1.1.0 2017-04-21 [1] CRAN (R 3.5.3)
#> pkgbuild 1.0.3 2019-03-20 [1] CRAN (R 3.5.3)
#> pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.5.3)
#> prettyunits 1.0.2 2015-07-13 [1] CRAN (R 3.5.3)
#> processx 3.3.1 2019-05-08 [1] CRAN (R 3.5.3)
#> ps 1.3.0 2018-12-21 [1] CRAN (R 3.5.3)
#> R6 2.4.0 2019-02-14 [1] CRAN (R 3.5.3)
#> RApiDatetime 0.0.4 2018-10-21 [1] CRAN (R 3.5.2)
#> Rcpp 1.0.1 2019-03-17 [1] CRAN (R 3.5.3)
#> remotes 2.1.0 2019-06-24 [1] CRAN (R 3.5.3)
#> rlang 0.3.4 2019-04-07 [1] CRAN (R 3.5.3)
#> rmarkdown 1.13 2019-05-22 [1] CRAN (R 3.5.3)
#> rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.5.3)
#> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.5.3)
#> stringi 1.4.3 2019-03-12 [1] CRAN (R 3.5.3)
#> stringr 1.4.0 2019-02-10 [1] CRAN (R 3.5.3)
#> testthat 2.1.1 2019-04-23 [1] CRAN (R 3.5.3)
#> usethis 1.5.0 2019-04-07 [1] CRAN (R 3.5.3)
#> withr 2.1.2 2018-03-15 [1] CRAN (R 3.5.3)
#> xfun 0.8 2019-06-25 [1] CRAN (R 3.5.3)
#> yaml 2.2.0 2018-07-25 [1] CRAN (R 3.5.2)
#>
#> [1] D:/Users/christoph/Documents/R/win-library/3.5
#> [2] C:/Program Files/R/R-3.5.3/library |
|
I am puzzled. I just reached out to @joshuaulrich to get another set of eyes on; he is current hamstrung by what you fixed in #97. (A few mins later.) He can replicate. We'll get to this. Maybe ... it is just Boost. We'll see. |
|
Here's my other (perhaps related) observation copied from #98: TZ is striped by 3 characters when using Update: copied this wrongly in the initial version. tz <- "America/Atikokan"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "America/Atikokan"
library(anytime)
anytime:::.pkgenv$tz
#> [1] "America/Atikokan"
anytime("2016-09-01 10:11:12", useR = TRUE)
#> Warning in as.POSIXlt.POSIXct(x, tz): unknown timezone 'rica/Atikokan'
#> [1] "2016-09-01 04:11:12 EST"
Sys.timezone()
#> [1] "rica/Atikokan"Created on 2019-06-26 by the reprex package (v0.3.0) |
|
That's good, and useful. At some point, though, we need to disentangle the code paths. The package has a hook --- a/src/anytime.cpp
+++ b/src/anytime.cpp
@@ -225,6 +225,8 @@ double stringToTime(const std::string s, const bool asUTC=false, const bool asDa
if (pt == ptbase) return NA_REAL; // NA for non-parsed dates
+ if (debug) Rcpp::Rcout << "Boost parsed as " << pt << std::endl;
+
if (asUTC) {
return ptToDoubleUTC(pt, asDate);
} else {
edd@rob:~/git/anytime(master)$ yielding edd@rob:~/git/anytime(master)$ Rscript -e 'library(anytime); anytime:::setDebug(TRUE); anytime("2019-06-27 05:57:00")'
[1] TRUE
before tests: 2019-06-27 05:57:00
In: 2019-06-27 05:57:00 out: 2019-06-27 and 05:57:00
s: 2019-06-27 len: 10 res: 0
s: 05:57:00 len: 8 res: 0
One: 2019-06-27 two: 05:57:00
before parse: 2019-06-27 05:57:00
Boost parsed as 2019-Jun-27 05:57:00
[1] "2019-06-27 05:57:00 CDT"
edd@rob:~/git/anytime(master)$ We need to figure out where the Australia parse goes wrong. |
|
And I am now one commit (9361acf) ahead after I merged so please pull / sync before next PR, if any. |
|
I just realized that one possible issue ... is that we set TZ but to a value Boost does not recognise. Then we would get 'no value' aka UTC. The bulk test tries to work on that, and that had long been turned off for Windows methinks... |
|
Here is a better version of that diff: diff --git a/src/anytime.cpp b/src/anytime.cpp
index b1dd19f..25df6a0 100644
--- a/src/anytime.cpp
+++ b/src/anytime.cpp
@@ -225,6 +225,11 @@ double stringToTime(const std::string s, const bool asUTC=false, const bool asDa
if (pt == ptbase) return NA_REAL; // NA for non-parsed dates
+ if (debug) Rcpp::Rcout << "Boost parsed as " << pt
+ << " which is UTC " << static_cast<int>(ptToDoubleUTC(pt))
+ << " local " << static_cast<int>(ptToDouble(pt))
+ << " diff " << ptToDoubleUTC(pt)-ptToDouble(pt) << std::endl;
+
if (asUTC) {
return ptToDoubleUTC(pt, asDate);
} else {With that I get That should help shed some light on where Windows goes off the rails. (And we could make the debug switch local and in the code to reduce output and avoid the one call...) |
|
Australia/Sydney example with debug info. Seems the problem is after Boost? Mactz <-"Australia/Sydney"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "Australia/Sydney"
library(anytime)
anytime:::setDebug(TRUE)
#> [1] TRUE
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"
as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 AEST"
anytime("2016-09-01 10:11:12")
#> before tests: 2016-09-01 10:11:12
#> In: 2016-09-01 10:11:12 out: 2016-09-01 and 10:11:12
#> s: 2016-09-01 len: 10 res: 0
#> s: 10:11:12 len: 8 res: 0
#> One: 2016-09-01 two: 10:11:12
#> before parse: 2016-09-01 10:11:12
#> Boost parsed as 2016-Sep-01 10:11:12 which is UTC 1472724672 local 1472688672 diff 36000
#> [1] "2016-09-01 10:11:12 AEST"Created on 2019-06-28 by the reprex package (v0.2.1) Wintz <-"Australia/Sydney"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "Australia/Sydney"
library(anytime)
anytime:::setDebug(TRUE)
#> [1] TRUE
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"
as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 AEST"
anytime("2016-09-01 10:11:12")
#> before tests: 2016-09-01 10:11:12
#> In: 2016-09-01 10:11:12 out: 2016-09-01 and 10:11:12
#> s: 2016-09-01 len: 10 res: 0
#> s: 10:11:12 len: 8 res: 0
#> One: 2016-09-01 two: 10:11:12
#> before parse: 2016-09-01 10:11:12
#> Boost parsed as 2016-Sep-01 10:11:12 which is UTC 1472724672 local 1472717472 diff 7200
#> [1] "2016-09-01 18:11:12 AEST"Created on 2019-06-28 by the reprex package (v0.3.0) |
|
ptToDouble() calls boost::date_time::c_local_adjustor(), which uses the
"machine tz settings" via call to localtime(). It would be helpful to see
the output the two commented lines would produce. I don't have time to test
right now though.
…On Thu, Jun 27, 2019, 12:26 PM Christoph Sax ***@***.***> wrote:
Australia/Sydney example with debug info. Seems the problem is after Boost?
Mac
tz <-"Australia/Sydney"
Sys.timezone()#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()#> [1] "Australia/Sydney"
library(anytime)anytime:::setDebug(TRUE)#> [1] TRUEanytime:::.pkgenv$tz#> [1] "Australia/Sydney"
as.POSIXct("2016-09-01 10:11:12")#> [1] "2016-09-01 10:11:12 AEST"
anytime("2016-09-01 10:11:12")#> before tests: 2016-09-01 10:11:12#> In: 2016-09-01 10:11:12 out: 2016-09-01 and 10:11:12#> s: 2016-09-01 len: 10 res: 0#> s: 10:11:12 len: 8 res: 0#> One: 2016-09-01 two: 10:11:12#> before parse: 2016-09-01 10:11:12#> Boost parsed as 2016-Sep-01 10:11:12 which is UTC 1472724672 local 1472688672 diff 36000#> [1] "2016-09-01 10:11:12 AEST"
Created on 2019-06-28 by the reprex package <https://reprex.tidyverse.org>
(v0.2.1)
Win
tz <-"Australia/Sydney"
Sys.timezone()#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()#> [1] "Australia/Sydney"
library(anytime)anytime:::setDebug(TRUE)#> [1] TRUEanytime:::.pkgenv$tz#> [1] "Australia/Sydney"
as.POSIXct("2016-09-01 10:11:12")#> [1] "2016-09-01 10:11:12 AEST"
anytime("2016-09-01 10:11:12")#> before tests: 2016-09-01 10:11:12#> In: 2016-09-01 10:11:12 out: 2016-09-01 and 10:11:12#> s: 2016-09-01 len: 10 res: 0#> s: 10:11:12 len: 8 res: 0#> One: 2016-09-01 two: 10:11:12#> before parse: 2016-09-01 10:11:12#> Boost parsed as 2016-Sep-01 10:11:12 which is UTC 1472724672 local 1472717472 diff 7200#> [1] "2016-09-01 18:11:12 AEST"
Created on 2019-06-28 by the reprex package <https://reprex.tidyverse.org>
(v0.3.0)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#96?email_source=notifications&email_token=AAHZZWKQE7Q7CKKJMSYWMHLP4TZ5PA5CNFSM4H3U6RR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYX2DVI#issuecomment-506438101>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAHZZWMIYB2BB5PILUNARU3P4TZ5PANCNFSM4H3U6RRQ>
.
|
Mactz <-"Australia/Sydney"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "Australia/Sydney"
library(anytime)
anytime:::setDebug(TRUE)
#> [1] TRUE
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"
as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 AEST"
anytime("2016-09-01 10:11:12")
#> before tests: 2016-09-01 10:11:12
#> In: 2016-09-01 10:11:12 out: 2016-09-01 and 10:11:12
#> s: 2016-09-01 len: 10 res: 0
#> s: 10:11:12 len: 8 res: 0
#> One: 2016-09-01 two: 10:11:12
#> before parse: 2016-09-01 10:11:12
#> Boost parsed as 2016-Sep-01 10:11:12 which is UTC 1472724672 local tdiff is 409080:11:12
#> pt is 2016-Sep-01 10:11:12
#> 1472688672 diff tdiff is 409080:11:12
#> pt is 2016-Sep-01 10:11:12
#> 36000
#> tdiff is 409080:11:12
#> pt is 2016-Sep-01 10:11:12
#> [1] "2016-09-01 10:11:12 AEST"Created on 2019-06-28 by the reprex package (v0.3.0) Mac (updated)tz <-"Australia/Sydney"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "Australia/Sydney"
library(anytime)
anytime:::setDebug(TRUE)
#> [1] TRUE
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"
as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 AEST"
anytime("2016-09-01 10:11:12")
#> before tests: 2016-09-01 10:11:12
#> In: 2016-09-01 10:11:12 out: 2016-09-01 and 10:11:12
#> s: 2016-09-01 len: 10 res: 0
#> s: 10:11:12 len: 8 res: 0
#> One: 2016-09-01 two: 10:11:12
#> before parse: 2016-09-01 10:11:12
#> Boost parsed as 2016-Sep-01 10:11:12 which is UTC 1472724672 local tdiff is 409080:11:12
#> pt is 2016-Sep-01 10:11:12
#> 1472688672 diff tdiff is 409080:11:12
#> pt is 2016-Sep-01 10:11:12
#> 36000
#> tdiff is 409080:11:12
#> pt is 2016-Sep-01 10:11:12
#> [1] "2016-09-01 10:11:12 AEST"Created on 2019-07-01 by the reprex package (v0.3.0) Wintz <-"Australia/Sydney"
Sys.timezone()
#> [1] "Europe/Berlin"
Sys.setenv("TZ"=tz)
Sys.timezone()
#> [1] "Australia/Sydney"
library(anytime)
anytime:::setDebug(TRUE)
#> [1] TRUE
anytime:::.pkgenv$tz
#> [1] "Australia/Sydney"
as.POSIXct("2016-09-01 10:11:12")
#> [1] "2016-09-01 10:11:12 AEST"
anytime("2016-09-01 10:11:12")
#> before tests: 2016-09-01 10:11:12
#> In: 2016-09-01 10:11:12 out: 2016-09-01 and 10:11:12
#> s: 2016-09-01 len: 10 res: 0
#> s: 10:11:12 len: 8 res: 0
#> One: 2016-09-01 two: 10:11:12
#> before parse: 2016-09-01 10:11:12
#> tdiff is 409089:11:12
#> pt is 2016-Sep-01 10:11:12
#> tdiff is 409089:11:12
#> pt is 2016-Sep-01 10:11:12
#> Boost parsed as 2016-Sep-01 10:11:12 which is UTC 1472724672 local 1472717472 diff 7200
#> tdiff is 409089:11:12
#> pt is 2016-Sep-01 10:11:12
#> [1] "2016-09-01 18:11:12 AEST"Created on 2019-06-28 by the reprex package (v0.3.0) |
|
Nicer diff: diff --git a/src/anytime.cpp b/src/anytime.cpp
index b1dd19f..57146e1 100644
--- a/src/anytime.cpp
+++ b/src/anytime.cpp
@@ -157,7 +157,7 @@ static std::string setupTZ;
// account for localtime, and also account for dst
double ptToDouble(const bt::ptime & pt, const bool asDate=false) {
- //This local adjustor depends on the machine TZ settings-- highly dangerous!
+ // This local adjustor depends on the machine TZ settings-- highly dangerous!
typedef boost::date_time::c_local_adjustor<bt::ptime> local_adj;
const bt::ptime timet_start(boost::gregorian::date(1970,1,1));
@@ -165,8 +165,10 @@ double ptToDouble(const bt::ptime & pt, const bool asDate=false) {
// seconds sincr epoch (in local time) -- misses DST adjustment
bt::time_duration tdiff = pt - local_timet_start;
- //Rcpp::Rcout << "tdiff is " << tdiff << std::endl;
- //Rcpp::Rcout << "pt is " << pt << std::endl;
+ if (debug) {
+ Rcpp::Rcout << "tdiff in sec is " << tdiff.total_seconds()
+ << " and pt is " << pt << std::endl;
+ }
if (asDate) {
if (debug) Rcpp::Rcout << "days " << pt.date().day_number() - timet_start.date().day_number() << std::endl;
@@ -225,6 +227,14 @@ double stringToTime(const std::string s, const bool asUTC=false, const bool asDa
if (pt == ptbase) return NA_REAL; // NA for non-parsed dates
+ if (debug) {
+ int utc = static_cast<int>(ptToDoubleUTC(pt));
+ int loc = static_cast<int>(ptToDouble(pt));
+ Rcpp::Rcout << "Boost parsed as " << pt
+ << " which is UTC " << utc
+ << " local " << loc
+ << " diff " << utc-loc << std::endl;
+ }
if (asUTC) {
return ptToDoubleUTC(pt, asDate);
} else {I think what we are coming up against may be local adjustment to machine time being less solid with Boost than it is with R. |
|
@christophsax it looks like the Mac output in #96 (comment) was mangled a bit. Can you please re-run and update the Mac results? After looking at this a bit more, I agree with Dirk. The issue looks like it's somewhere in Boost. My current guess is that the issue happens on anytime.c:223. It would be useful if the debugging output would include the timezone. I'm not sure how to force that output format in C++ though. @dirk, do you know? |
|
Yes indeed. If either one of you could try build a Boost example such as this one on local/UCT changes I would appreciate it. Should not need more than |
|
I am away from my laptop for 2 days. Can only re run on Sunday. |
|
@joshuaulrich what do you mean with mangled? I updated re-run the example and get the same results (updated the comment you mentioned).
Unless I got instructions designed for dummies, this is over my head. |
It looks intimidating, but isn't. Save the file as .e.g. In other words, tell |
|
Ah shucks, never mind. It needs linking for string formatting (which we get from R in anytime) so you need a full boost. Using my (system-level) Boost libraries edd@rob:/tmp$ g++ -o boostex boostex.cpp -lboost_date_time
edd@rob:/tmp$ ./boostex
UTC <--> Zone base on TZ setting
2002-Jan-01 01:00:00 in your TZ is 2002-Jan-01 07:00:00 UTC time
A difference of: -06:00:00
UTC <--> New York while DST is NOT active (5 hours)
2001-Dec-31 19:00:00 in New York is 2002-Jan-01 00:00:00 UTC time
2002-Jan-01 00:00:00 UTC is 2001-Dec-31 19:00:00 New York time
UTC <--> New York while DST is active (4 hours)
2002-May-31 20:00:00 in New York is 2002-Jun-01 00:00:00 UTC time
2002-Jun-01 00:00:00 UTC is 2002-May-31 20:00:00 New York time
UTC <--> Arizona (7 hours)
2002-May-31 17:00:00 in Arizona is 2002-Jun-01 00:00:00 UTC time
edd@rob:/tmp$ I could cook this up as an Rcpp-using example. We still need to test string conversion from Boost. |
|
It seems not Australia specific: Win> input <- "2019-07-05 09:37:58.541235"
>
> Sys.setenv("TZ"="Japan")
> as.POSIXct(input)
[1] "2019-07-05 09:37:58 JST"
>
>
> anytime::anytime(input, useR = TRUE)
[1] "2019-07-05 17:37:58 JST"
> anytime::anytime(input)
[1] "2019-07-05 17:37:58 JST"Mac> input <- "2019-07-05 09:37:58.541235"
>
> Sys.setenv("TZ"="Japan")
> as.POSIXct(input)
[1] "2019-07-05 09:37:58 JST"
>
>
> anytime::anytime(input, useR = TRUE)
[1] "2019-07-05 09:37:58 JST"
> anytime::anytime(input)
[1] "2019-07-05 09:37:58 JST" |
|
Running the script input <- "2019-07-05 09:37:58.541235"
Sys.setenv("TZ"="Japan")
library(anytime)
anytime(input, useR = TRUE)
anytime(input)I get what you get for the mac, ie $ Rscript tmp/issue96.R
[1] "2019-07-05 09:37:58.541234 JST"
[1] "2019-07-05 09:37:58.541234 JST"
$Otherwise I am out of steam. Someone who a) has a windows box and b) cares enough needs to c) dive into the code. I can't run on Windows. Maybe @joshuaulrich or someone else can corrobate what happens there. Can we exclude any influence from your VM? |
|
For now, it seems sufficient to say that TZ setting does not work on Windows, as you did above. Just don't offer Since I am using anytime with I use a virtual desktop on https://aws.amazon.com/workspaces/ which generally works well for my debugging needs. I also have a physical Windows Laptop but without RTools, so I cannot check there. |
|
Something is clearly off. There is close to no code difference. There is one block in |
|
I can confirm this isn't specific to Australia. I chose some other
timezones ahead of UTC and got similar results.
… |
|
Thanks |
|
I love working with timezones as it is the surest way to loose my sanity quickly and reliably. We were conflating parsing, and formatting. I am now suspecting that there is another error in formatting. But the following now works for me on Linux and via rhub: input <- "2019-07-05 09:37:58"
Sys.setenv("TZ"="Japan")
library(anytime)
n1 <- as.numeric(anytime(input, useR = TRUE))
n2 <- as.numeric(anytime(input))
n3 <- as.numeric(as.POSIXct("2019-07-05 09:37:58"))
expect_equal(n1, n2, tol=1.0e-4)
expect_equal(n1, n3, tol=1.0e-4)
expect_equal(n2, n3, tol=1.0e-4)Of course, I started with |
|
It is confusing. On my AWS machine, it won't pass: > input <- "2019-07-05 09:37:58"
>
> Sys.setenv("TZ"="Japan")
>
> library(anytime)
>
> n1 <- as.numeric(anytime(input, useR = TRUE))
> n2 <- as.numeric(anytime(input))
> n3 <- as.numeric(as.POSIXct("2019-07-05 09:37:58"))
>
> library(tinytest)
> expect_equal(n1, n2, tol=1.0e-4)
----- PASSED : <-->
call| expect_equal(n1, n2, tol = 1e-04)
> expect_equal(n1, n3, tol=1.0e-4)
----- FAILED[attr]: <-->
call| expect_equal(n1, n3, tol = 1e-04)
diff| Mean relative difference: 1.843417e-05
> expect_equal(n2, n3, tol=1.0e-4)
----- FAILED[attr]: <-->
call| expect_equal(n2, n3, tol = 1e-04)
diff| Mean relative difference: 1.843417e-05 But in the |
|
I needed a few iterations with the |
|
Also worked on Travis FWIW: https://travis-ci.org/eddelbuettel/anytime/builds/554843509 |
|
I don't think a problem with > input <- "2019-07-05 09:37:58"
>
> Sys.setenv("TZ"="Japan")
>
> library(anytime)
>
> n1 <- as.numeric(anytime(input, useR = TRUE))
> n2 <- as.numeric(anytime(input))
> n3 <- as.numeric(as.POSIXct("2019-07-05 09:37:58"))
>
> n1 - n2
[1] 0
> n1 - n3
[1] 28800
> n2 - n3
[1] 28800 |
|
Yes, I also realized later that the numbers being large enough, changed the tolerance washes the result away. I do not know why n1 and n2 are different from n3 on Windows. One thing I realized, though, is that Still. I have no idea why Windows differs. When you set no TZ, the results remain the same though, correct? |
|
Here is without setting it, staying with > input <- "2019-07-05 09:37:58"
>
> library(anytime)
>
> n1 <- as.numeric(anytime(input, useR = TRUE))
> n2 <- as.numeric(anytime(input))
> n3 <- as.numeric(as.POSIXct("2019-07-05 09:37:58"))
>
> n1 - n2
[1] 0
> n1 - n3
[1] 3600
> n2 - n3
[1] 3600 |
|
So even the DST field makes it differ. Weird. There is nothing I can do from here. POSIXct and POSIXlt can be complicated. You can try to poke around interactively, maybe do a #if defined(_WIN32)
if (totsec > 0 && // on Windows before 1970-01-01: segfault
localAsTm->tm_year < 1100) { // tm_year is year-1900, so year 3000 is it
dstadj = localAsTm->tm_isdst*60*60;
}
#else
dstadj = localAsTm->tm_isdst*60*60;
#endifbut as we see it sets the DST adjustment for all OSs so I have no idea anymore. I could just turn Windows builds at CRAN off to have some peace and quiet. |
|
I had one more idea about |
|
I had one more thought while on the airplane: maybe the way we return the Datetime object by relying on Rcpp slips up on Windows. We could backtrack and see if the parsed text, when represented "still inside anytime.cpp as a double" has the correct value on Windows. Can you insert a printf? I can probably post a diff here too. |
|
I added the line ``return totsec - dstadj;`c++ I get edd@rob:~/git/anytime(master)$ Rscript local/gh_issue_96.R
[1] "2019-07-05 09:37:58.541234 CEST"
totsec secs is 1562315878.541235 and dstadj is 3600
[1] "2019-07-05 09:37:58.541234 CEST"
[1] "2019-07-05 09:37:58.541234 CEST"
totsec secs is 1562315878.541235 and dstadj is 3600
[1] 1562312278.541235
n1 - n2: 0
n1 - n3: 0
n2 - n3: 0
edd@rob:~/git/anytime(master)$showing the agreement for Japan. For Berlin we see a difference as there is a DST: [1] "2019-07-05 09:37:58.541234 CEST"
totsec secs is 1562315878.541235 and dstadj is 3600
[1] "2019-07-05 09:37:58.541234 CEST"
[1] "2019-07-05 09:37:58.541234 CEST"
totsec secs is 1562315878.541235 and dstadj is 3600
[1] 1562312278.541235
n1 - n2: 0
n1 - n3: 0
n2 - n3: 0 |
|
Same here (Windows): Japan> input <- "2019-07-05 09:37:58"
> Sys.setenv("TZ"="Japan")
> library(anytime)
>
> as.numeric(anytime(input, useR = TRUE))
[1] 1562315878
> as.numeric(anytime(input))
totsec secs is 1562319478 and dstadj is 3600
[1] 1562315878CEST> input <- "2019-07-05 09:37:58"
> Sys.setenv("TZ"="CEST")
> library(anytime)
>
> as.numeric(anytime(input, useR = TRUE))
[1] 1562315878
> as.numeric(anytime(input))
totsec secs is 1562319478 and dstadj is 3600
[1] 1562315878 |
|
That is the first bit of encouraging news, I think. I think we can trace from here. Maybe it merely is a bug in Datetime object creation by Rcpp on Windows. To be continued .... |
|
That wasn't the full test on your side though. Can you rebuild Rcpp::Rcout << "totsec secs is " << std::setprecision(16) << totsec << " and dstadj is " << dstadj << std::endl;
return totsec - dstadj;and then run this script for timezones with/without dst: input <- "2019-07-05 09:37:58.541235"
#Sys.setenv("TZ"="Japan")
Sys.setenv("TZ"="Europe/Berlin")
library(anytime)
anytime(input, useR = TRUE)
anytime(input)
as.POSIXct(input)
n1 <- as.numeric(anytime(input, useR = TRUE))
n2 <- as.numeric(anytime(input))
n3 <- as.numeric(as.POSIXct(input))
print(n3,digits=16)
cat("n1 - n2: ", n1 - n2, "\n")
cat("n1 - n3: ", n1 - n3, "\n")
cat("n2 - n3: ", n2 - n3, "\n") |
|
And I also committed the change and helper script so if you pull from the repo you should get it. |
Berlin> input <- "2019-07-05 09:37:58.541235"
>
> #Sys.setenv("TZ"="Japan")
> Sys.setenv("TZ"="Europe/Berlin")
> library(anytime)
> anytime:::setDebug(TRUE)
[1] TRUE
>
> anytime(input, useR = TRUE)
before tests: 2019-07-05 09:37:58.541235
In: 2019-07-05 09:37:58.541235 out: 2019-07-05 and 09:37:58.541235
s: 2019-07-05 len: 10 res: 0
s: 09:37:58.541235 len: 15 res: 0
One: 2019-07-05 two: 09:37:58.541235
before parse: 2019-07-05 09:37:58.541235
[1] "2019-07-05 10:37:58 CEST"
> anytime(input)
before tests: 2019-07-05 09:37:58.541235
In: 2019-07-05 09:37:58.541235 out: 2019-07-05 and 09:37:58.541235
s: 2019-07-05 len: 10 res: 0
s: 09:37:58.541235 len: 15 res: 0
One: 2019-07-05 two: 09:37:58.541235
before parse: 2019-07-05 09:37:58.541235
tdiff in sec is 1562319478 and pt is 2019-Jul-05 09:37:58.541235
totsec secs is 1562319478.541235 and dstadj is 3600
Boost parsed as 2019-Jul-05 09:37:58.541235 which is UTC 1562319478 local 1562315878 diff 3600
tdiff in sec is 1562319478 and pt is 2019-Jul-05 09:37:58.541235
totsec secs is 1562319478.541235 and dstadj is 3600
[1] "2019-07-05 10:37:58 CEST"Japan> input <- "2019-07-05 09:37:58.541235"
>
> Sys.setenv("TZ"="Japan")
> # Sys.setenv("TZ"="Europe/Berlin")
> library(anytime)
> anytime:::setDebug(TRUE)
[1] TRUE
>
> anytime(input, useR = TRUE)
before tests: 2019-07-05 09:37:58.541235
In: 2019-07-05 09:37:58.541235 out: 2019-07-05 and 09:37:58.541235
s: 2019-07-05 len: 10 res: 0
s: 09:37:58.541235 len: 15 res: 0
One: 2019-07-05 two: 09:37:58.541235
before parse: 2019-07-05 09:37:58.541235
[1] "2019-07-05 17:37:58 JST"
> anytime(input)
before tests: 2019-07-05 09:37:58.541235
In: 2019-07-05 09:37:58.541235 out: 2019-07-05 and 09:37:58.541235
s: 2019-07-05 len: 10 res: 0
s: 09:37:58.541235 len: 15 res: 0
One: 2019-07-05 two: 09:37:58.541235
before parse: 2019-07-05 09:37:58.541235
tdiff in sec is 1562319478 and pt is 2019-Jul-05 09:37:58.541235
totsec secs is 1562319478.541235 and dstadj is 3600
Boost parsed as 2019-Jul-05 09:37:58.541235 which is UTC 1562319478 local 1562315878 diff 3600
tdiff in sec is 1562319478 and pt is 2019-Jul-05 09:37:58.541235
totsec secs is 1562319478.541235 and dstadj is 3600
[1] "2019-07-05 17:37:58 JST" |
|
So I realized belatedly that I could test via |
|
@christophsax @joshuaulrich : I completely rewrote This is currently in a branch feature/no_rapidatetime I made yesterday (but only pushed just now, my bad). There are rhub artifacts if you want a zipfile built for Windows. Please take a look, I think this may be a better version. In short, Windows (default) mode remains complicated because how Boost does this, and we may now have a better fallback for Windows. Let me know what you think. I am even thinking we could make Edit: A fresher set of build artifacts with the updated documentation etc pp: anytime 0.3.4.5: OK Build ID: anytime_0.3.4.5.tar.gz-489c96ab840f412fa5134d674708fe26 See the full build log: |
|
And more with the comparison above across four times zone. No difference between "R code" and anytime with This demonstrates the difficulty of setting TZ on Windows, but that the now-added workaround is effective. Do you concur? |
|
That looks fantastic! My system's time zone is set to Berlin, so Otherwise, no differences between I have no opinion on what the default of library(anytime)
>
> input <- format( utctime( 1.5e9 ) )
> options(scipen=999)
>
> for (tz in c("UTC", "Europe/Berlin", "Europe/Athens", "Japan", "America/Chicago")) {
+ cat("Setting tz to", tz, "\n")
+ Sys.setenv("TZ"=tz)
+ anytime:::setTZ(tz)
+ Sys.sleep(0.25)
+
+ #print(anytime(input, tz=tz))
+ #print(anytime(input, useR = TRUE, tz=tz))
+ #print(format(as.POSIXct(input), tz=tz))
+
+ n0 <- as.numeric(as.POSIXct(input))
+ n1 <- as.numeric(anytime(input, tz=tz))
+ n2 <- as.numeric(anytime(input, useR = TRUE, tz=tz))
+ print(c(n0,n1,n2),digits=16)
+
+ #cat("n0 - n1: ", n0 - n1, "\n")
+ #cat("n0 - n2: ", n0 - n2, "\n")
+ #cat("n1 - n2: ", n1 - n2, "\n")
+ print(c(n0-n1, n0-n2, n1-n2))
+ }
Setting tz to UTC
[1] 1500000000 1499992800 1500000000
[1] 7200 0 -7200
Setting tz to Europe/Berlin
[1] 1499992800 1499992800 1499992800
[1] 0 0 0
Setting tz to Europe/Athens
[1] 1499989200 1499992800 1499989200
[1] -3600 0 3600
Setting tz to Japan
[1] 1499967600 1499992800 1499967600
[1] -25200 0 25200
Setting tz to America/Chicago
[1] 1500018000 1499992800 1500018000
[1] 25200 0 -25200 |
|
Yay! This issue was really bugging me. Looks like we have it squashed. I'll make the PR and merge, and maybe work a little on documentation. I am still thinking about what to do with the |
|
Also, the PR, once I merge it, will close this bug report. We may open a new wishlist one to see if any of the currently "paused" tests can be relaxed on Windows when setting |
Many of the Windows failures are probably due to something like the following. It seems TZ is not set correctly, unless you use
useR = TRUE. Perhaps related to the code piece inr_stringToTime()?If this were resolved, more tests could pass under Windows.
Created on 2019-06-26 by the reprex package (v0.3.0)