Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Date bug in fwrite()? #2995

Closed
mdzorn opened this issue Aug 10, 2018 · 4 comments
Closed

Date bug in fwrite()? #2995

mdzorn opened this issue Aug 10, 2018 · 4 comments
Labels
Milestone

Comments

@mdzorn
Copy link

@mdzorn mdzorn commented Aug 10, 2018

I discovered unexpected behavior in data.table::fwrite: future dates reset to 1901. I was using the year 2999 to mask an unspecified date for determining whether an event occurred before a certain date.

Here's my example:

library("data.table", lib.loc="~/Library/R/3.5/library")
data.table 1.11.4 Latest news: http://r-datatable.com

DT <- data.table(A=1:5)
DT[1, D:=as.POSIXct("2018-01-01")]
DT[2, D:=as.POSIXct("2020-12-31")]
DT[3, D:=as.POSIXct("2040-12-31")]
DT[4, D:=as.POSIXct("2099-12-31")]

dput(DT)
structure(list(A = 1:5, D = structure(c(1514793600, 1609401600,
2240553600, 4102387200, NA), class = c("POSIXct", "POSIXt"), tzone = "")), row.names = c(NA, -5L), class = c("data.table", "data.frame"), .internal.selfref = <pointer: 0x103833ce0>)

It prints on the console as expected:

DT

A D
1: 1 2018-01-01
2: 2 2020-12-31
3: 3 2040-12-31
4: 4 2099-12-31
5: 5

However, when using fwrite() either to a file or console, some of the dates change:

fwrite(DT, file = "")
A,D
1,2018-01-01T08:00:00Z
2,2020-12-31T08:00:00Z
3,1901-12-14T.�48364Z
4,1901-12-14T.�48364Z
5,

Somewhere between 2030 and 2040 the dates change to 1901. I (10 decimal digits) shouldn't be running up against the limit of double number representation (15 decimal digits). Any ideas?

macos 10.13.6; R version 3.5.1 (2018-07-02); data.table 1.11.4

@mdzorn mdzorn changed the title Bug in fwrite()? Date bug in fwrite()? Aug 10, 2018
@HughParsonage
Copy link
Member

@HughParsonage HughParsonage commented Aug 11, 2018

https://en.wikipedia.org/wiki/Year_2038_problem ?

library(data.table)
fwrite(data.table(D = as.POSIXct(seq.Date(as.Date("2038-01-15"),
                                          as.Date("2038-01-20"),
                                          by = "day"))),
       "")
#> D
#> 2038-01-15T00:00:00Z
#> 2038-01-16T00:00:00Z
#> 2038-01-17T00:00:00Z
#> 2038-01-18T00:00:00Z
#> 2038-01-19T00:00:00Z
#> 1901-12-14T.“48364Z

philippechataignon pushed a commit to philippechataignon/data.table that referenced this issue Aug 21, 2018
Internal variables in writePOSIXct are defined as int. This commit
defined then as int64_t to avoid year 2038 bug as reported in
issue Rdatatable#2995.
@philippechataignon
Copy link
Contributor

@philippechataignon philippechataignon commented Aug 21, 2018

Internal variables xi, d, t in function writePOSIXct in fwrite.c are defined as int which is 32 bits long. To avoid year 2038 bug, they must be defined as int64_t.

@MichaelChirico
Copy link
Member

@MichaelChirico MichaelChirico commented Aug 21, 2018

@philippechataignon if you have a minute feel free to please file a PR, it would be appreciated :)

@mattdowle
Copy link
Member

@mattdowle mattdowle commented Aug 21, 2018

Closed by PR #3005. The PR didn't have "closes" keyword so GitHub didn't auto close this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants