POSIXct values become numeric in reshape2 dcast #26

Closed
gkaupas opened this Issue Sep 12, 2012 · 2 comments

2 participants

@gkaupas

When using dcast where the value.var is a POSIXct type, the date values lose their POSIXct class and become numeric.

Per thread here, underlying cause appears to be the as.vector call within as.data.frame.matrix.

Example session:

> x <- c("a","b");
> y <- c("c","d");
> z <- as.POSIXct(c("2012-01-01 01:01:01","2012-02-02 02:02:02"));
> d <- data.frame(x, y, z, stringsAsFactors=FALSE);
> str(d);
'data.frame':   2 obs. of  3 variables:
 $ x: chr  "a" "b"
 $ y: chr  "c" "d"
 $ z: POSIXct, format: "2012-01-01 01:01:01" "2012-02-02 02:02:02"
> library(reshape2);
> e <- dcast(d, formula = x ~ y, value.var = "z");
> str(e);
'data.frame':   2 obs. of  3 variables:
 $ x: chr  "a" "b"
 $ c: num  1.33e+09 NA
 $ d: num  NA 1.33e+09
@gkaupas

While I feel this should be addressed, denormalizing POSIXct values runs orders of magnitude longer than the same data in integer (epoch second) form.

Given two files, 1,000,000 rows each, identical except for the formatting of the date:

$ head example-yyyymmdd.dat
X10001  Y30001  2012-08-14 22:06:40
X10001  Y30002  2012-08-14 22:06:41
X10001  Y30003  2012-08-14 22:06:42
X10001  Y30004  2012-08-14 22:06:43
X10001  Y30005  2012-08-14 22:06:44

$ head example-epochsec.dat
X10001  Y30001  1345000000
X10001  Y30002  1345000001
X10001  Y30003  1345000002
X10001  Y30004  1345000003
X10001  Y30005  1345000004

The dcast on a data.frame made from the first, with colClasses=c("character", "character", "POSIXct"), required 5,255 seconds. The dcase on a data.frame made from the second, with colClasses=c("character", "character", "integer"), required 9 seconds, 0.17% of the time for the POSIXct dcast.

Based on that I will have to change my underlying data format.

@hadley
Owner

Unfortunately I don't think I can fix this in reshape2 without adding a lot of extra code.

@hadley hadley closed this Mar 10, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment