Inconsistent Treatment of dates

[gerrymanoim]

I'm hitting an issue where depending on whether I put dates into arrays or not I get different object types in R.

Here is my TOML file:

[section]
date = "2017-01-01"
date_array = ["2017-01-01", "2017-01-01", "2017-01-01"]
date_unqoted = 2016-01-01
date_array_unquoted = [2016-01-01, 2016-01-01, 2016-01-01, 2016-01-01]

And R code:

library(RcppTOML)
toml <- parseTOML("date_bug.toml")
str(toml)

Which results in:

List of 1
 $ section:List of 4
  ..$ date               : chr "2017-01-01"
  ..$ date_array         : chr [1:3] "2017-01-01" "2017-01-01" "2017-01-01"
  ..$ date_array_unquoted: num [1:4] 16801 16801 16801 16801
  ..$ date_unqoted       : Date[1:1], format: "2016-01-01"
 - attr(*, "class")= chr [1:2] "toml" "list"
 - attr(*, "file")= chr "date_bug.toml"

as.Date(toml$date_array_unquoted, origin="1970-01-01") will return the right result, but it seems to me that this should be a date vector by default.

[eddelbuettel]

Thanks for the report. I concur that that should (ideally) not be happening.

The easy part first. Once you quote them, TOML leaves them alone and just treats them as character. So they never become dates. That is what JSON, XML, ... and all other formats without native dates do.

Next, single date. That works as expected.

Leaving the vector / array of dates. That is actually a known issue in R as well. I.e. the following has been biting me a large number of times:

R> dvec <- Sys.Date() + 0:2
R> dvec
[1] "2017-03-17" "2017-03-18" "2017-03-19"
R> for (d in dvec) print(d)
[1] 17242
[1] 17243
[1] 17244
R> 

I think we may just have the same thing happening here.

[eddelbuettel]

I.e.:

R> dvec
[1] "2017-03-17" "2017-03-18" "2017-03-19"
R> as.vector(dvec)
[1] 17242 17243 17244
R> 

[joshuaulrich]

This is a consequence of how for loops work in R.

?"for" says that seq (the part after in) is "[A]n expression evaluating to a vector (including a list and an expression) or to a pairlist or 'NULL'".

And "vector" refers to the "strict R" sense of the word. So, essentially, all attributes are ignored, including "class". See this StackOverflow Q&A for more details.

[eddelbuettel]

The same thing happens with Datetime. I.e. if I add

[another]
datetime = 2017-03-17T14:09:00
datetime_array = [2017-03-17T14:09:00, 2017-03-17T14:10:00, 2017-03-17T14:11:00]

I get

R> parseTOML("/tmp/datebug.toml")
List of 2
 $ another:List of 2
  ..$ datetime      : POSIXct[1:1], format: "2017-03-17 14:09:00"
  ..$ datetime_array: num [1:3] 1.49e+09 1.49e+09 1.49e+09
 $ section:List of 4
  ..$ date               : chr "2017-01-01"
  ..$ date_array         : chr [1:3] "2017-01-01" "2017-01-01" "2017-01-01"
  ..$ date_array_unquoted: num [1:4] 16801 16801 16801 16801
  ..$ date_unqoted       : Date[1:1], format: "2016-01-01"
R> 

But I don't think I have an easy fix.

[gerrymanoim]

Yeah - why its happening makes sense. Apologies, I'm not too familiar with Rcpp, but when you're creating vectors, can you set attributes on them inside of Rcpp?

> dvec <- Sys.Date() + 0:2
> attributes(dvec)
$class
[1] "Date"

> dvec <- as.vector(dvec)
> attributes(dvec)
NULL
> attr(dvec, "class") <- "Date"
> dvec
[1] "2017-03-17" "2017-03-18" "2017-03-19"

[eddelbuettel]

Nothing like some casual coding / debugging with a second set of (remote) eyes.

So thanks to @joshuaulrich for moral support. Adding a handful of lines to take care of Date and Datetime vectors fixes it:

~/git/rcpptoml(master)$ r -lRcppTOML -e'print(parseTOML("/tmp/datebug.toml"))'
List of 2
 $ another:List of 2
  ..$ datetime      : POSIXct[1:1], format: "2017-03-17 14:09:00"
  ..$ datetime_array: POSIXct[1:3], format: "2017-03-17 09:09:00" "2017-03-17 09:10:00" "2017-03-17 09:11:00"
 $ section:List of 4
  ..$ date               : chr "2017-01-01"
  ..$ date_array         : chr [1:3] "2017-01-01" "2017-01-01" "2017-01-01"
  ..$ date_array_unquoted: Date[1:4], format: "2016-01-01" "2016-01-01" "2016-01-01" "2016-01-01"
  ..$ date_unqoted       : Date[1:1], format: "2016-01-01"
~/git/rcpptoml(master)$ 

Will commit in a moment; feel free to update to GitHub before this gets to CRAN.

[joshuaulrich]

Glad to help. Also don't forget to thank past @eddelbuettel's answers on StackOverflow! ;)

[eddelbuettel]

I'd upvote him if I could :)

[joshuaulrich]

I noticed you only added code to handle Date and POSIXt in the REALSXP case. In general, it's possible for Date objects to be INTSXP. Could that happen here, or is there something that ensures they're always REALSXP?

Just thinking defensively. Feel free to ignore. :)

[eddelbuettel]

They come in as Rcpp types, and those are "REALSXP with a class attribute" because eg

R> as.numeric(Sys.Date())
[1] 17242
R> as.numeric(Sys.Date() + 0.5)
[1] 17242.5
R> 

At one point in the past I came across uses of intra-daily dates with fractions. That's when Rcpp::Date changes from int to numeric...