Allow to parse dates from long numbers #302

jiho · 2015-01-30T08:45:40Z

With as.character(), long numbers get converted to scientific
notation which the parse of course fails to recognise. Using
format(..., scientific = FALSE) prevents that.

jiho · 2015-01-30T09:06:00Z

Should really have run check(). Additional commits fix mistakes.

jiho · 2015-01-30T09:08:05Z

Example of what used to fail:

> x <- 20140101000000
> as.character(x)
[1] "2.0140101e+13"
> format(x, scientific = FALSE, trim = TRUE)
[1] "20140101000000"
> .num_to_date(x)
[1] "20140101000000"

It seems like a bug in as.character actually but it is probably safer to fix it here rather than wait for as.character to change...

vspinu · 2015-01-30T09:10:04Z

Thanks. Could you please add a small test for this?

Could you also squash your commits into one? We try to stick to one-commit-per-change rule to make it easier to navigate the history.

jiho · 2015-01-30T09:10:23Z

Regarding performance, format would actually be even a bit better:

> x <- round(runif(10^7) * 100000)
> system.time(as.character(x))
   user  system elapsed 
  7.786   0.015   7.816 
> system.time(format(x, scientific = FALSE, trim = TRUE))
   user  system elapsed 
  6.412   0.013   6.431

jiho · 2015-01-30T09:39:03Z

Tests are added and everything is squashed into one commit

vspinu · 2015-01-30T11:39:00Z

inst/tests/test-parsers.R

@@ -156,6 +156,20 @@ test_that("ymd functions correctly parse dates with no separators and no quotes"
              equals(as.POSIXct("2010-01-02 23:59:59", tz = "UTC")))
 })

+test_that("numbers are correctly converted into character for parsing", {
+  # numbers with 000000 at the end are wrongly converted by as.character


They are not wrongly converted. That's the feature which is customized by scipen option.

vspinu · 2015-01-30T11:39:46Z

inst/tests/test-parsers.R

+              equals(as.POSIXct("2010-01-02 00:00:00", tz = "UTC")))
+  expect_that(.num_to_date(20100102000000),
+              equals("20100102000000"))
+  # check for a fix in as.character


I don't think we need these tests. as.character is not broken and will never be fixed.

vspinu · 2015-01-30T21:25:23Z

This PR made me think, are there other places as.character will fail similarly?

jiho · 2015-02-02T14:02:37Z

Potential problems:

R/parse.r l. 455 as.character is used with randomly generated numbers which may end with many zeros by chance.
inst/tests/test-parser.r l. 520 same as above.
R/parse.r l. 489, as.character is used when it is not necessary (.num_to_date always returns a character)

The other uses in the source code do not seem problematic to me.

jiho · 2015-02-02T14:05:36Z

I've addressed the comments above. Let me know what you think. I guess I'll need to squash the commits again but I'll rather do that when all is accepted.

vspinu · 2015-02-02T19:18:24Z

R/parse.r l. 455 as.character is used with randomly generated numbers which may end with many zeros by chance.

Probably not an issue as as.character is applied to POSIXct object. as.character(.POSIXct(1000000000000)) works as expected.

R/parse.r l. 489, as.character is used when it is not necessary (.num_to_date always returns a character)

Good catch. That makes me think that .num_to_date is a misnomer. It doesn't convert to date.

I've addressed the comments above. Let me know what you think. I guess I'll need to squash the commits again but I'll rather do that when all is accepted.

Yes. Looks good. Please squash. Instead of adding new commits you can always amend previous commit and then force push.

Thanks.

With as.character(), long numbers with many zeros at the end get converted to scientific notation which the parse of course fails to recognise. Using format(..., scientific = FALSE) prevents that. trim = TRUE is necessary for variable length input. Add tests for these problematic numbers. Tests also check wether the behaviour of as.character() changes but format() outperforms as.character() and may be preferred anyway.

jiho · 2015-02-05T06:17:33Z

Squashed commits

Allow to parse dates from long numbers

vspinu · 2015-02-05T10:10:00Z

👍 Thanks!

jiho · 2015-02-05T11:10:52Z

You are welcome. Happy to help.

jiho force-pushed the jiho:fix_parse_numeric_dates branch from 35a62a2 to 369b9cc Jan 30, 2015

vspinu reviewed Jan 30, 2015
View changes

jiho force-pushed the jiho:fix_parse_numeric_dates branch from e88e055 to eee3e05 Feb 5, 2015

vspinu added a commit that referenced this pull request Feb 5, 2015

Merge pull request #302 from jiho/fix_parse_numeric_dates

Loading status checks…

7e62fb9

Allow to parse dates from long numbers

vspinu merged commit 7e62fb9 into tidyverse:master Feb 5, 2015
1 check passed

1 check passed

continuous-integration/travis-ci The Travis CI build passed
Details

tidyverse / lubridate

Allow to parse dates from long numbers #302

Allow to parse dates from long numbers #302

jiho commented Jan 30, 2015

jiho commented Jan 30, 2015

jiho commented Jan 30, 2015

vspinu commented Jan 30, 2015

jiho commented Jan 30, 2015

jiho commented Jan 30, 2015

This comment has been minimized.

This comment has been minimized.

vspinu commented Jan 30, 2015

jiho commented Feb 2, 2015

jiho commented Feb 2, 2015

vspinu commented Feb 2, 2015

jiho commented Feb 5, 2015

vspinu commented Feb 5, 2015

jiho commented Feb 5, 2015

tidyverse / lubridate

Join GitHub today

Allow to parse dates from long numbers #302

Allow to parse dates from long numbers #302

Conversation

jiho commented Jan 30, 2015

jiho commented Jan 30, 2015

jiho commented Jan 30, 2015

vspinu commented Jan 30, 2015

jiho commented Jan 30, 2015

jiho commented Jan 30, 2015

This comment has been minimized.

vspinu Jan 30, 2015 Member

This comment has been minimized.

vspinu Jan 30, 2015 Member

vspinu commented Jan 30, 2015

jiho commented Feb 2, 2015

jiho commented Feb 2, 2015

vspinu commented Feb 2, 2015

jiho commented Feb 5, 2015

vspinu commented Feb 5, 2015

jiho commented Feb 5, 2015

Essential cookies

Always active

Analytics cookies

vspinu Jan 30, 2015
Member

vspinu Jan 30, 2015
Member