Fixes #19 JSON download error #102
Conversation
…deal with missing columns in JSON files. Changes JSON download tests because now JSON files download logical data as a logical data type, instead of character.
|
Haven't done a full code review yet, but a few things. The unit testing failed. Looks like the Also, be sure to sign the Contributor License Agreement. |
| @@ -125,7 +125,7 @@ test_that("read Socrata JSON as default", { | |||
| expect_equal(1007, nrow(df), label="rows") | |||
| expect_equal(11, ncol(df), label="columns") | |||
| expect_equal(c("character", "character", "character", "character", "character", | |||
| "character", "character", "character", "character", "character", | |||
| "character", "logical", "character", "character", "character", | |||
There was a problem hiding this comment.
Why is the "logical" return now needed? Is it for the needs_recoding field?
There was a problem hiding this comment.
Yes, it is for needs_recoding. It had been coming in as a "character" but with this code change it started coming in as a "logical". I think using fromJSON() is what caused this change.
There was a problem hiding this comment.
Ok, but we'll have to be very careful to note this since this could mess-up someone's existing code. That is, someone downloading from Socrata may have code that works because that field is downloaded as chr, but may not because it's logical.
I think this closes #19. It uses fromJSON to parse the download and then uses rbind.fill from plyr to deal with the NAs and missing columns in the JSON files.