Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow map_* to treat NULLs as NAs #231

Closed
jeremystan opened this issue Aug 27, 2016 · 6 comments
Closed

Allow map_* to treat NULLs as NAs #231

jeremystan opened this issue Aug 27, 2016 · 6 comments

Comments

@jeremystan
Copy link

When working with sparse nested lists (like JSON), it is common to have missing keys or NULL values, which are difficult to coerce into a desired type with purrr.

For example:

list(list("a" = 1L), list("b" = 2L)) %>% map_int("a")
#> Error: Result 2 is not a length 1 atomic vector

We can achieve the desired effect by mapping twice and using the null-replace operator

> list(list("a" = 1L), list("b" = 2L)) %>% map("a") %>% map_int(`%||%`, NA)
#> [1]  1 NA

Ideally, map_* and flatten_* would treat NULLs as NAs, possibly by default or if not then by an argument. It could throw a warning when this occurs.

The options I see are:

  1. Throw an error
  2. Return a vector that excludes the NULL entries
  3. Return a vector with appropriate NAs where NULLs were encountered

Currently, purrr implements option 1, which will lead to difficult to debug failures when working with JSON, and a lot of duplicated code to prevent it. I believe option 2 is very dangerous (this is what unlist does). Hence the suggestion for option 3.

@jennybc
Copy link
Member

jennybc commented Aug 27, 2016

This ground has been explored some before (#110 Default value to replace NULL), which lead to the .null argument.

Does this help?

library(purrr)
list(list("a" = 1L), list("b" = 2L)) %>% map_int("a", .null = NA_integer_)
#> [1]  1 NA

@jeremystan
Copy link
Author

I totally missed that (and this issue) - my apologies.

I missed it in part because it's not in the parameters section of the documentation. Should it be added there? It is mentioned under the .f description of the arguments section (which I obviously didn't read closely enough).

@jeremystan
Copy link
Author

OK, so on further inspection, the .null argument solves for the missing key, but doesn't solve for the null value:

list(list("a" = 1L), list("a" = NULL)) %>% map_int("a", .null = NA)
#> Error: Result 2 is not a length 1 atomic vector

Reading through #110, it seems like the discussion lost the need to treat NULL values as well?

@jennybc
Copy link
Member

jennybc commented Aug 28, 2016

Good point about explicit NULLs! Obviously that's what I was struggling with too. I don't know if that case was intentionally excluded from the .null treatment? But maybe we will find out ...

lionel- added a commit to lionel-/lowliner that referenced this issue Aug 29, 2016
@lionel-
Copy link
Member

lionel- commented Aug 29, 2016

I also think actual NULLs should be replaced by .null. Else we should have called that argument something like .void.

@cboettig
Copy link

cboettig commented Nov 7, 2022

I always struggle with this one.

stuff <-  c(list("a"), list(NULL))
stuff |> purrr::map_chr(as.character, .null=NA_character_)
#> Error in `stop_bad_type()`:
#> ! Result 2 must be a single string, not a character vector of length 0

#> Backtrace:
#>     ▆
#>  1. ├─purrr::map_chr(stuff, as.character, .null = NA_character_)
#>  2. └─purrr:::stop_bad_element_vector(...)
#>  3.   └─purrr:::stop_bad_vector(...)
#>  4.     └─purrr:::stop_bad_type(...)
#>  5.       └─rlang::abort(...)

Created on 2022-11-07 with reprex v2.0.2

Same with .default=NA_character_. (also .null is not mentioned in the map_chr docs? What's the recommended strategy here?

Apologies for posting on a closed issue. Happy to open another issue if that's preferred, though this one remains in my testing the top google hit for "purrr map NULL to NA" and related searches.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants