Implement keep_empty for unnest_longer()

[DavisVaughan]

Closes #1363
Closes #1339

We are going to use keep_empty = FALSE as the default to be consistent with unnest(), unchop(), separate_longer_position(), etc. This is a breaking change as previously NULL elements implicitly had keep_empty = TRUE applied to them. However, we are now consistent with how we handle NULL and empty vectors like integer(), which I think is important.

No changes to worse in revdeps, so I feel good about this!
"322b4064-9f31-474f-9750-36e8c7c2d8df"

[DavisVaughan]

I am slightly questioning our decision to use NA when there aren't any names over "", like what names2() would return

[hadley]

It's not that there are no names here, it's that there are no values. So I think NA is correct.

[DavisVaughan]

I'm really referring to this

tibble(
  x = list(1:2, c(a = 3, b = 4))
) %>%
  unnest_longer(x)
#> # A tibble: 4 × 2
#>       x x_id 
#>   <dbl> <chr>
#> 1     1 <NA> 
#> 2     2 <NA> 
#> 3     3 a    
#> 4     4 b

[DavisVaughan]

Maybe we could change ^ to use "", and then consistently use NA_integer_ or NA_character_ when we promote empty vectors to size 1? So its like:

No names anywhere:

• Positional names for existing values
• NA_integer_ if a zero length vector was promoted to size 1

At least one name somewhere:

• Elements with names use their names
• Elements without names use ""
• NA_character_ if a zero length vector was promoted to size 1

[DavisVaughan]

Using "" for values without names is probably more consistent in this case

tibble(
  x = list(c(a = 1, 2), c(3, 4))
) %>%
  unnest_longer(x)
#> # A tibble: 4 × 2
#>       x x_id 
#>   <dbl> <chr>
#> 1     1 "a"  
#> 2     2 ""   
#> 3     3 <NA>  # <- better as ""
#> 4     4 <NA>  # <- better as ""

[hadley]

I kind of like using "" for unnamed elements and NA for elements that didn't exist. But I can see how that might be slicing things too fine.

[DavisVaughan]

Isn't that what I'm suggesting? 3 and 4 above are "unnamed elements" to me

[hadley]

Like this case:

library(tidyr)
tibble(x = list(c(a = 1, 2), c(3, 4), NULL)) %>%
  unnest_longer(x, keep_empty = TRUE)
#> # A tibble: 5 × 2
#>       x x_id 
#>   <dbl> <chr>
#> 1     1 "a"  
#> 2     2 ""   
#> 3     3 <NA>  # should be ""
#> 4     4 <NA>  # should be ""
#> 5    NA <NA> # should be NA

^{Created on 2023-01-05 with reprex v2.0.2}

[hadley]

It's not that there are no names here, it's that there are no values. So I think NA is correct.

[DavisVaughan]

This PR grew a bit, but I realized that I could vectorize col_to_long() a bit more by:

• Moving all names handling to collect_indices_info()
• Adding list_init_null() to handle converting NULLs to either size 0 or size 1 vectors
• Adding list_init_typed() to follow that up and convert size 0 vectors to size 1 vectors

We already have list_init_empty() that tries to do both of these at once, and we use it in a few other places. It isn't quite right for our needs here, so I had to break it apart into its two functions.

After this PR I'll go back and remove list_init_empty() in favor of using these two helpers. I'm also confident that when I do that I will be able to fix #1327 based on the way I've implemented these two new helpers.

---

I've also implemented the changes to the name handling we discussed here #1442 (comment). It was heavily tied to this so it was easiest to change it all at once.

[DavisVaughan]

The restriction of "no list-of" isn't technically necessary, in particular the previous list_init_empty() helper tried to allow list-ofs and use their information. But this is part of what caused the failure in #1327, and it is cleaner to extract out the ptype of list-ofs ahead of time, strip them down to bare lists, and pass all that through instead.

[hadley]

This makes sense to me because list_of() can't contain NULLs, right?

[DavisVaughan]

They can actually (that's the missing value) but can't contain unspecified() (although that branch probably wouldn't be hit for list-ofs because you'd typically also be supplying ptype)

[hadley]

This makes sense to me because list_of() can't contain NULLs, right?

[hadley]

Can't you assume that vec_assign() will do this?

[DavisVaughan]

It definitely will. But since I use null in two separate assignments below, it seemed slightly more efficient to which() it once here