group_by() and a filter() fails when using a factor column in a data.frame

[schloerke]

When using dplyr v0.7.8, there are no issues.

Using dplyr v0.8.0.9000, data.frames cause issues when grouped and then filtered. The missing levels found within the data.frame are creating unwanted combinations in the final result.

Using a tibble from the beginning does not cause an issue. In my case, it is useful to preserve the levels to use at a later time.

This was originally found using a dplyr::mutate command.

- Barret

Broken

Missing warning statements within the reprex that reference (converted from warning) no non-missing arguments to min; returning Inf.

library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union
packageVersion("dplyr")
#> [1] '0.8.0.9000'
packageVersion("tibble")
#> [1] '2.0.0'

dt <- data.frame(
  X = c("A", "B", "C", "A", "B", "C"),
  Y = 1:6,
  stringsAsFactors = TRUE
)

# I expect
dt[1,] %>%
  rename(minY = Y)
#>   X minY
#> 1 A    1

# Produces extra rows for combinations that don't exist in the filtered data
dt %>%
  filter(X == "A") %>%
  group_by(X) %>%
  summarise(minY = min(Y))
#> # A tibble: 3 x 2
#>   X      minY
#>   <fct> <dbl>
#> 1 A         1
#> 2 B       Inf
#> 3 C       Inf


# does not work when converted to tibble
dt %>%
  as_tibble() %>%
  filter(X == "A") %>%
  group_by(X) %>%
  summarise(minY = min(Y))
#> # A tibble: 3 x 2
#>   X      minY
#>   <fct> <dbl>
#> 1 A         1
#> 2 B       Inf
#> 3 C       Inf

# works when data being supplied is a character
dt %>%
  mutate(X = as.character(X)) %>%
  filter(X == "A") %>%
  group_by(X) %>%
  summarise(minY = min(Y))
#> # A tibble: 1 x 2
#>   X      minY
#>   <chr> <dbl>
#> 1 A         1

# works when data is originally a tibble
tibble(X = dt$X, Y = dt$Y) %>%
  filter(X == "A") %>%
  group_by(X) %>%
  summarise(minY = min(Y))
#> # A tibble: 3 x 2
#>   X      minY
#>   <fct> <dbl>
#> 1 A         1
#> 2 B       Inf
#> 3 C       Inf

^{Created on 2019-01-10 by the reprex package (v0.2.1)}

Working on CRAN

library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union
packageVersion("dplyr")
#> [1] '0.7.8'
packageVersion("tibble")
#> [1] '2.0.0'

dt <- data.frame(
  X = c("A", "B", "C", "A", "B", "C"),
  Y = 1:6,
  stringsAsFactors = TRUE
)

# I expect
dt[1,] %>%
  rename(minY = Y)
#>   X minY
#> 1 A    1

# Produces extra rows for combinations that don't exist in the filtered data
dt %>%
  filter(X == "A") %>%
  group_by(X) %>%
  summarise(minY = min(Y))
#> # A tibble: 1 x 2
#>   X      minY
#>   <fct> <dbl>
#> 1 A         1


# does not work when converted to tibble
dt %>%
  as_tibble() %>%
  filter(X == "A") %>%
  group_by(X) %>%
  summarise(minY = min(Y))
#> # A tibble: 1 x 2
#>   X      minY
#>   <fct> <dbl>
#> 1 A         1

# works when data being supplied is a character
dt %>%
  mutate(X = as.character(X)) %>%
  filter(X == "A") %>%
  group_by(X) %>%
  summarise(minY = min(Y))
#> # A tibble: 1 x 2
#>   X      minY
#>   <chr> <dbl>
#> 1 A         1

# works when data is originally a tibble
tibble(X = dt$X, Y = dt$Y) %>%
  filter(X == "A") %>%
  group_by(X) %>%
  summarise(minY = min(Y))
#> # A tibble: 1 x 2
#>   X      minY
#>   <fct> <dbl>
#> 1 A         1

^{Created on 2019-01-10 by the reprex package (v0.2.1)}

[romainfrancois]

This is a duplicate to #4061 and dealt with in the #4091 pull request. Once the pr is merged, you'll be able to use this:

library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union
packageVersion("dplyr")
#> [1] '0.8.0'
packageVersion("tibble")
#> [1] '2.0.0'

dt <- data.frame(
  X = c("A", "B", "C", "A", "B", "C"),
  Y = 1:6,
  stringsAsFactors = TRUE
)

# Produces extra rows for combinations that don't exist in the filtered data
dt %>%
  filter(X == "A") %>%
  group_by(X, .drop = TRUE) %>%
  summarise(minY = min(Y))
#> # A tibble: 1 x 2
#>   X      minY
#>   <fct> <dbl>
#> 1 A         1

^{Created on 2019-01-11 by the reprex package (v0.2.1.9000)}

[lock]

This old issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with reprex) and link to this issue. https://reprex.tidyverse.org/