error in arrange when using across(everything())

[mgacc0]

Using dplyr_1.0.99.9000:

    library(tidyverse)
    df <- tibble(
      letter1 = c("A", "C", "B"),
      letter2 = c("B", "C", "B"),
      number = c(3, 1, 2)
    )
    df %>%
      arrange(across(everything()))
#> # A tibble: 3 × 3
#>   letter1 letter2 number
#>   <chr>   <chr>    <dbl>
#> 1 A       B            3
#> 2 B       B            2
#> 3 C       C            1
    df %>%
      arrange(
        letter1 != letter2,
        across(everything())
        )
#> Error in `arrange()`:
#> ℹ In argument: `..2 = across(everything())`.
#> Caused by error:
#> ! ..1 used in an incorrect context, no ... to look in

#> Backtrace:
#>      ▆
#>   1. ├─df %>% arrange(letter1 != letter2, across(everything()))
#>   2. ├─dplyr::arrange(., letter1 != letter2, across(everything()))
#>   3. ├─dplyr:::arrange.data.frame(., letter1 != letter2, across(everything()))
#>   4. │ └─dplyr:::arrange_rows(.data, dots = dots, locale = .locale)
#>   5. │   ├─dplyr::mutate(new_data_frame(data), !!!quosures, .keep = "none")
#>   6. │   └─dplyr:::mutate.data.frame(...)
#>   7. │     └─dplyr:::mutate_cols(.data, dplyr_quosures(...))
#>   8. │       ├─base::withCallingHandlers(...)
#>   9. │       └─dplyr:::mutate_col(dots[[i]], .data, mask, new_columns)
#>  10. │         └─mask$eval_all_mutate(quo)
#>  11. │           └─dplyr (local) eval()
#>  12. ├─dplyr::across(everything())
#>  13. │ └─mask$pick(vars)
#>  14. │   └─self$current_cols(vars)
#>  15. │     └─rlang::env_get_list(parent.env(private$mask), vars)
#>  16. ├─`<fn>`()
#>  17. └─base::.handleSimpleError(...)
#>  18.   └─dplyr (local) h(simpleError(msg, call))
#>  19.     └─rlang::abort(message, class = error_class, parent = parent, call = error_call)

[hadley]

Somewhat more minimal reprex:

library(dplyr, warn.conflicts = FALSE)
df <- tibble(x = c(3, 1, 2))

arrange(df, TRUE, across(everything()))
#> Error in `arrange()`:
#> ! Can't compute `..2 = across(everything())`.
#> Caused by error:
#> ! ..1 used in an incorrect context, no ... to look in

#> Backtrace:
#>      ▆
#>   1. ├─dplyr::arrange(df, TRUE, across(everything()))
#>   2. ├─dplyr:::arrange.data.frame(df, TRUE, across(everything())) at dplyr/R/arrange.R:82:2
#>   3. │ └─dplyr:::arrange_rows(.data, dots = dots, locale = .locale) at dplyr/R/arrange.R:97:2
#>   4. │   ├─dplyr::mutate(new_data_frame(data), !!!quosures, .keep = "none") at dplyr/R/arrange.R:129:2
#>   5. │   └─dplyr:::mutate.data.frame(...) at dplyr/R/mutate.R:146:2
#>   6. │     └─dplyr:::mutate_cols(.data, dplyr_quosures(...)) at dplyr/R/mutate.R:175:2
#>   7. │       ├─base::withCallingHandlers(...) at dplyr/R/mutate.R:244:2
#>   8. │       └─(function() {... at dplyr/R/mutate.R:244:2
#>   9. │         └─dplyr:::mutate_col(dots[[i]], .data, mask, new_columns) at dplyr/R/mutate.R:249:6
#>  10. │           └─mask$eval_all_mutate(quo) at dplyr/R/mutate.R:365:8
#>  11. │             └─dplyr (local) eval() at dplyr/R/data-mask.R:68:6
#>  12. ├─dplyr::across(everything())
#>  13. │ └─mask$pick(vars) at dplyr/R/across.R:235:4
#>  14. │   └─self$current_cols(vars) at dplyr/R/data-mask.R:77:6
#>  15. │     └─rlang::env_get_list(parent.env(private$mask), vars) at dplyr/R/data-mask.R:91:6
#>  16. ├─`<fn>`()
#>  17. └─base::.handleSimpleError(...)
#>  18.   └─dplyr (local) h(simpleError(msg, call))
#>  19.     └─rlang::abort(message, class = error_class, parent = parent, call = error_call) at dplyr/R/conditions.R:223:4

^{Created on 2022-10-10 with reprex v2.0.2}

@DavisVaughan this is probably work taking a look at as part of your pick() work

[DavisVaughan]

There are quite a few things going on here, one of which is this interesting tidbit:

fn <- function() {
  ..1 <- 5
  ..1
}
fn()
#> Error in fn(): ..1 used in an incorrect context, no ... to look in

Which is fairly similar to how our active binding setup works.

So you can get stuff like this

library(dplyr, warn.conflicts = FALSE)
df <- tibble(x = 1)
mutate(df, `..1` = 2, y = `..1` + 2)
#> Error in `mutate()`:
#> ℹ In argument: `y = ..1 + 2`.
#> Caused by error:
#> ! ..1 used in an incorrect context, no ... to look in

Now the specific problem in arrange():

Remember that in arrange() we internally call mutate() with the ..., and we evaluate the ... all at once. So:

• We give dummy names to the dots, and the first happens to be named ..1
• Because across(everything()) looks at the "current" data, it tries to select both x and the new ..1 column through the active binding mechanism, which triggers the error we saw in fn().

So two things to think about:

• Should we even allow names like ..1 in mutate() calls (and elsewhere)? I'm not sure if we can fight that holistically.
• Is it a bug that arrange() is evaluating the dots all at once, resulting in this updated "current" view of the data for across()? i.e. should that across() call have really resulted in just "x" from the "original" data? We could evaluate the dots one at a time using a for loop like cols[[i]] <- mutate(data, ..arrange.. = !!dots[[i]])$..arrange.. to avoid this.