error in group_by(df, Var1) %>% filter(some_list$x < Var2)

[tjmahr]

Here's a minimal example of the error.

library("dplyr")
# Suppose some analysis options are set much earlier in the script
analysis_opts <- list(min_outcome = .25)

# Generate some dummy data
d <- expand.grid(Subject = 1:3, TrialNo = 1:2, Time = 1:3) %>% tbl_df %>%
  arrange(Subject, TrialNo, Time) %>%
  mutate(Outcome = (1:18 %% c(5, 7, 11)) / 10)

# Do some aggregation
trial_outcomes <- d %>% group_by(Subject, TrialNo) %>%
  summarise(MeanOutcome = mean(Outcome))
trial_outcomes

## Source: local data frame [6 x 3]
## Groups: Subject
## 
##   Subject TrialNo MeanOutcome
## 1       1       1      0.2000
## 2       1       2      0.5000
## 3       2       1      0.4000
## 4       2       2      0.1667
## 5       3       1      0.2333
## 6       3       2      0.3667

Now try to filter the aggregated data using that list of analysis options.

# Works when some_list$x comes second
bad_trials <- trial_outcomes %>% filter(MeanOutcome < analysis_opts$min_outcome)

# Errors when some_list$x comes first
bad_trials <- trial_outcomes %>% filter(analysis_opts$min_outcome > MeanOutcome)

## Error: object 'MeanOutcome' not found

bad_trials <- filter(trial_outcomes, analysis_opts$min_outcome > MeanOutcome)

## Error: object 'MeanOutcome' not found

# But no errors when using brackets
bad_trials <- trial_outcomes %>% filter(analysis_opts["min_outcome"] > MeanOutcome)
bad_trials <- trial_outcomes %>% filter(analysis_opts[["min_outcome"]] > MeanOutcome)

# But no errors after ungrouping
bad_trials <- trial_outcomes %>% ungroup %>%
  filter(analysis_opts$min_outcome > MeanOutcome)

sessionInfo()

## R version 3.1.1 (2014-07-10)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## 
## locale:
## [1] LC_COLLATE=English_United States.1252 
## [2] LC_CTYPE=English_United States.1252   
## [3] LC_MONETARY=English_United States.1252
## [4] LC_NUMERIC=C                          
## [5] LC_TIME=English_United States.1252    
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
## [1] knitr_1.6      dplyr_0.2.0.99
## 
## loaded via a namespace (and not attached):
## [1] assertthat_0.1 evaluate_0.5.5 formatR_0.10   magrittr_1.0.1
## [5] parallel_3.1.1 Rcpp_0.11.2    stringr_0.6.2  tools_3.1.1

The exact error message is:

Error in filter_impl(.data, dots(...), environment()) : 
  object 'MeanOutcome' not found

[hadley]

Hmmm, putting something complicated on the LHS seems rather unnatural to me - it's like doing 10 > x.

@romainfrancois how hard would this be to fix?

[romainfrancois]

ah. Probably not that hard, it is another case of me not doing the right thing.

[romainfrancois]

Ok so the problem was that when analysing this call analysis_opts$min_outcome > MeanOutcome, we were replacing analysis_opts$min_outcome correctly but then stopped there, instead of dealing with the rest of the expression.

So definitely a case of me not doing the right thing.