Allow DF() as a selector in select and combine

[bkamins]

Allow in selector => fun => outcol for selector to be a DF() in which case an AbstractDataFrame would be passed to fun.

Alternatively we could just allow fun => outcol and fun (as we currently do in combine with the fun form).

To be discussed what is better. Using DF() is explicit. Just using fun is shorter and it is not ambiguous so both have pros and cons.

CC @pdeffebach @matthieugomez

[matthieugomez]

I think DF() is safer. Otherwise, in the macro-based DataFrames, syntax such as transform!(df, mean(:x)) may end up being ambiguous. If it ends up being too cumbersome, it can always be removed later.

[nalimilan]

I'm not totally convinced DF() => fun => outcol is better than just passing fun => outcol. I don't see why you would like to use it in DataFramesMeta (since it detects which columns you want automatically, that's the whole point), so the ambiguity problem isn't clear to me.

[bkamins]

So just to give my preference I also prefer fun => outcol.

[pdeffebach]

We don't really want people to use the fun => outcol method, though, right? Its less performant. Is there an argument for not allowing less performant, but easier to type, syntax? Or is that user-hostile?

[bkamins]

Its less performant

If it is slower or not depends on the use case, but often it is faster (even if we exclude compulation cost), e.g.:

julia> df = DataFrame(rand(10^6, 2));

julia> gdf = groupby(df, :x1);

julia> @time combine((x -> (n=nrow(x),)), gdf);
  0.394009 seconds (7.20 M allocations: 261.568 MiB, 5.49% gc time)

julia> @time combine(AsTable(:) => (x -> (n=length(x[1]),)), gdf);
  0.960185 seconds (17.69 M allocations: 522.495 MiB, 20.14% gc time)

The more columns you have in your df the more expensive it becomes not to use SubDataFrames:

julia> df = DataFrame(rand(10^6, 100));

julia> gdf = groupby(df, :x1);

julia> @time combine((x -> (n=nrow(x),)), gdf);
  0.404221 seconds (7.20 M allocations: 261.569 MiB)

julia> @time combine(AsTable(:) => (x -> (n=length(x[1]),)), gdf);
 11.342024 seconds (313.67 M allocations: 10.886 GiB, 11.85% gc time)

[bkamins]

When #2410 will be implemented this should be closed.

[bkamins]

I would close it. The operation specification syntax is already complex enough. AsTable does almost the same (except that it produces a NamedTuple), so the only limitation is for very wide tables. But then user can just pass a function and then a data frame is passed to it and user can name output columns inside this function.

If you feel this issue should be re-opened please comment.