Ignore grouping when arrange()ing

[ajdamico]

set.seed( 2014 )

library(dplyr)
library(nycflights13)


x <- flights
x[ sample( seq( nrow( x ) ) , 300000 ) , 'origin' ] <- NA
x[ sample( seq( nrow( x ) ) , 300000 ) , 'dest' ] <- NA


# arrange( desc( num ) ) fails

x %>%
    group_by( dest , origin ) %>%
        summarize( 
            num = n() ,  
            something = n_distinct( arr_time ) ,
            another = round( 100 * mean( is.na( arr_time ) ) , 1 ) ,
            yet = round( 100 *mean( arr_time , na.rm = TRUE ) , 1 )
        ) %>%
            arrange( desc( num ) )

[jmarywien]

I think what you are expecting is for num to be arranged in descending order, regardless of flight destination and origin.

However, group_by returns a grouped_df object, so any subsequent arranges will be applied within group according to https://github.com/hadley/dplyr/blob/master/src/arrange.cpp#L13.

If you want to arrange in descending order, regardless of flight and origin, you must cast the grouped_df to data_frame using

    x %>% data.frame %>% arrange(desc(num))

[ajdamico]

you're absolutely right, thanks! and shame on me for missing this sentence from http://cran.rstudio.com/web/packages/dplyr/vignettes/introduction.html

grouped arrange() orders first by grouping variables

i've proposed a change to the ?arrange help page.

master...ajdamico:patch-1

[hadley]

I'm reopening this because I think it's rather confusing, and I'm considering changing back to the old behaviour where arrange ignores groups.

[jmarywien]

I think that's a good idea Hadley. Have a good weekend.
On Sat, Jul 4, 2015 at 2:48 AM Hadley Wickham notifications@github.com
wrote:

I'm reopening this because I think it's rather confusing, and I'm
considering changing back to the old behaviour where arrange ignores groups.

—
Reply to this email directly or view it on GitHub
#1206 (comment).

[stevencb]

To be honest, I think it makes sense that arrange would sort within groups (i.e. I don't think it should be changed back). I seems to make sense that if you apply something like arrange to a grouped data frame, that it acts within groups, not globally. One can of course ungroup to get the other behavior. I actually find the current behavior more intuitive.

If you do change it back, can you provide an option to arrange just within groups ? I have found this useful.

[stevencb]

FYI,
An alternative to

x %>% data.frame %>% arrange(desc(num))

is

x %>% ungroup %>% arrange(desc(num))

Using "ungroup" will maintain the class types of x. So if x is a tbl_df, it will still be after ungroup but not after piping to data.frame.

[hadley]

It's not more intuitive to most people

[kevinushey]

IMO there should be separate functions for sorting within groups vs sorting without groups.

[nicholasjhorton]

Would a separate option (defaulting to sorting without groups) be feasibe to consider?

Nick

On Jul 4, 2015, at 8:37 PM, Kevin Ushey notifications@github.com wrote:

IMO there should be separate functions for sorting within groups vs sorting without groups.

—
Reply to this email directly or view it on GitHub.

Nicholas Horton
Professor of Statistics
Department of Mathematics and Statistics, Amherst College
Box 2239, 31 Quadrangle Dr
Amherst, MA 01002-5000
https://www.amherst.edu/people/facstaff/nhorton

[ajdamico]

perhaps something like arrange and arrange_by -- arrange could be equivalent to x %>% ungroup %>% arrange_by and then arrange_by behaves groupwise like the current arrange function does. since both could be on the same help page, typing ?arrange would make it immediately obvious if a user wants the _by

[hadley]

I don't think adding more options or function variants will help here.

[AMarthaller]

This bit me today. How about just preserving an arrange if it's called before group_by? Then you can support both with the same existing method. Order of operations would control the behavior.

[AMarthaller]

Of course my previous suggestion would not help with the ordering of newly created variables. It is also important to maintain an option for sorting within groups as row_number() queries often leverage specific ordering.