scale_binned

[thomasp85]

This PR is created in advance to allow for discussion of a potential new fundamental scale (that is, something at the same level as continuous and discrete). The purpose of scale_binned is to discretise continuous data for use with discrete palettes (e.g. shape or brewer colours), or with geoms expecting discrete positions (e.g. geom_bar())...

The main advantage of this setup is that it allows the guides to automatically place ticks at the border between bins, rather than at the middle of each bin - something that has been requested before (e.g. #2673)

Currently the scale work for positions, but the main holdback for the other aesthetics is the creation of a new type of legend that supports the desired output.

ggplot(mtcars) + 
  geom_bar(aes(mpg)) + 
  scale_x_binned()

as can be seen, the ticks are between the bars which provide a much more intuitive understanding of the range of each bin compared to using geom_histogram() where the scale remain continuous and thus has tick marks at "nice" locations from a numerical point of view... For colour scales as desired in #2673 this effect is also of great use.

Open questions

• what should the naming be for the shorthand scales for colour? scale_colour_brewer() etc are already taken (for good reasons) — is there a pattern that can be applied to all? The most graceful thing would be to have discrete scales that are used on continuous data be binned automatically, but I don't think this is doable (and the magic might be limiting in other ways)
• should the outer ticks be shown (e.g. the limits of the data)?
• can we support non-linear binning gracefully (e.g. have a bin twice as wide as the others and have the bar expand to that size automatically)
• Currently the behaviour is to default to 10 bins - is this a reasonable choice or is there anything better?
• whatever questions and suggestions you have @hadley, @clauswilke, @karawoo, @yutannihilation

[yutannihilation]

what should the naming be for the shorthand scales for colour?

Do you think we can implement this as such options of scale_colour_continuous() as bins and binwidth, so that the developers of color scale packages don't need to implement their own binned version of the scale by themselves? (The shorthand names are still useful, though).

[clauswilke]

@yutannihilation I think other parameters would have to be changed, such as the default guide, so having a separate scale function may be needed after all.

If we go with the naming for the viridis scale, scale_color_viridis_c() for continuous and scale_color_viridis_d() for discrete would suggest scale_color_viridis_b() for binned.

Also, is it time to overhaul the guide system and port it to ggproto? It's the only part of ggplot2 that hasn't been ported over, and the code is difficult to maintain and extend. Lots of code duplication between guide_legend() and guide_colorbar().

[yutannihilation]

I think other parameters would have to be changed, such as the default guide, so having a separate scale function may be needed after all.

Hmm..., agreed. Thanks.

[thomasp85]

I've been eying the guide internals for years... I might rewrite them as part of making the new bin guide...

[ptoche]

Currently the behaviour is to default to 10 bins - is this a reasonable choice or is there anything better?

A popular rule of thumb is Sturges's rule, i.e. round base-2 log of sample size plus 1. Other rules are listed here. The Freedman-Diaconis rule may be worth looking into. But for simplicity, Sturges's rule or 10 bins can hardly be beat.

[thomasp85]

I've gotten the scale to work nicely with uneven bin sizes (point 3 above):

df <- data.frame(x = rnorm(1e4))
ggplot(df) + 
  geom_bar(aes(x)) + 
  scale_x_binned(breaks = c(-2, -1, 1, 2))

limits will control the scale range of the bins. Default oob behaviour is squish() which means that values outside of the limits will be part of the terminal bins. This, I think, reflect how most people will use bins as it allows to collect the tails of the data into the last bins

ggplot(df) + 
  geom_bar(aes(x)) + 
  scale_x_binned(limits = c(-1.5, 1.5))

as can be seen, there is still some weird interplay between the back transformation and the expansion - I'll look into that of course

[hadley]

@ptoche I don't think Sturge's rule is not appropriate because it is motivated by minimising some mathematical quantity that is not directly related to needs of a visualisation. I'd recommend using 30 bins to match geom_histogram().

[thomasp85]

Hmm... I think 30 may be too much as a default as it would clutter the axes with labels

[thomasp85]

Case in point

[hadley]

Hmmmm, in that case maybe it's better to behave like the default labels for continuous legends? i.e. pick n (5? 10?) nicely spaced numbers

[thomasp85]

I'll look into that... that would give some sort of consistency

[thomasp85]

hmm... it's a shame that the current trans implementation doesn't allow for modifying the number of breaks after creation

a n_breaks argument would make sense for both binned and continuous scales...

[thomasp85]

We could set n in the breaks() method environment from ggplot2 in order to allow this, but it feels kinda hacky...

[clauswilke]

Just a minor comment: I think it makes sense to have a default bin number of 30 for position scales, but you'd want a default of on the order of five to seven for color scales. And odd numbers have the advantage that they contain the midpoint for diverging scales. Ideally we'd have different defaults for position and color.

[clauswilke]

Also, the scale could simply produce empty labels by default if it judges that there are too many bin boundaries. E.g., if there are 30 bin boundaries, leave 20 of the label strings empty.

[thomasp85]

I though about this as well... essentially making some of the breaks into minor breaks...

problem is if the user chose to have breaks of uneven length and the scale then choose to remove some of the labels - there is no way of deducing where the cut is made in that case...

[thomasp85]

Using the same break algorithm as continuous scales:

ggplot(df) + geom_bar(aes(x)) + scale_x_binned()

Support for an n_breaks argument is build in:

ggplot(df) + geom_bar(aes(x)) + scale_x_binned(n_breaks = 10)

[thomasp85]

I think the colour scales should have a default n_breaks matching the number of discrete colours in the palette or something

[thomasp85]

Does it make sense to have a binned guide for anything but colour/fill (same constraints as colourbar)?

On one hand binning allows continuous data to be used with aesthetics that only have discrete representation (e.g. shape), but on the other hand I'm not sure when it will ever be a good idea to use something without an inherent order to symbolise different bins.

For something like size I cannot figure out why you would want it binned in the first place instead of a direct mapping...

[hadley]

I think it makes sense for size for the same reason it does for colour: it’s almost certainly easier to perceive differences with a small number of categories.

[thomasp85]

that is a fair point...

[thomasp85]

I don't think I want automatic arrows TBH

[adrfantini]

Maybe ends = 'auto'? Either way I'm fine, looks great!

[thomasp85]

arrow() is a grid function so I can’t really change that...

[clauswilke]

You could also draw a column/row of rectangles, as in @adrfantini's example, and then instead of filling the rectangles with color you place the discrete symbols inside each rectangle.

Something like this.

library(ggplot2)

df <-data.frame(x = 1:5)

ggplot(df, aes(x = x, y = 1)) +
  geom_tile(color = "black", fill = NA, size = 1.5) +
  geom_point(aes(color = x), size = 5) +
  guides(color = "none") +
  ylim(-2, 3) +
  theme_void()

^{Created on 2019-09-24 by the reprex package (v0.3.0)}

[thomasp85]

I don’t really like that look though — too heavy and a strong depart from the look of guide_legend()

[thomasp85]

If you want that look you can probably get it by turning off the axis and giving the key background a border. I’ll have to check though

[thomasp85]

@hadley I consider this done now. Open questions are the two unresolved comments. Can I get you to chime in if you consider them answered and then finish off the review?

[lock]

This old issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with reprex) and link to this issue. https://reprex.tidyverse.org/