Nonstandard aesthetics

[clauswilke]

I realized yesterday that with very little work I could address most of the issues I raised in #2345. So I would like to propose this pull request to substantially improve ggplot2's handling of nonstandard aesthetics. The pull request consists primarily of removing hardcoded aesthetics names in various places, and a few minor adjustments to handle the fallout. All the regression tests pass, and none of the changes should impact anybody who is using ggplot2 as they always have. Below follow some examples of what this code can do.

First, we can now specify the aesthetics independently of the scale name,
e.g., define the fill scale using scale_color_brewer():

library(ggplot2)
ggplot(iris, aes(x = Sepal.Length, fill = Species)) +
  geom_density(alpha = 0.7) +
  scale_colour_brewer(type = "qual", aesthetics = "fill")

Second, we can specify multiple aesthetics at once. The legends get merged if appropriate:

ggplot(iris, aes(x = Species, y = Sepal.Length, color = Species, fill = Species)) +
  geom_boxplot(alpha = 0.2) +
  geom_point(position = "jitter") +
  scale_colour_hue(c = 50, l = 40, aesthetics = c("colour", "fill"))

This also works if the aesthetics represent different data columns:

ggplot(iris, aes(x = Sepal.Length, y = Petal.Length, color = Sepal.Length, fill = Petal.Length)) +
  geom_point(shape = 21, size = 3, stroke = 2) +
  scale_colour_viridis_c(name = "Length",
                         option = "B", aesthetics = c("colour", "fill"))

The limits get merged automatically, as can be seen here more clearly:

df <- data.frame(x = c(1, 2, 3),
                 y = c(6, 5, 7))

ggplot(df, aes(x, y, color = x, fill = y)) +
  geom_point(shape = 21, size = 3, stroke = 2) +
  scale_colour_viridis_c(name = "value",
                         option = "B", aesthetics = c("colour", "fill"))

All of this also works with non-standard aesthetics, which I have implemented in ggridges.
First an example with guide_legend():

library(ggridges)
ggplot(iris, aes(x = Sepal.Length, y = Species, fill = Species, point_color = Species)) +
  geom_density_ridges(jittered_points = TRUE, position = "raincloud", alpha = .4) +
  scale_color_manual(values = c("#924658", "#366A17", "#126490"),
                     aesthetics = c("fill", "point_color"))

And now an example with guide_colorbar():

ggplot(iris, aes(x = Sepal.Length, y = Species, fill = ..x.., point_fill = Sepal.Length)) +
  geom_density_ridges_gradient(jittered_points = TRUE, position = "raincloud",
                               point_color = "black", point_shape = 21, scale = 1.1) +
  scale_color_viridis_c(
    option = "A",
    aesthetics = c("fill", "point_fill"),
    guide = guide_colorbar(available_aes = c("fill", "point_fill")),
    name = "Sepal Length"
  )

[clauswilke]

Two additional comments:

• I didn't add any regression tests, visual or otherwise. I'm happy to do so if otherwise this patch looks acceptable.

• There is one breaking change, in that guide_train() now requires an additional argument. This should only be a problem for code that defines custom guides, and I'm not aware of any code that does that. It's certainly not something people commonly do.

[thomasp85]

I do in ggraph but that shouldn’t weight against this PR

[clauswilke]

@thomasp85 Also, the fix takes a minute. You simply need to use the provided aesthetic instead of the first one from scale$aesthetics: https://github.com/clauswilke/ggplot2/blob/3841ddaee3d549e15df520cf59c89cbcc4565770/R/guide-legend.r#L225
I needed to do this because we also want to build guides for the second or third aesthetic listed for a scale.

Would you mind installing this PR and testing if it causes any problems with ggraph?

[thomasp85]

sure, but not before next week. I’ll get back to you

[hadley]

I really dislike scale_colour_brewer(type = "qual", aesthetics = "fill") because we have an argument value that conflicts with the function name. The implementation seems simple enough to be worth doing (I haven't deeply reviewed yet, but it seems reasonable), but I don't like the weirdness this introduces into the API. What could we do to avoid it?

[thomasp85]

Maybe have an aesthetic-less scale available where the aesthetic can be set in the function call, so you’d do:

p + scale_brewer(..., aes = c(“fill”, “colour”))

It would require a huge amount of new exported functions, but api wise I think it is the most pretty approach

[clauswilke]

I previously commented on the naming issue here: #2345 (comment)

Words such as "color", "shape", "size" etc serve two purposes in the current ggplot2. On the one hand, they identify the aesthetic. On the other hand, they indicate the type of scale (color, shape, size, etc). If we only focus on the latter for a moment, it's clear that scale_colour_* is a color scale, scale_size_* is a size scale, scale_shape_* is a shape scale, etc. And I believe this is on some level the more intuitive perspective, given how often people on stackoverflow try to use scale_colour_* to style a fill aesthetic. And also, as I said in my earlier comment, the generic version of scale_shape() would be scale(), which doesn't make sense.

For these reasons, I'm very comfortable with scale_colour_brewer(..., aesthetics = "fill"). If ggplot2 had been written like this from the beginning, nobody would have batted an eye.

[clauswilke]

Another way to think about it (which I'm proposing more as a thought experiment): Remove scale_fill_*(), and you'll end up with a logical and consistent interface. scale_colour_*() / scale_fill_*() is the only scenario in standard ggplot2 where you have two scales that do the same thing (map to color) but have different names because they handle different aesthetics.

[hadley]

Ok, that seems like a compelling argument.

We could also (eventually) consider making scale_colour_blah() have aesthetic = c("colour", "fill") — that would require more effort if you wanted to map fill and colour aesthetics to different scales, but that seems totally reasonable.

[clauswilke]

I have rebased and added a few regression tests.

[hadley]

Can you have a look at why travis is failing please?

[clauswilke]

Not sure what the problem is. CRAN check works for me. I don't seem to be able to see the travis build log for this repository, and that makes figuring these things out more difficult. The one thing I noticed was an out-of-date .Rd file, so maybe that was it. I have committed it now.

[karawoo]

It looks like the Travis build timed out during installation of the dependencies.

[clauswilke]

Either way it succeeded now. And I finally figured out where I can see the build log, so I'll be more informed in the future.

[hadley]

I think if we back off the user facing changes just slightly this is good to merge.

[hadley]

For this version, maybe we could just add that argument to colour scales? You'll still be able to set it via ... but it won't be so in your face.

[clauswilke]

The problem is that you can't set the aesthetics via ..., that's what brought me down this rabbit hole in the first place. The reason is that aesthetics is hard-coded as an argument to discrete_scale() or continuous_scale() in all scale functions.

As an example, let's look at scale_colour_brewer(), which is currently defined as follows:

scale_colour_brewer <- function(..., type = "seq", palette = 1, direction = 1) {
  discrete_scale("colour", "brewer", brewer_pal(type, palette, direction), ...)
}

If we try to call it with a custom aesthetic, the following happens:

ggplot(iris, aes(Sepal.Length, fill = Species)) + 
  geom_density() + 
  scale_colour_brewer(aesthetics = "fill")
#  Error in self$palette(n) : attempt to apply non-function 

Since we provide an aesthetics argument, it is handed off to discrete_scale() via ..., but now all the other arguments are shifted by one and so the palette function gets assigned to the wrong argument.

Here is a very simplified example to demonstrate the issue:

f <- function(a, b, c) {
  print(a)
  print(b)
  print(c)
}

g <- function(...) {
  f(4, 5, ...)
}

g(1)
# [1] 4
# [1] 5
# [1] 1

g(a = 1)
# [1] 1
# [1] 4
# [1] 5

I could process the ellipsis with match.call() and then call the lower-level scale function with do.call(), but that would be a lot of extra code in every single scale function, and it'd be not very readable. I can ponder some more whether there's some other way to make this happen, but I haven't come up with one yet.

Is there a way to test whether the ellipsis contains a specific argument, without actually evaluating the ellipsis? Maybe some rlang feature I'm not aware of?

[hadley]

Ugh, of course, blech. What we just leave off the non-colour aesthetics for now and consider the user API in the future? I'd like to get this into this release, but I don't have the time to think through any big API changes.

[clauswilke]

In that case, how do you feel about exporting manual_scale() (possibly under a different name, e.g. scale_discrete_manual())?

ggplot2/R/scale-manual.r

Line 74 in 0dab044

manual_scale <- function(aesthetic, values, ...) {

The problem is that there is currently literally no simple way of defining, for example, a discrete shape scale for an aesthetic other than shape. One always has to reimplement manual_scale(), and it's the reason I invented scale_discrete_manual() in ggridges:
https://github.com/clauswilke/ggridges/blob/4e94e9a32b976c5a2eedc6fa04b104178f3e9767/R/scale-.R#L26

[hadley]

Yeah, that sounds good, and I think that name is fine.

[clauswilke]

Ok, to summarize, I will:

1. Revert addition of aesthetics argument for non-color scales.
2. Add and export scale_discrete_manual() which works just like manual_scale().

I'll try to get this done by Friday.

[clauswilke]

I have made these changes now. The aesthetics argument remains only for scales that set a color. Throughout the documentation, I have explained this as an option for setting colour and fill aesthetics at the same time, so it shouldn't be too confusing.

I also added three new generic scales that I think are needed, scale_discrete_manual(), scale_discrete_identity(), and scale_continuous_identity().

[hadley]

Thanks!

[lock]

This old issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with reprex) and link to this issue. https://reprex.tidyverse.org/