Replace "jet" as the default colormap #875

Closed
endolith opened this Issue May 17, 2012 · 68 comments

Comments

Projects
None yet
Contributor

endolith commented May 17, 2012

This is subjective: a matter of sane defaults vs consistency with matlab, but here are the arguments against jet as default:

Rainbow Color Map (Still) Considered Harmful:

Research has shown that the rainbow color map is rarely the optimal choice when displaying data with a pseudocolor map. The rainbow color map confuses viewers through its lack of perceptual ordering, obscures data through its uncontrolled luminance variation, and actively misleads interpretation through the introduction of non-data-dependent gradients.

Despite much published research on its deficiencies the rainbow color map is prevalent in the visualization community. We present survey results showing that the rainbow color map continues to appear in more than half of the relevant papers in IEEE Visualization Conference proceedings; for example, it appeared on 61 pages in 2005. Its use is encouraged by its selection as the default color map used in most visualization toolkits that we inspected. The visualization community must do better.
...
In the absence of feedback about the data or task, the best approach for situations where color is the only display technique is probably the black-body radiation spectrum, because of its perceptual ordering and use of color to avoid contrast effects.

So this would presumably mean using one of hot, afmhot, or gist_heat as the default colormap:

hot, afmhot, and gist_heat

How NOT to Lie with Visualization:

This rainbow hue colormap is widely used in visualization, but produces several well-documented artifacts (e.g., Lefkowitz and Herman [1992]; Robertson [1988]; Rogowitz, Ling and Kellogg [1992]). In this MRI image, for example, the colormap creates perceived contours which do not reflect discrete transitions in the data, structures in the data which fall within one of these artificial bands are not represented, and attention is drawn to the yellow areas because they are the brightest, not because they are in any way the most important.

Let’s talk colormaps:

What kind of insane colormap has the property that values spanning the extreme ends of the scale stand out less, and can’t be distinguished as easily as values in the noisy middle? Why, it’s MATLAB’s default colormap, of course!

The 'jet' colormap must die!:

The main issue here is that in grayscale representation, information about the 'height' of map regions gets lost. The center of the plot (where the highest intensity is found) has the same gray value as the border regions.

Contributor

cgohlke commented May 17, 2012

There have been two related discussions on the [matplotlib-devel] mailing list: Another colormap and Implementation of cubehelix color scheme. Btw, the hot/heat colormaps perform poorly with 3D shading.

Contributor

endolith commented May 17, 2012

Btw, the hot/heat colormaps perform poorly with 3D shading.

In what way? Here's jet compared to hot:

jet bump
hot bump

Contributor

endolith commented May 17, 2012

That seems to be an objection against using it for 3D CAD? Can you even do things like that in matplotlib? I don't see anything like it in the gallery.

I agree with this comment from the mailing list:

I'd prefer a two-tone colormap as the default (two-distinct tones at the limits with a gradient in-between---dubbed "sequential" in the paper) instead of a three-tone colormap (three-distinct tones---dubbed "diverging" in the paper). (I think this is a more common use case, and I think using a "diverging" colormap effectively requires setting vmin/vmax.)

The coolwarm is meant for "diverging" data, with dark blue negative values, bright white middle 0 values, and dark red positive values (same dark-light-dark problem as jet, but not as bad), while the more common case is "sequential" data from dark zero to bright positive (which is why the first paper above recommends heat/hot when nothing else is known about the use case).

But obviously, nothing will be perfect for every case. Different maps are more appropriate for different situations. Unless there's a way to set different defaults for different types of plots, we just have to pick one that will work better than jet in the common types of plots where people don't change the defaults.

Member

ivanov commented May 17, 2012

I'm +1 for changing colormap to something more sane than jet. Thanks to everyone for providing links to the relevant resources.

We probably have examples in the test suite that rely on the reference images being rendered in 'jet' - those reference images would have to either be changed, or the default cm switched to jet for testing, and back to the default again post testing.

It should be noted that the default colormap can currently be set via an rcParam. In your matplotlibrc just put:

image.cmap   : hot 

or

image.cmap   : bwr

etc...

If there are places in the code where this cmap setting is not respected (except for instances where it is explicitly specified), those should be considered bugs and should be filed as such.

Owner

efiring commented May 27, 2012

I don't see any polite way to make such a change (different default colormap). Everyone is used to the nice, colorful jet popping up. I predict that if it is changed, there will be a lot of unhappy users. User code will not behave the way it used to, in a very obvious and major way. Instead of changing the default (at least initially), it would be better to change the gallery examples and documentation to explain and illustrate better colormap choices for various purposes. This can be done incrementally, without breaking any user code.

Member

WeatherGod commented May 27, 2012

@efiring, I think that is an excellent idea. We might one day change the default, but this approach might begin the movement from within the userbase. Over time, we might even be able to figure out what a suitable default is.

Contributor

endolith commented May 31, 2012

I still think the default should be changed sooner than later, and that the user who doesn't care enough to change the default isn't going to care if the default changes, but improving the demos is a good idea, too.

It looks like it's not as simple as changing the cmap alone, though, since some have bipolar data and want a diverging colormap, but the colormap isn't currently centered. The seismic colormap highlights the midpoint well. I used this and it worked:

vmax = max( amax(Z), -amin(Z)), 
vmin = min(-amax(Z),  amin(Z)),

Is there a better way to center the colormap?

off-center
centered

Owner

efiring commented May 31, 2012

It is actually the norm, not the colormap, that is being centered. You raise a good question. There should be an automatic way (selected by a kwarg) to achieve what you are doing manually, but in general there isn't. I am working on a branch to take care of it, so I expect to have a pull request ready within a few days.

Contributor

endolith commented Jun 4, 2012

Ok, so actually instead of setting vmax vmin, I should be using something like norm=Normalize(-abs(Z1).max(), abs(Z1).max())?

What will be the syntax for your addition? If the data happens to vary from 4 to 10, and the defined midpoint is 5, we should be able to just say "the midpoint of the colormap is 5" and it will automatically use the extreme values to set the colormap range from 0 to 10.

Contributor

endolith commented Jun 4, 2012

These are some changes in the branch where I'm changing the example colormaps:https://github.com/endolith/matplotlib/compare/c6fac36ed3c5f14c2a9863d9c68f6a8fa068fcbc (Not sure how else to link to a branch)

Owner

efiring commented Jun 4, 2012

On 06/04/2012 04:46 AM, endolith wrote:

Ok, so actually instead of setting vmax vmin, I should be using
something like norm=Normalize(-abs(Z1).max(), abs(Z1).max())?

Not necessarily; when vmin, vmax kwargs are available, they are being
passed internally to the norm, so there is no problem with using them as
you did. I was only trying to clarify the terminology, because this is
a somewhat subtle but important point--the distinction between the
actual colormap (cmap kwarg) and the norm, and the fact that they work
together to achieve color mapping.

What will be the syntax for your addition? If the data happens to
vary from 4 to 10, and the defined midpoint is 5, we should be able
to just say "the midpoint of the colormap is 5" and it will
automatically use the extreme values to set the colormap range from 0
to 10.

Yes, that is exactly what I have in mind. I have a branch started to do
this, but it is not ready yet. I am thinking of a kwarg called "vmid"
to go with "vmin" and "vmax", the idea being that if it is not None then
it expands any specified or autoscaled range in the manner you describe.

Contributor

endolith commented Jun 5, 2012

That sounds like a good keyword to me.

If you do include vmid and vmax, then vmin is vmid-(vmax-vmid)?

if you include all three, the scale is non-linear, normalized independently from the midpoint to the max point and from the midpoint to the min point? http://stackoverflow.com/q/7404116/125507

Owner

efiring commented Jun 5, 2012

On 06/05/2012 05:18 AM, endolith wrote:

That sounds like a good keyword to me.

If you do include vmid and vmax, then vmin is vmid-(vmax-vmid)?

if you include all three, the scale is non-linear, normalized independently from the midpoint to the max point and from the midpoint to the min point?

No to both. It's much simpler than that. Think of it this way. First,
if either or both of vmin, vmax is provided, then they override the
previous values, which may have resulted from autoscaling. Second, the
upper or lower limit is moved outward as needed to center the scale on
vmid. That's all.

Member

WeatherGod commented Jun 6, 2012

Just thinking out loud, I remember being able to specify just vmin or just vmax and let the autoscaling handle the unspecified end. Is that still the case?

Owner

efiring commented Jun 6, 2012

On 06/06/2012 08:37 AM, Benjamin Root wrote:

Just thinking out loud, I remember being able to specify just vmin or
just vmax and let the autoscaling handle the unspecified end. Is
that still the case?

Yes.

Member

pelson commented Jun 13, 2012

Just wanted to note here that I have a SymmetricNorm class which auto-scales evenly around a centre point.
It sounds like it could do the trick (and potentially avoid mutually exclusive keywords) and I'd be happy to share.

Contributor

endolith commented Jun 18, 2012

Maybe the symmetric norm stuff should be added under #632?

Owner

efiring commented Aug 13, 2012

Closed after #921 merge

efiring closed this Aug 13, 2012

Contributor

endolith commented Aug 13, 2012

(I'm still in favor of changing the default, for the record.) :)

Owner

tacaswell commented Mar 10, 2013

I (very belatedly) add a vote for changing the default color map.

FYI, "jet" won't be the default colormap for MATLAB anymore. The new release has finally improved this and other related aspects. http://www.mathworks.es/products/matlab/matlab-graphics/#new_look_for_matlab_graphics

Contributor

endolith commented Oct 13, 2014

There is also a new default colormap in R2014b called parula [named after a similarly-colored bird?]. The colors in the parula colormap are ordered from dark to light and are perceptually uniform. Smooth changes in the data appear as smooth changes in color, while sharp changes in the data appear as sharp changes in color.

colormap_parula

Well, at least someone is doing it. Hopefully open source projects copy this now instead of copying jet?

Owner

tacaswell commented Oct 13, 2014

Regardless, changing the default color map is still an api break which we won't do until 2.0.

As this point was a bit lost in the discussion if to break the API, which of the MPL's colormaps would be considered as the best alternative to jet these days? Is there something like the Kindleman set available? Do some of the new stylesheets already improve this?

Contributor

NelleV commented Oct 14, 2014

As this point was a bit lost in the discussion if to break the API,
which of the MPL's colormaps would be considered as the best
alternative these days?

Probably gray_r or gray


Reply to this email directly or view it on GitHub
#875 (comment)
.

Owner

tacaswell commented Oct 14, 2014

👍 on gray.

The style module does take care of this in principle via the image.cmap rcparam. mpl.style.use() can look up arbitrary paths, including URLs. A trick from @danielballan is to put a stlye-file in a gist: https://gist.github.com/danielballan/be066529de85e87a5fe7

Contributor

endolith commented Oct 15, 2014

could the default be different for different types of plots? bone for 2D heatmaps like imshow(), coolwarm or rainbow for 3D surfaces like plot_surface(), etc.?

Also see prettyplotlib, which tries to guess whether your data is diverging or sequential and chooses a colormap accordingly, either RdBu, Reds or Blues.

Owner

efiring commented Oct 15, 2014

@NelleV I very much doubt that a large proportion of mpl users would be happy to see a gray colormap as the default. Jet is not ideal, but I don't think that giving up on color entirely, or going to a single hue, is better.
@endolith I don't think it would be wise to put that level of complexity in the mpl core. PEP 20: "In the face of ambiguity, refuse the temptation to guess." No amount of logic and heuristics will be able to tell whether I am contouring velocity, for which a diverging red-blue color scale works well, or oxygen, for which a sequential scale might be more appropriate. Even if I am contouring temperature, for which a purist might say I should use a sequential map, I might legitimately choose a diverging map so as to have more visual range and resolution, and to give a clear sense of "warm" versus "cold".
If a new default is chosen, it should be chosen to be visually pleasing to a majority of people, and to be a tolerable compromise among all the competing criteria for a range of applications; it needs to work for new users and for quick interactive work. It looks like the Mathworks made a nice choice.

Contributor

endolith commented Oct 15, 2014

@efiring by "complexity" do you mean the auto-guessing, or the different defaults for different functions?

Spectral_r is similar to jet, but without the bright bands. It's sort of diverging (symmetrical brightness) and sequential (color) at the same time.

7298887108_d8978791d0_o

Matlab's choice looks good too

@efiring Please note, I believe @NelleV only answered to my question what's the most recommended colormap these days in general, independently of the question if or when to break the current standard. I tried to make that clear in the way I formulated the question, but maybe this would have been better on the mailing list or on another issue.

@endolith I'm actually surprised that above's rainbow is relatively smooth compared to other rainbow cmaps I've seen in some articles, that over-emphasized green much more than here. Was it already designed with avoiding banding or non-iso-luminosity as good as possible?

Owner

efiring commented Oct 15, 2014

On 2014/10/14, 5:28 PM, endolith wrote:

@efiring https://github.com/efiring by "complexity" do you mean the
auto-guessing, or the different defaults for different functions?

Both. The guessing idea is the worst, but I really don't see any sense
in different defaults for different functions, either--it's still a form
of guessing. Any given function involving a ScalarMappable can be used
in a wide variety of applications, with a corresponding variety of
suitable color maps.

Owner

efiring commented Oct 15, 2014

Does anyone know how to figure out whether Matlab's perula is their IP? I suspect it is; it probably exists only in the form of their m-file (or compiled in the core or in a mexfile). It's a bit like our YlGnBu, but it has a yellower yellow end. Overall, it seems to take advantage of more hues.

njsmith commented Oct 21, 2014

The paper paper describing coolwarm (another strong contender I think -- it was carefully designed to be a good default colormap) is here. It finesses the diverging/sequential issue in an interesting way -- it goes from blue to red, but through grey instead of through white, so the center is visible but not emphasised. The paper also has extensive (and I would argue, completely compelling) examples demonstrating how much better it is than jet.

The blog posts about parula also provide lots of extremely compelling examples -- including what sounds like a senior MathWorks engineer who misinterprets jet when tested, and a user who had gone who-knows-how-long having no idea how noisy their data was because jet was hiding the noise.

@efiring:

I don't see any polite way to make such a change (different default colormap). Everyone is used to the nice, colorful jet popping up. I predict that if it is changed, there will be a lot of unhappy users.

I guess watching the reaction to Matlab's change will be informative :-). The coolwarm paper also gives some empirical evidence here: they switched the default in their package, and over the next 10 months got exactly user zero complaints.

@tacaswell:

Regardless, changing the default color map is still an api break which we won't do until 2.0.

This may sound strong, but honestly, I think this way of thinking about releases will be very harmful to the project in the long run. Big-bang everything-changes releases are basically impossible to get right (because getting things right requires iteration), and are actually harder for users to adapt to then small incremental changes. And bunching changes up like this just has the effect of delaying important changes like this one until lots of totally unrelated work gets done.

Thought question: Why not call the next release 2.0, with the only "API breaks" being better visual defaults? All code would keep working so the transition would be super easy... I know this does not match the 20th century idea of a "major release", but what, concretely, is the advantage of sticking to that idea of "major releases"?

(Also this is a very very weak sort of API break, given that all unmodified code will continue to be 100% functional; if anyone wants to stick to jet then they can set that explicitly, but they can do this at their leisure because in the mean time their software will still be usable.)

Owner

tacaswell commented Oct 21, 2014

@njsmith I disagree this is a minor API change, drastically changing the appearance of a plot is about as big of an API change as mpl can have.

You seem to want to have it both ways, arguing that changing the default color map is important and we need to do it right now and that it's no big deal as users adversely affected can always explicitly fix their code.

Member

WeatherGod commented Oct 21, 2014

Indeed, to give a (non-mpl) example that I experienced recently. I produce
customized shapefile products for customers. Due to a long-standing issue,
one of the columns in the shapefile's dbf file is one character too short
in some cases. This particular customer hadn't cared that the end of the
value was truncated for years. We offered to fix it, but they declined and
we had forgotten about that detail.

Recently, we refreshed our processing system with new library components
(particularly GDAL) to support new features. Everything seemed ok, all
tests were passing on our end. However, it appears that this new version of
GDAL would ignore the explicitly set column widths whenever a value didn't
fit and silently make them wider. Nobody noticed this until it hit
production. This broke the customer's data ingest into their database. We
ended up in hot water over that.

From GDAL's perspective, they must have thought that they were being
helpful and improving their product. But from my perspective, they broke a
perfectly valid behavior. One man's feature is another man's bug.

On Tue, Oct 21, 2014 at 1:44 PM, Thomas A Caswell notifications@github.com
wrote:

@njsmith https://github.com/njsmith I disagree this is a minor API
change, drastically changing the appearance of a plot is about as big of an
API change as mpl can have.

You seem to want to have it both ways, arguing that changing the default
color map is important and we need to do it right now and that it's no
big deal as users adversely affected can always explicitly fix their code.


Reply to this email directly or view it on GitHub
#875 (comment)
.

njsmith commented Oct 21, 2014

I'm not having anything both ways. My argument is that the positive
benefits are large and unambiguous, while the negative effects are
relatively minor. A majorly disruptive change is one that, like, renames
some key function so everyone's code crashes, and everyone downstream has
to rewrite their code immediately (breaking compatibility with older
versions in the process). That's the kind of change that "2.0" numbers are
used to justify. This, by contrast, is a change that leaves everyone's code
working, in 99% of cases working better than before, and in the remaining
1% they can fix their code at their leisure because it's still usable until
they have time to make the fix. And of course the fix is totally compatible
with older versions of matplotlib as well.

I'm also not saying we should be hasty, as your "right now" seems to
imply. But putting something off indefinitely is the same as not doing it.
This bug has been open for 2 years. The many problems with jet have been
well-documented for even longer. There are heaps of evidence that
defaulting to jet produces misinterpretation of data and is thus harmful to
science; basically everyone who has considered the issue agrees on this,
and consensus has just gotten stronger over time. Even Matlab, a pretty
serious bastion of conservatism, has made this change, and is now arguing
that jet is terrible. It might well make sense to wait a few months to see
the reaction from Matlab users, but beyond that how long would you suggest
we wait? AFAIK planning for "matplotlib 2.0" has not even started. So what
are you proposing? 5 years from now? 10 years from now? Jet is being bad
right now.

On Tue, Oct 21, 2014 at 6:44 PM, Thomas A Caswell notifications@github.com
wrote:

@njsmith https://github.com/njsmith I disagree this is a minor API
change, drastically changing the appearance of a plot is about as big of an
API change as mpl can have.

You seem to want to have it both ways, arguing that changing the default
color map is important and we need to do it right now and that it's no
big deal as users adversely affected can always explicitly fix their code.


Reply to this email directly or view it on GitHub
#875 (comment)
.

Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

Owner

efiring commented Oct 21, 2014

Despite my general tendency to be conservative about things like this, I think @njsmith is making good points. The biggest practical problem is making a good choice for the next default; it affects people's work, and it affects their view of mpl. I haven't found any of the suggestions so far to be compelling. The paper cited by @njsmith above is helpful; to my eye, it shows that "coolwarm" is a better default than gray or any sort of heatmap. But can we do better, from either or both important standpoints: aesthetics, and ability to provide a detailed and accurate view of a wide variety of plot types and datasets?

Considering the mathematical effort Moreland went through to reach the conclusions, I somehow doubt that the very diverging (even contradicting in parts) requirements quoted in the paper can be met much more effective.
For convenience of the discussion I cite the requirements as given in that paper:

  • The map yields images that are aesthetically pleasing.
  • The map has a maximal perceptual resolution.
  • Interference with the shading of 3D surfaces is minimal.
  • The map is not sensitive to vision deficiencies.
  • The order of the colors should be intuitively the same for all people.
  • The perceptual interpolation matches the underlying scalars of the map.

While the grey colormap might be the safest choice for avoiding bias, it clearly is not the most optimal for exploiting the dynamic ranges the human eye is able to perceive. Color, IMHO, is just a too valuable dimension for packing information than to totally give up on that. That's why I find Moreland's paper very telling and helpful, for example reporting that Paraview's default change really did not seem to upset a single user enough to ask about it in the mailing list.

A thought that I had: I know open-source is not really a democracy, but would it hurt to make a user-base poll about this? If provided with the relevant information like this paper and others, the ones that care about will vote according to their preference, and the ones that don't vote might be indifferent to it. This might be against MPL's policies, but I just wanted to drop that in here.

njsmith commented Oct 21, 2014

Further references:

In addition to the blog posts linked on the mailing list, The MathWorks has published a review and annotated bibliography of critiques of jet.

They cite an interesting paper, (Borkin et al, 2011), which finds that while medical students express a verbal preference for rainbow color maps, it causes them to make more diagnostic errors than a perceptually uniform colormap. Another interesting observation is that they found a pure greyscale map to be misleading, because their participants interpreted the greyscale is indicating that they were literally looking at a raw black-and-white image (e.g. an X-ray in their case), while pseudo-color is obviously the result of some processing.

Another problem with greyscale maps is simultaneous contrast (see page 15 of the MathWorks article or this blog post -- basically the issue is that brightness adaptation effects in human vision mean that the same grey color can vary perceptually depending on what other greys are nearby; this makes it useful to have hue as a secondary cue, since hue is not affected by brightness adaptation).

Contributor

NelleV commented Oct 21, 2014

I think choosing a good default colormap is very different than choosing a good colormap. The reasoning we had at the matplotlib summit at scipy (when we discussed this exact matter) was that people were more likely to change colormap if we set the default as being gray: I don't think anyone has done any study on this, but that seems a plausible hypothesis (but indeed, people will probably not be happy).

FYI, if there is a vote on the colormap to use as default (which should come after a vote on whether to break API or not), I'm voting against doing what matlab is doing, however well MathWorks justify their choices. I don't think it is a good strategy to have the same defaults nowadays, and I'm pretty sure there is more than one good colormap in this world. We should be able to have a good default that is matplotlib's colormap. (In addition, I really don't like the aesthetics of matlab's new colormap…)

Contributor

endolith commented Oct 22, 2014

coolwarm ... but through grey instead of through white, so the center is visible but not emphasised.

This was done so that it works well on 3D shaded surfaces: "Maps with big changes in luminance hide shading cues important for determining 3D structure (left image), whereas isoluminant maps minimize shading interference (right image)." "The grayscale and heated-body color maps work poorly in conjunction with 3D shaded surfaces."

But 2D heatmaps don't have this problem, and you typically do want something with wide luminance changes to show more dynamic range, so that's why I suggested 3D plots like plot_surface or plot_trisurf could have a different default than 2D heatmaps like imshow or hexbin.

Also, note that the existing YlGnBu_r from ColorBrewer is pretty similar to Matlab's parula, but doesn't scrunch all the luminance change up at the lower end.

colormap_parula

ylgnbu_r colorbar

coolwarm:
MRI coolwarm
YlGnBu_r:
MRI YlGnBu_r
coolwarm:
bumps coolwarm
YlGnBu_r
bumps YlGnBu_r

(Anyway, don't forget http://en.wikipedia.org/wiki/Parkinson%27s_law_of_triviality)

Owner

efiring commented Oct 22, 2014

@endolith, I agree that coolwarm and YlGnBu are reasonable choices for diverging and sequential maps. Suppose we had new rcParams for a default divergent map (e.g., image.divergent_cmap), and for a default sequential map. A kwarg specification of "cmap='divergent'" (maybe with abbrev 'div') would yield the former, and "cmap='sequential'" (or seq) the latter. It's easier to remember how to type 'sequential', and to remember what it means, than 'YlGnBu', for example. There would still be an image.cmap rcParam, which could be set to any named map, but which would default to either 'divergent' or 'sequential'. I could be happy with either, if the respective defaults are decent maps such as the ones you suggest. I'm reluctant to go so far as having a different rcParam for 3D (image3d.cmap?), but I would defer to @WeatherGod on that.

Member

WeatherGod commented Oct 22, 2014

@efiring, I see what you are getting at, and it is an intriguing idea, but
I worry about possible confusion over whether you are asking for a
particular colormap, or a type of colormap. I.e., would the reader of the
code think that "div" is the name of a colormap, or would they think that
it is an alias for a default colormap?

On Wed, Oct 22, 2014 at 2:00 AM, Eric Firing notifications@github.com
wrote:

@endolith https://github.com/endolith, I agree that coolwarm and YlGnBu
are reasonable choices for diverging and sequential maps. Suppose we had
new rcParams for a default divergent map (e.g., image.divergent_cmap), and
for a default sequential map. A kwarg specification of "cmap='divergent'"
(maybe with abbrev 'div') would yield the former, and "cmap='sequential'"
(or seq) the latter. It's easier to remember how to type 'sequential', and
to remember what it means, than 'YlGnBu', for example. There would still be
an image.cmap rcParam, which could be set to any named map, but which would
default to either 'divergent' or 'sequential'. I could be happy with
either, if the respective defaults are decent maps such as the ones you
suggest. I'm reluctant to go so far as having a different rcParam for 3D
(image3d.cmap?), but I would defer to @WeatherGod
https://github.com/weathergod on that.


Reply to this email directly or view it on GitHub
#875 (comment)
.

Owner

tacaswell commented Oct 22, 2014

Given that this is a live discussion again, I am going to re-open the issue.

tacaswell reopened this Oct 22, 2014

Member

pelson commented Oct 23, 2014

I don't have anything to add about choosing a sensible default colormap (which I do support), but I would like to back @njsmith in the suggestion of not being afraid of a 2.0 release. There will never be a good time to break people's code, and I'd far sooner do it in small easily documented iterations than to wait for lots of big API changes in a single big-bang release. Getting over the hurdle of the 2.0 will make it easier to consider further major releases down the line and ultimately will result in matplotlib being a better product.

Contributor

toddrjen commented Oct 24, 2014

"I guess watching the reaction to Matlab's change will be informative :-)."

Probably not. Matlab users are used to constant, unexpected, and sometimes undocumented breaks in backwards compatibility with every release. So Matlab users either stick to a single version forever or get used to having things break unexpectedly. Further, unit testing has never been a key part of Matlab (they only added it to the standard library last year), so there aren't much in the way of third-party unit tests that could break.

On the other hand, unit tests have been emphasized as a key part of python development for a long time, and both python and matplotlib have kept pretty strict backwards-compatibility and deprecation policies. So there are a tony of third-party library and user unit tests that will be affected, and users really aren't used to major backwards-compatibility breaks.

Since the expectations of both downstream libraries (to the extent that such a thing even exists with Matlab) and individual users are very different between Matlab and matplotlib, I don't think a lack of complaint from Matlab users would be all that informative.

mwaskom commented Nov 21, 2014

I think choosing a good default colormap is very different than choosing a good colormap. The reasoning we had at the matplotlib summit at scipy (when we discussed this exact matter) was that people were more likely to change colormap if we set the default as being gray: I don't think anyone has done any study on this, but that seems a plausible hypothesis (but indeed, people will probably not be happy).

FWIW I think this is correct (though I also lack data) and it's what motivates the use of Greys as the default in seaborn. There's no way to choose one colormap that is optimal for all datasets. So the default should encourage people to think about their data and make a choice themselves while not being actively misleading about data the way Jet is for those who nevertheless stick with the default.

A sequential colormap is probably a better default also; my sense is more data is sequential than diverging, but more importantly, to use a diverging colormap correctly you also need to know where to put the midpoint which is hard to do with generic defaults.

Owner

efiring commented Nov 21, 2014

Sequential versus diverging can be a false dichotomy here; even when one can argue that data is fundamentally sequential (e.g., temperature), it can be useful to plot it with a diverging colormap, to gain resolution. Nevertheless, if a single default is provided, I agree that a sequential map could be a reasonable choice. I do not agree that the idea should be to choose a default no one will actually want, however, to try to force people to make an explicit choice. This is not the way to attract and retain users. A default should work reasonably well under a wide range of expected circumstances, and it should be visually pleasing to as broad an audience as possible. I don't think a gray scale meets these criteria; but @endolith's suggestion of YlGnBu_r does, for example.

Member

pelson commented Nov 21, 2014

Let's bring this conversation into the matplotlib-devel mailing list where a threaded discussion can freely take place. Hopefully we can put this issue to bed in the near future.

@tacaswell tacaswell modified the milestone: v2.0 Nov 22, 2014

endolith changed the title from Replace "jet" with "hot" as the default colormap to Replace "jet" as the default colormap Dec 27, 2014

endolith referenced this issue in scikit-image/scikit-image Dec 27, 2014

Merged

DOC: change colormap away from "jet" in examples #1327

gregcaporaso referenced this issue in biocore/scikit-bio Feb 26, 2015

Open

ENH: add Alignment.heatmap method #816

Member

ivanov commented Jun 3, 2015

Here's a recent discussion on the mailing lists about this, with a proposal

Member

WeatherGod commented Jun 3, 2015

Paul, this is fantastic! A nice, well-organized page showing the three
proposed colormaps and a common set of graphs for each. We should get this
out onto the user list.

On Wed, Jun 3, 2015 at 12:41 AM, Paul Ivanov notifications@github.com
wrote:

Here's a recent discussion on the mailing lists
http://matplotlib.1069221.n5.nabble.com/RFC-candidates-for-a-new-default-colormap-td45652.html
about this, with a proposal http://bids.github.io/colormap/


Reply to this email directly or view it on GitHub
#875 (comment)
.

Owner

tacaswell commented Jun 3, 2015

@njsmith Can that page get integrated with/replace https://github.com/matplotlib/matplotlib/blob/color_overhaul/doc/devel/color_changes.rst so we have a record in the docs?

Would be nice to add also some of the alternatives on http://bids.github.io/colormap/ to matplotlib.

njsmith commented Jul 15, 2015

Yep, that's the plan.

On Wed, Jul 15, 2015 at 3:22 AM, Werner Beroux notifications@github.com
wrote:

Would be nice to add also some of the alternatives on
http://bids.github.io/colormap/ to matplotlib.


Reply to this email directly or view it on GitHub
#875 (comment)
.

Nathaniel J. Smith -- http://vorpus.org

Owner

tacaswell commented Jul 15, 2015

There is a discussion over names happening in a scikit-image PR thread (
scikit-image/scikit-image#1599)

There is a proposal for naming options A-C as {'ignis', 'ortus', 'fyrian'}
in some order.

Commenting here and copying the devel list so it gets some visibility.

On Wed, Jul 15, 2015 at 11:56 AM Nathaniel J. Smith <
notifications@github.com> wrote:

Yep, that's the plan.

On Wed, Jul 15, 2015 at 3:22 AM, Werner Beroux notifications@github.com
wrote:

Would be nice to add also some of the alternatives on
http://bids.github.io/colormap/ to matplotlib.


Reply to this email directly or view it on GitHub
<
#875 (comment)

.

Nathaniel J. Smith -- http://vorpus.org


Reply to this email directly or view it on GitHub
#875 (comment)
.

In the proposal, what's the motivation for using XYZ colorspace instead of the Lab colorspace (like Matlab) when calculating perceptual differences?

This question is motivated by Matlab's review of rainbow colorspaces. This includes

Color Map Construction Principles

Use these guidelines for building effective color maps:

  • Construct a perceptual color map by using smooth, equal-step interpolation in a perceptually uniform color space such as Lab*.
  • Use monotonic lightness (L*) to achieve perceptual ordering and assist color-impaired viewers.
  • Do not use lightness variation alone because of the problem of simultaneous contrast. Use hue or saturation variation in addition to lightness variation. (Hue variation can also improve the attractiveness of a color map.)

That perceptually uniform stuck out at me and I looked into it. From the Lab wikipedia page, the definition of perceptually uniform is

Perceptually uniform means that a change of the same amount in a color value should produce a change of about the same visual importance.

We want the default colormap in mpl to be perceptually uniform, correct? According to the same wiki page,

XYZ color space ... is not particularly perceptually uniform. [3]
...
the intention of both "Lab" color spaces is to create a space that can be computed via simple formulas from the XYZ space but is more perceptually uniform than XYZ. [4]

If mpl cares about perceptual uniformity, it sounds like it should us the Lab color space to calculate perceptual differences.

Member

WeatherGod commented Jul 15, 2015

Scott,

I would suggest watching this presentation which explains the limitations
of Lab space and exactly how the viridis colormap was constructed. It
certainly changed my mind about Lab:
https://www.youtube.com/watch?v=xAoljeRJ3lU

Ben Root

On Wed, Jul 15, 2015 at 3:03 PM, Scott S. notifications@github.com wrote:

In the proposal http://bids.github.io/colormap/, what's the motivation
for using XYZ colorspace instead of the Lab colorspace (like Matlab) when
calculating perceptual differences?

This question is motivated by Matlab's review of rainbow colorspaces
http://www.mathworks.com/tagteam/81137_92238v00_RainbowColorMap_57312.pdf.
This includes

Color Map Construction Principles

Use these guidelines for building effective color maps:

  • Construct a perceptual color map by using smooth, equal-step
    interpolation in a perceptually uniform color space such as L_a_b*.

  • Use monotonic lightness (L*) to achieve perceptual ordering and
    assist color-impaired viewers.

  • Do not use lightness variation alone because of the problem of
    simultaneous contrast. Use hue or saturation variation in addition to
    lightness variation. (Hue variation can also improve the attractiveness of
    a color map.)

    That perceptually uniform stuck out at me and I looked into it. From the Lab
    wikipedia page https://en.wikipedia.org/wiki/Lab_color_space, the
    definition of perceptually uniform is

Perceptually uniform means that a change of the same amount in a color
value should produce a change of about the same visual importance.

We want the default colormap in mpl to be perceptually uniform, correct?
According to the same wiki page,

XYZ color space ... is not particularly perceptually uniform. [3
http://www.brucelindbloom.com/UPLab.html]
...
the intention of both "Lab" color spaces is to create a space that can be
computed via simple formulas from the XYZ space but is more perceptually
uniform than XYZ. [4 http://www.handprint.com/HP/WCL/color7.html#CIELUV]

If mpl cares about perceptual uniformity, it sounds like it should us the
Lab color space to calculate perceptual differences.


Reply to this email directly or view it on GitHub
#875 (comment)
.

Member

ivanov commented Jul 15, 2015

(Ben, I took the liberty of updating your link to the direct youtube one - google doesn't play nicely when you try to copy-link from search results)

Member

WeatherGod commented Jul 15, 2015

Thanks Paul, I was wondering about the length of that URL...

Anyway, the tl;dr; of the video is that, yes, Lab space does measure color
distances better than XYZ, but Stefan and Nathanial didn't use XYZ. They
used newer model that had much lower errors for both nearby and far away
colors. They even show examples of parula having apparent gradients that
didn't really exist.

On Wed, Jul 15, 2015 at 3:13 PM, Paul Ivanov notifications@github.com
wrote:

(Ben, I took the liberty of updating your link to the direct youtube one -
google doesn't play nicely when you try to copy-link from search results)


Reply to this email directly or view it on GitHub
#875 (comment)
.

njsmith commented Jul 15, 2015

@scottsievert: Is there text somewhere saying that XYZ space was used to calculate perceptual differences? If so it'd be good to know so we can correct it -- we used CAM02-UCS, which is a perceptually uniform space that improves on CIEL_a_b_. To get to either CAM02-UCS and CIEL_a_b_ space you have to pass through XYZ -- here's the transformation graph supported by colorspacious 0.1.0 --

colorspacious

but that's an implementation detail...

[Edit for clarify: CAM02-UCS is the J'a'b' node in the lower-right.]

Got it. In your talk mentioned above, you referred to the XYZ space more often and I assumed that's what you were using to calculate the perceptual differences. It's good to know otherwise and I'll trust you on this.

njsmith commented Jul 15, 2015

Ah, gotcha, right, that could definitely have been clearer. Something to
keep in mind if I end up giving the talk again :-)

On Wed, Jul 15, 2015 at 2:44 PM, Scott S. notifications@github.com wrote:

Got it. In your talk https://www.youtube.com/watch?v=xAoljeRJ3lU
mentioned above, you referred to the XYZ space more often and I assumed
that's what you were using to calculate the perceptual differences. It's
good to know otherwise and I'll trust you on this.


Reply to this email directly or view it on GitHub
#875 (comment)
.

Nathaniel J. Smith -- http://vorpus.org

jhamman referenced this issue in pydata/xarray Jul 18, 2015

Merged

Feature plotting #466

Contributor

stefanv commented Jul 30, 2015

See also #4707

@mdboom mdboom added a commit to mdboom/matplotlib that referenced this issue Nov 16, 2015

@mdboom mdboom Change default color map (Fix #875) 6c6c0a3

@mdboom mdboom added a commit to mdboom/matplotlib that referenced this issue Nov 17, 2015

@mdboom mdboom Change default color map (Fix #875) 7d9b1d4

@mdboom mdboom added a commit to mdboom/matplotlib that referenced this issue Nov 23, 2015

@mdboom mdboom Change default color map (Fix #875) 04f34eb

@mdboom mdboom added a commit to mdboom/matplotlib that referenced this issue Nov 23, 2015

@mdboom mdboom Change default color map (Fix #875) b13e456

@mdboom mdboom added a commit to mdboom/matplotlib that referenced this issue Nov 25, 2015

@mdboom mdboom Change default color map (Fix #875) f61e747

@mdboom mdboom added a commit to mdboom/matplotlib that referenced this issue Nov 27, 2015

@mdboom mdboom Change default color map (Fix #875) 6d1f588

@mdboom mdboom added a commit to mdboom/matplotlib that referenced this issue Nov 27, 2015

@mdboom mdboom Change default color map (Fix #875) 0500545

@mdboom mdboom added a commit to mdboom/matplotlib that referenced this issue Dec 14, 2015

@mdboom mdboom Change default color map (Fix #875) a96a259

@mdboom mdboom added a commit to mdboom/matplotlib that referenced this issue Dec 14, 2015

@mdboom mdboom Change default color map (Fix #875) df633fe

@mdboom mdboom added a commit to mdboom/matplotlib that referenced this issue Dec 14, 2015

@mdboom mdboom Change default color map (Fix #875) 500e092

@mdboom mdboom added a commit to mdboom/matplotlib that referenced this issue Dec 17, 2015

@mdboom mdboom Change default color map (Fix #875) b648c02

@mdboom mdboom added a commit to mdboom/matplotlib that referenced this issue Dec 31, 2015

@mdboom mdboom Change default color map (Fix #875) 00f3800

@mdboom mdboom added a commit to mdboom/matplotlib that referenced this issue Dec 31, 2015

@mdboom mdboom Change default color map (Fix #875) ffa2d9a

@mdboom mdboom added a commit to mdboom/matplotlib that referenced this issue Jan 4, 2016

@mdboom mdboom Change default color map (Fix #875) 628ad3f

mycarta commented Jan 29, 2016

Is there something like the Kindleman set available? Do some of the new stylesheets already improve this?

@michaelaye I only noticed this comment of yours today.

I have a version of Kindleman linear lightness colormap in the second half of this notebook tutorial.

Using the same data and function in the notebook I also replicated his circular colormap. it's been sitting on my home computer for a while, I just uploaded it on GitHub here.

tacaswell closed this in 28c632f Feb 8, 2016

njsmith commented Feb 9, 2016

🌟 👯 🌟

tbreloff referenced this issue in JuliaPlots/Plots.jl Oct 20, 2016

Closed

Missing colorschemes in heatmap plot #542

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment