[WIP] Smart color scale: add percentile limits for smart color scale.

[Dominik-Vogel]

This is marked as WIP as it will need to be merged together with the corresponding qdev wrappers pr.

Enable to limit the maximum percentage of datapoints that may be clipped by the smart color scale on either side of the distribution.
This method has the advantage not enable to control how much to discard of a signal on a noisy background as an outlier. The simulated data tries to resemble a relevant measurement situation.

Side effects:

• includes a ipynb stub for an overview of all tutorials
• fixes the issue of plot_by_id not using the color map as defined in qcodesrc.json

no smart color scale:

no limiting of percentiles:

with limiting percentiles = 0.5

[codecov]

Codecov Report

Merging #1259 into master will increase coverage by 1.52%.
The diff coverage is 82.89%.

@@            Coverage Diff             @@
##           master    #1259      +/-   ##
==========================================
+ Coverage   70.92%   72.45%   +1.52%     
==========================================
  Files          74       74              
  Lines        8352     8415      +63     
==========================================
+ Hits         5924     6097     +173     
+ Misses       2428     2318     -110

[jenshnielsen]

It would be super cool if we marked the clipped areas. The following prototype does that

import matplotlib.pyplot as plt
import numpy as np

cmap = plt.cm.get_cmap("viridis")
cmap.set_over('magenta')
cmap.set_under('cyan')
fig, ax = plt.subplots(1,1)
pc = ax.pcolormesh(np.random.rand(100,100), cmap=cmap)
cb =fig.colorbar(pc, extend='both')
pc.set_clim(0.1, 0.9)

[Dominik-Vogel]

That is great! Thank you for the code, I initially also tried that but couldn't get it to work. I'll include this feature.

[jenshnielsen]

could we add a info log message here with the limits used and how much is clipped

[Dominik-Vogel]

I just remembered the problem: It is all fine to set up the extend as you said, but I need to set it up after the colorbar has been created, and this does not seem to work well....

[Dominik-Vogel]

Ok finally found it @jenshnielsen . It seems to be an inconsistency in matplotlib. I will also need to redefine the slice that defines the points inside the colorbar:

cb.extend = 'both'
cb._inside = cb._slice_dict[cb.extend]
cb.mappable.set_clim(0, 0.3)

Maybe we can make a PR for matplotlib next week...

[Dominik-Vogel]

Now I get :-) :

axes, cb = plot_by_id(54,smart_colorscale=False)
apply_auto_color_scale(cb[0], cutoff_percentile=(0.5,0.5))

The lower part is not cutoff because it does not fall into the criterium of the IQR-selector.

[astafan8]

is indentation ok? (it probably does not matter though... i forgot)

[Dominik-Vogel]

yeah, that is apparently two spaces vs one tab...