Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve bioRad density colour scale #38

Closed
peterdesmet opened this issue Jan 15, 2020 · 6 comments
Closed

Improve bioRad density colour scale #38

peterdesmet opened this issue Jan 15, 2020 · 6 comments
Assignees

Comments

@peterdesmet
Copy link
Member

The density colour scale in bioRad for height profiles is defined here: https://github.com/adokter/bioRad/blob/e0ede427eb34007dc9985302d40cbdab158e0636/R/color_scale.R#L65-L85

The resulting 256 values are:

"#C8C8C8", "#FFFFFF", "#FFFFFB", "#FFFFF8", "#FFFFF5", "#FFFFF2",
"#FFFFEE", "#FFFFEB", "#FFFFE8", "#FFFFE5", "#FFFFE1", "#FFFFDE",
"#FFFFDB", "#FFFFD8", "#FFFFD4", "#FFFFD1", "#FFFFCE", "#FFFFCB",
"#FFFFC7", "#FFFFC4", "#FFFFC1", "#FFFFBE", "#FFFFBA", "#FFFFB7",
"#FFFFB4", "#FFFFB1", "#FFFFAD", "#FFFFAA", "#FFFFA7", "#FFFFA4",
"#FFFFA1", "#FFFF9D", "#FFFF9A", "#FFFF97", "#FFFF94", "#FFFF90",
"#FFFF8D", "#FFFF8A", "#FFFF87", "#FFFF83", "#FFFF80", "#FFFF7D",
"#FFFF7A", "#FFFF76", "#FFFF73", "#FFFF70", "#FFFF6D", "#FFFF69",
"#FFFF66", "#FFFF63", "#FFFF60", "#FFFF5C", "#FFFF59", "#FFFF56",
"#FFFF53", "#FFFF50", "#FFFF4C", "#FFFF49", "#FFFF46", "#FFFF43",
"#FFFF3F", "#FFFF3C", "#FFFF39", "#FDFF36", "#F8FF32", "#F4FD2F",
"#EFF72C", "#EAF129", "#E5EA25", "#E0E422", "#DBDE1F", "#D6D81C",
"#D2D218", "#CDCC15", "#C8C612", "#C3BF0F", "#BEB90B", "#B9B308",
"#B4AD05", "#B0A702", "#ABA300", "#A6A300", "#A5A300", "#ADA300",
"#B4A300", "#BCA300", "#C4A300", "#CBA300", "#D3A300", "#DBA300",
"#E3A300", "#EAA300", "#F2A300", "#FAA300", "#FFA300", "#FFA300",
"#FFA300", "#FFA302", "#FFA308", "#FFA30D", "#FFA313", "#FFA319",
"#FFA31F", "#FFA325", "#FFA32B", "#FFA331", "#FFA337", "#FFA33D",
"#FFA342", "#FFA348", "#FFA34E", "#FFA04F", "#FF9B4B", "#FF9646",
"#FF9141", "#FF8C3C", "#FF8737", "#FF8232", "#FF7C2E", "#FF7729",
"#FF7224", "#FF6D1F", "#FF681A", "#FF6315", "#FF5E10", "#FF590C",
"#FF5407", "#FF4E02", "#FF4900", "#FF4400", "#FF3F00", "#FF3A00",
"#FF3500", "#FF3000", "#FF2B00", "#FF2600", "#FF2000", "#FF1B00",
"#FF1600", "#FF1100", "#FF0C00", "#FF0700", "#FF0200", "#FF0000",
"#FF0000", "#FF0000", "#FB0000", "#F60000", "#F00000", "#EA0000",
"#E50000", "#DF0000", "#D90000", "#D40000", "#CE0000", "#C90000",
"#C30000", "#BD0000", "#B80000", "#B20000", "#AC0003", "#A70008",
"#A1000E", "#9C0013", "#960019", "#90001E", "#8B0024", "#850029",
"#7F002E", "#7A0034", "#740039", "#6E003F", "#690044", "#63004A",
"#5E004F", "#580055", "#52005A", "#51005F", "#510065", "#51006A",
"#510070", "#510075", "#51007B", "#510080", "#510086", "#51008B",
"#510091", "#510096", "#51009B", "#5100A1", "#5100A6", "#5100AC",
"#4D00B1", "#4800B7", "#4300BC", "#3F00C2", "#3A00C7", "#3500CC",
"#3000D2", "#2B00D7", "#2700DD", "#2200E2", "#1D00E8", "#1800ED",
"#1400F3", "#0F00F8", "#0A00FD", "#0500FA", "#0000F5", "#0000F0",
"#0000EB", "#0000E5", "#0000E0", "#0000DB", "#0000D6", "#0000D0",
"#0000CB", "#0000C6", "#0000C1", "#0000BC", "#0000B6", "#0000B1",
"#0000AC", "#0000A7", "#0000A1", "#00009C", "#000097", "#000092",
"#00008D", "#000087", "#000082", "#00007D", "#000078", "#000072",
"#00006D", "#000068", "#000063", "#00005E", "#000058", "#000053",
"#00004E", "#000049", "#000043", "#00003E", "#000039", "#000034",
"#00002F", "#000029", "#000024", "#00001F", "#00001A", "#000014",
"#00000F", "#00000A", "#000005", "#000000"
@niconoe
Copy link
Collaborator

niconoe commented Feb 11, 2020

@peterdesmet: since the R code is not immediately obvious to me, I have two questions:

  • What's the mapping (density values) for those colors?
  • Can the large colour table be made smaller? For example by interpolating from a few "pivot" colors?

Don't worry if you can't easily answer, I'll dig deeper into the R implementation.

@niconoe
Copy link
Collaborator

niconoe commented Feb 21, 2020

Update: we got details by e-mail, copying here for future references:

Wat er in bioRad zit [2] komt me inderdaad bekend voor ;-). De IDL
kleurenschaal is een redelijk gekke schaal met erg veel (17)
ijkkleuren waartussen geïnterpoleerd wordt. Wat er gebeurt om de
uiteindelijke kleurenschaal te berekenen is dat ik een lijst waarden
wil (van 1 tot 256), met daarbij de bijbehorende kleur (rgb). Daarvoor
maak ik drie paren van vectoren (voor elke kleur (rgb) één paar), met
in de eerste vector de waarde (van 1 tot 256) en in de tweede vector
het niveau van de corresponderende kleur. Vervolgens interpoleer ik
die kleurenniveau’s op alle waarden van 1 tot 256 (met de R-functie
approx()). En daarna bereken ik de rgb code van de kleuren (met de
R-functie rgb()). Ik voeg ook nog een grijze kleur toe (rgb(200, 200,
200)) die gebruikt kan worden voor gebieden zonder data. Ik hoop dat
het hiermee een beetje duidelijker wordt. Ik heb ook even
een plaatje gemaakt met de waarden tegen de niveau’s van de
verschillende kleuren als functie van de waarde (zie bijlage). Het is
in ieder geval een gekke kleurenschaal, die wat reverse-engineering
vergde om uit te zoeken.

RGB_IDL

@peterdesmet peterdesmet changed the title VP chart: Use bioRad density colour scale Use bioRad density colour scale Oct 21, 2020
@niconoe
Copy link
Collaborator

niconoe commented Oct 21, 2020

Check the correctness of scale and colours

@peterdesmet peterdesmet added bug Something isn't working and removed bug Something isn't working labels Oct 21, 2020
@peterdesmet peterdesmet added this to the v1 milestone Oct 21, 2020
@niconoe
Copy link
Collaborator

niconoe commented Feb 3, 2021

I've done some work regarding this colour scale, but also updated related/interleaved matters at the same time. I'm relatively happy with the result I have to admit I'm not 100% comfortable with the the pure science/data-related behind, so I'd like to explain what I did so I can have other pair of eyes looking at both the end result (https://inbo.github.io/crow/#/) and my explanations:

  • First, I manually checked on the VTPS chart that the data density and the rectangle colours on the heatmap are consistent with the colour legend displayed above, for the bioRad colour scale and also for viridis. It's ok. I guess that's already a small part of (One shot) manual data display validation #45 :)
  • bioRad scale: I fixed a bug in the custom interpolator (the function that takes an input value between 0 and 1, and return the corresponding colours - see the intensity curves above). I'm pretty sure it's fine now, since the colours are now nicely spread along the colour legend, without any harsh jumps between them (2 issue we encountered before).
  • After experimenting with multiple D3 scales, I noticed the symlog scale appears to give the more satisfying results (a lot of nuances are visible + generally visually pleasing). I previously made the exact same choice for viridis, for the same reasons. What I don't know is:
    • does this scale/transformations makes (mathematically/theoretically) sense, given the data we deal with?
    • is the output identical (or just "looking similar") to BioRad. And should we aim for that?
  • I adapted the way the colour legend above is displayed to better fit that symlog scale: the ticks are not anymore evenly spaced, but the colours are (before that change the colours for small values were all very "collapsed" on the left of the legend bar).

@peterdesmet
Copy link
Member Author

The bioRad colour scale in our plots is off compared to the bioRad package, see #45 (comment)

@peterdesmet
Copy link
Member Author

peterdesmet commented Feb 11, 2021

See #45 (comment) for the current state on this issue: there is a slight colour difference caused by real log vs sym log and pushing very small values to 0.5. We could try to get it closer, but it has low priority.

@niconoe niconoe changed the title Use bioRad density colour scale Improve bioRad density colour scale Feb 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants