Decorator consistency

[JayPanoz]

This builds upon the discussion in #170

• Removes mix-blend-mode from fallback implementation when the highlight API cannot be used so that there is no side-effect Improves mix-blend-mode from the fallback implementation by running an additional check on the computed value of documentElement if no ReadiumCSS property is present
• Additionally sets an observer to update mix-blend-mode value if needed – background switches to/from light–dark
• For Highlight API, checks the background-color and picks the text-color for contrast
• Renames r2-highlight-0 to readium-highlight, and adds data-readium=true to the templated fallback, to follow the current pattern when injecting things

In practice I am not sure we need to add data-readium to the templated item since it already is in a container for which it is set, but at least it makes it clear it is injected by us if you query this item/className directly.

[chocolatkey]

Adding data-readium is a good idea.
If you've tested removing mix-blend-mode and it worked, then I'm fine with this change. I essentially brute-forced this solution while at DM and testing with light and dark mode styling at the time.

[JayPanoz]

Glad you said that because I can see issues on testing again. Looks like I forgot to replace mix-blend-mode with z-index. And on additional testing I could catch some edge cases.

I will put that back in draft, as it's quite a nasty one – we can no longer derive dark mode from --USER__appearance as it is no longer in use. And well, the challenge is that the background and text colors can be completely arbitrary.

I'll take time to check how Thorium Desktop got around this issue as I can recall Daniel telling he had to do that too.

[JayPanoz]

So yeah it is really hard to do without mix-blend-mode, I'll have to check Daniel's code and file history to understand how he resolved it properly.

I've improved the color helpers slightly, and removed the one that could not be tested properly.

There are two obvious issues that can be addressed immediately:

• isDarkMode is relying on ReadiumCSS custom properties, I added the computed style for documentElement as a fallback for when they are missing or not compatible with the helpers, although that is not catching the case where there is another color set in-between;
• isDarkColor helper is trying its best to infer the color is actually dark but there is some conversion to rgba beforehand, and it does not support all the possible values.

RGB and HEX are covered – adding the computed style should return rgba directly so there's that –, but if we wanted to cover more, I guess we would need a dependency such as this one.

[JayPanoz]

On a related note, this currently fails with images when using exclusion because in practice, the image has a background itself, and we apply this based on the documentElement. To be consistent it would have to be multiply for the div on top of the image here, and exclusion for the rest. Unless the image has an invert filter, etc.

I’m not sure what’s best between not caring, trying to handle this complexity, or removing the check for img, audio, video, etc. but at some point, we will have to discuss this I guess.

[JayPanoz]

This has been reworked…

Sorry to bother you again, @chocolatkey

Mainly, since we cannot get around mix-blend-mode that easily:

• Improves mix-blend-mode from the fallback implementation by running an additional check on the computed value of documentElement if no ReadiumCSS property is present
• Additionally sets an observer to update mix-blend-mode value if needed – background switches to/from light–dark
• For Highlight API, checks the background-color and picks the text-color for contrast

This brings pseudo consistency between the fallback implementation and the Highlight API one. Of course blending is well… blending so yellow will not be the same yellow on mint, sepia, etc. But it now reacts to background-changes to stay yellow-ish.

Note I also improved highlight API implementation by checking contrast of text on the background.

For images, unfortunately I couldn’t find something obvious. So I refrained from diving into a rabbit hole at this point in time.

[danielweck]

a long time ago i implemented an experiment with z-index (to send div rectangles to the background) and it was specularly good compared with mix-blend-mode (multiply, or another function ... I can't remember exactly). it was good because the solid highlight colour layered underneath the text glyphs looked exactly as intended. however it was bad because it only worked with some EPUBs (authored CSS), and I couldn't find a reliable solution to make it universal. furthermore there was no control over the foreground colour of highlighted text characters.

of course, nowadays this is a moot point thanks to native CSS highlights which control both background and foreground.

the mix-blend-mode technique for topmost-rendered div overlays works well-enough as a fallback ... with the caveat of text foreground colour contrast being problematic / suboptimal.

[danielweck]

thanks for the pointer to:

https://github.com/elchininet/ColorTranslator

IIRC we're using this at the moment:

https://github.com/Qix-/color

[chocolatkey]

It's not the most ideal, but it should be good enough for now. For the highlight color of things like images, I don't think implementors care as much if it's accurate to the intended color compared to text, and for text hopefully more and more of available browsers will support the Highlight API

[danielweck]

are you sure the rgb() parsing regular expression works as intended?

what about rgba() which normally uses the comma-separated 4th numerical value for alpha channel.

what about rgb() with modern non-legacy syntax?

https://developer.mozilla.org/en-US/docs/Web/CSS/Reference/Values/color_value/rgb

[JayPanoz]

@danielweck no that’s a good point, many shortcuts were taken based on expected inputs + handling #170 in the immediate term and I guess we’re clearly lacking a bulletproof set of helpers across projects that we could improve collectively for color management.

Effectively injectables should also be independent of Readium CSS if we wanted to do things properly, as you could use them without the Navigator, etc.

I can see the shortcut to canvas to get rgba instead of relying on a lib for color conversion may fail as well with some color form, depending on the input: https://stackoverflow.com/a/79292386

So… back to the drawing board.

[danielweck]

the canvas trick is useful but requires a DOM. would it be better to offer a conversion function that is purely "computational"?

I was also thinking that the canvas (lazy) instance should be cached to reduce the overhead when the function is called repeatedly with colour notations other than # or RGB()

[danielweck]

PS: in Thorium Desktop we also rely on syntactical expectations to minimise implementation complexity. internally, colours are expressed with the # RGB hex notation (upstream, app level) which we convert to RGB() as needed for downstream rendering (and the annotation interchange format uses symbolic names iirc)

[danielweck]

strictly-speaking this regexp is also "incorrect" (edge cases) but the conversion function does the job in our use-case ... and of course it looks cool because bitshift operator :D

export function convertColorHexadecimalToRGBA(cssHex: string, alpha?: number): string | undefined {
    if (!/^#([A-Fa-f0-9]{3}){1,2}$/.test(cssHex)) return undefined;
        const hex = cssHex.substring(1);
        const hex_ = hex.length === 3 ?
            `0x${hex[0]}${hex[0]}${hex[1]}${hex[1]}${hex[2]}${hex[2]}` :
            `0x${hex[0]}${hex[1]}${hex[2]}${hex[3]}${hex[4]}${hex[5]}`;
        const hexVal = parseInt(hex_, 16);
return `rgb${typeof alpha === "number" ? "a" : ""}(${(hexVal >> 16) & 255}, ${(hexVal >> 8) & 255}, ${hexVal & 255}${typeof alpha === "number" ? `, ${alpha}` : ""})`;
}

[JayPanoz]

Yeah that’s the thing here, I’d be expecting HEX or rgb(a), or at least something that, once computed, returns rgb(a), but as seen in the SO reply, browsers can return color() and then you’re toast – even libs don’t handle that after a Quick Look.

So canvas is fine if you have a DOM, computational would be ideal to cover all cases. But… unless we guard the tint value Decorator accepts and warn some forms we cannot support properly, I’m not sure what we can do in practice.

[danielweck]

there's this as well:

https://github.com/edrlab/thorium-reader/blob/develop/src/common/rgb.ts

(again, not generic, but specific to our use-case so supposes particular input/output formats)

[JayPanoz]

Ok so seeing this article, “Writing the perfect RGB regex and Failing”, I think I won’t take any chance and throw everything into a lazy-instantiated canvas first, with caching of already parsed strings. And not care about edge cases such as CSS custom properties, etc.

That should be fine for us at this point in time, knowing it is used in the context of Decorator so when the DOM is available, and there will be a warning the value could not be parsed.

In the foreseeable future, I think we should really share something more robust across projects. I’ve been taking inspiration from both the Kotlin and r2-navigator-js for this PR, so I am aware there is overlap, and such helpers would probably benefit all of us, with multiple pair of eyes fixing bugs and improving support. That means not relying on the DOM as well for some use cases, etc. So definitely some additional effort.

This also means opening helpers to more use cases such as picking a text color for a given background in theming, or adjusting one based on the other, etc. Something we planned in Thorium Web actually.

It’s already been a bit of a rabbit hole so I have to stop for ts-toolkit at some point, before it sidetracks other tasks, but it is definitely something we could talk about in the next meeting.

[JayPanoz]

Thanks for your patience and inputs.

@danielweck I know there is a lot on your plate right now. But given you have been contributing invaluable inputs and feedback so much, may I request a review from you? This would also probably help with the initial step of shaping such helpers so that they can be re-used across projects. 🙏