Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve highlights, especially in night mode #65

Closed
JayPanoz opened this issue Jun 20, 2019 · 15 comments
Closed

Improve highlights, especially in night mode #65

JayPanoz opened this issue Jun 20, 2019 · 15 comments
Assignees
Labels
Feature request Indicates the issue is a feature request feedback required

Comments

@JayPanoz
Copy link
Collaborator

I'm submitting a feature request (raised from a bug report during yesterday’s engineering call).

Short description of the issue/suggestion:

Highlights module has been kind of a placeholder for a long time as we never discussed those properly in the past, so not many people use it, and it’s not even compatible with night mode.

The selector isn’t even customisable like most others in src.

Steps to reproduce the issue/enhancement:

N/A.

What is the expected behaviour?

This module should at least be updated to solve the two issues above, but the opportunity to have a “standard” for all Readium-related apps is perhaps even more important there, as it could offer a good-enough default for a lot of apps.

What is the current behaviour?

See description.

Do you have screenshots, GIFs, demos or samples which demonstrate the problem or enhancement?

N/A

What is the motivation / use case for changing the behaviour?

Provide better defaults for highlights, in a way that makes sense for implementers.

Do you know which CSS modules (stylesheets) are impacted?

  • highlights
  • night mode

Please tell us about your environment:

  • Platform: all
  • Browser / Rendering Engine: all
@JayPanoz JayPanoz self-assigned this Jun 20, 2019
@JayPanoz JayPanoz added Feature request Indicates the issue is a feature request feedback required labels Jun 20, 2019
JayPanoz added a commit that referenced this issue Aug 4, 2019
@danielweck
Copy link
Member

The current prototype in r2-navigator-js utilises a RGB colour value (triplet) provided programatically by the API consumer of the "highlights" component (the default colour is red-ish). An opacity level is automatically added by the component (i.e. default translucency, plus an alternative value for mouse hover). In other words, API consumers can only set the colour, not the transparency values.
The same colour and opacity are used regardless of the ReadiumCSS "theme".

@danielweck
Copy link
Member

So, the current ReadiumCSS draft proposal is to use CSS class names: .readiumCSS-[yellow|green|orange|pink]-highlight

https://github.com/readium/readium-css/blob/master/css/src/modules/ReadiumCSS-highlights.css

This will not be usable in practice with the r2-navigator-js prototype, because the aforementioned RGB triplet is applied via CSS rgba() (added opacity as per the two aforementioned translucency levels) on various elements that make up the whole "highlight" visual object made of several rectangles (HTML or SVG), using different CSS properties depending on context (background, border, outline).

@JayPanoz
Copy link
Collaborator Author

JayPanoz commented Aug 5, 2019

Yeah I’m wondering whether we shouldn’t even get highlights out of ReadiumCSS now, given this also reminds me of the Highlight API doc.

And more generally, it will be a lot more complex to handle/customise if it’s in ReadiumCSS.

@danielweck
Copy link
Member

danielweck commented Aug 5, 2019

Yes, ReadiumCSS is by definition "opinionated" (that is what makes it useful), so at best ReadiumCSS could define highlight colours that "work" with its own predefined themes. That being said, if the styles authored at the source in EPUB content documents deviate from the typical "back text on white background", then all bets are off, and it is not possible to guarantee acceptable contrast levels / readability.
However, it would be nice if all R2 apps that implement support for "highlights" offered the same choice of colour palette. In other words, the RGB programmatic API in r2-navigator-js currently allows 256x256x256 possibilities, but this could be restricted to an "opinionated" set of triplets, maybe defined at the ReadiumCSS level?

@JayPanoz
Copy link
Collaborator Author

JayPanoz commented Aug 7, 2019

Sorry had to sleep over this following your remarks.

A few thoughts. Overall, I think there is a possibility I align the module with all others, while providing defaults, and extensibility.

the current ReadiumCSS draft proposal is to use CSS class names: .readiumCSS-[yellow|green|orange|pink]-highlight

It’s really still more of a placeholder than a draft at this point so classes here are more like references making “highlight colours” obvious.

That also means this module is completely different than all others in its design right now. For instance, a RGB colour value (triplet) could become 4 CSS variables (one for R, one for G, one for B, and one for alpha), as seen in this article for HSL(a) for instance.

But, as you highlighted…

using different CSS properties depending on context (background, border, outline).

That could be handled via a “type” flag probably, once again as a CSS variable.

That being said, if the styles authored at the source in EPUB content documents deviate from the typical "back text on white background", then all bets are off, and it is not possible to guarantee acceptable contrast levels / readability.

Just as a note because I haven’t had the time to read it thoroughly but there are attempts to handle text based on the background-color e.g. https://css-tricks.com/css-variables-calc-rgb-enforcing-high-contrast-colors/ (there are also the mix-blend-mode that can help, cf. end of article, but maybe that would introduce some weirdness in the form of text being inverted.)

this could be restricted to an "opinionated" set of triplets, maybe defined at the ReadiumCSS level

So I think there is a possibility of having opinionated set of triplets while allowing for customisation. I’ll try coming up with a proposal asap so that it’s clearer but, something like (pseudo CSS code)…

:root {
  --r: 255;
  --g: 255;
  --b: 0;
  --a: 0.5
}

*[style~="readiumCSS-highlight-bg"] {
  background-color: rgba(var(--r), var(--g), var(--b), var(--a));
}

*[style~="readiumCSS-highlight-border"] {
  border-color: rgba(var(--r), var(--g), var(--b), var(--a));
}

Knowing that if you had other values as inline styles on the element itself, those inline values will be used for background-color, etc.

<div style="… css-variables"></div>

Not a silver bullet, mind you, but at least that would offer defaults + could still be used dynamically if the RS hasn’t implemented the feature yet – I’m really fine with JS implementations for such features, given CSS is lacking for such complex issues.

@danielweck
Copy link
Member

Yes, this looks like a step in the right direction. Thanks Jiminy.

@JayPanoz
Copy link
Collaborator Author

Related notes:

Well at least now I have a clearer idea how the change will turn out, and it’s not massive work – documenting it will be the heavier task.

@JayPanoz
Copy link
Collaborator Author

Also for reference: styling + SVG handling in r2-navigator-js

@mickael-menu
Copy link
Member

Yeah I’m wondering whether we shouldn’t even get highlights out of ReadiumCSS now, given this also reminds me of the Highlight API doc.

At first glance the Android highlight implementation is not using Readium CSS's highlight classes, and I also wonder if it should be part of Readium CSS or not. To align platforms, we can use a shared specification with a set of default colors, but I would expect reading apps to want to customize these colors.

If there was a way to have different color variants per reading mode in Readium CSS, it could make it worthwhile though.

@JayPanoz
Copy link
Collaborator Author

So let’s use this comment for voting on the removal of this module in ReadiumCSS.

in this mobile issue, @mickael-menu said:

After thinking more about this, I believe that highlights should not be in the scope of Readium CSS. If we have some default highlight colors in Readium, they should maybe be documented in architecture, and declared in the test apps. There's just too many different ways to implement highlighting, and also we couldn't reuse the colors declared in Readium CSS for other formats such as PDF.

We also know the Thorium implementation doesn’t use ReadiumCSS, and have this r2-navigator-js issue re. dev needs, which can inform our decision.

What’s become clearer is that ReadiumCSS would be way too constraining implementation-wise as you would have to rely on what it provides as CSS variables for hsl(a)/rgb(a), and it might well be cumbersome given how many additional variables it would create.

In addition, we shouldn’t forget other formats – PDF as mentioned by Mickaël but I’m assuming web publications could have TTS with sync/overlaid highlights, notes, etc. as differentiating features from web browsers.

If you’re in favour of removal, use 👍 . Otherwise, use 👎

@danielweck
Copy link
Member

danielweck commented May 27, 2020

Further data points to consider - in r2-navigator-js there are now 3 "states" for a given highlight:

  1. normal display (for example: 50% opacity)
  2. during mouse hover (for example: 60% opacity, i.e. more emphasis)
  3. during mouse hover on other highlights (for example: 40% opacity, i.e. less emphasis so that overlapping highlights are easier to visually-identify)

Furthermore, CSS mix-blend-mode = multiply is used to ensure text is legible despite the transparent overlay, and this is applied to an "abstract" container div as opposed to "concrete" individual client rectangles (which are optimized to prevent duplicates, overlaps, jagged edges, etc.).

Without going into too much detail, you can see that there are several optimization paths to achieve decent performance when dealing with thousands of character-level drawings (e.g. to present search results inside HTML documents). These techniques make it hard to standardize on simple ReadiumCSS styles / rules.

Note that we use the same CSS styling techniques for both SVG and HTML render modes.

Also note that "highlights" in r2-navigator-js include "strike-through" and "underline" as well as the "background colour" type of visual annotation.

@JayPanoz
Copy link
Collaborator Author

JayPanoz commented May 27, 2020

Ah thanks for these details. If ReadiumCSS had to handle all of this, that would turn this module into a labyrinthine system indeed so it’s not worth bringing confusion into an already tricky issue.

@mickael-menu
Copy link
Member

during mouse hover on other highlights (for example: 40% opacity, i.e. less emphasis so that overlapping highlights are easier to visually-identify)

That's smart :) I wonder if a similar "static" strategy could be used for mobile, where we don't have mouse hover.

@JayPanoz
Copy link
Collaborator Author

JayPanoz commented Jun 3, 2020

So the highlights module has been removed in the develop branch – which means beta.2 release should be imminent.

Nobody complained about this change so we’re assuming this shouldn’t be a breaking change. But be aware that if you were relying on these highlights styles, there won’t be provided any longer.

@JayPanoz
Copy link
Collaborator Author

I guess closing this issue is long overdue. 😅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature request Indicates the issue is a feature request feedback required
Projects
None yet
Development

No branches or pull requests

3 participants