TypeScript implementation & polyfill

[Treora]

Hi all, sorry to open an issue for what is rather an introduction&announcement (but where else to share this?). I have been following this proposal as I have long been wanting the ability to link to arbitrary selections in web pages. A few years ago I made a very similar attempt called quoteurl (my proposed syntax was #(prefix)"startText...endText"(suffix)), and later made a browser extension (Precise links) that implements the suggestion from the Web Annotation selectors&states note (i.e. the syntax #selector(type=TextQuoteSelector,exact=blabla…)).

Now that this proposal has significantly more traction than any before it, as well as a much further worked-out specification, I figured it may be worth jumping on this bandwagon, and try to help make this feature part of the web at large. To better understand the details of the proposal and enable people to play with it in practice, I implemented most of the current draft spec, line by line, in TypeScript:

https://code.treora.com/gerben/text-fragments-ts

The implementation is found in src/index.ts. The repo includes a polyfill (src/polyfill.ts) and a small demo, which you can also play with here.

For your convenience I also mirror the repo on GitHub, so you can create issues and pull requests there (because distributed version control unfortunately still lacks distributed collaboration tools…).

While coding, I encountered small unclarities and mistakes in the spec, which are noted in the source code in comments marked with with XXX. I will open issues and pull requests for some of those, as well as for a few questions about the functionality.

[Treora]

Of course, right after posting this, I discover another effort to create a polyfill: https://github.com/GoogleChromeLabs/text-fragments-polyfill (by @tomayac and @tfmar). It looks like that one is more focussed on practicality, whereas my approach rather follows the spec line by line to function more like a reference implementation. Perhaps we can borrow some tricks from each other.

Also, it would be great to have a repository with tests, to compare various implementations’ compatibility.

[tomayac]

@Treora This is brilliant work! I'm sure @tfmar (who's taken over development of our polyfill) will be very interested in having a deep look.

One limitation in your highlighting approach, apart from the spec's discouragement you mention in the comment, is that it can only highlight one item at a time. I have played with this idea before, too.

[tilgovi]

I've been trying to keep an eye on this CSS Highlights API proposal: https://drafts.csswg.org/css-highlight-api-1/

Using the CSSOM, a polyfill could programmatically add highlights and styles for them, without loading any stylesheets, I think.

Of course, if a scroll-to-text-fragment polyfill relied on the Highlights API, then we'd need a polyfill for the CSS Highlights API 😁.

[tomayac]

#TIL about the CSS Highlights API proposal. This looks like exactly what we'd need. Do you know if this is still being actively pursued? The spec draft as well as all issues are from December 2019…

[tfmar]

Cool, thanks for tagging!

I'm currently working hard on getting our polyfill more in line with the spec--you'll notice some "WIP" functions at the bottom of text-fragment-utils.js that should be fairly familiar 😄. I haven't gone quite as far as commenting the relevant spec lines, but the structure of the algorithm is much closer.

Re: testing, the new code is all reasonably well unit-tested, and I've got the very rough basics in place for testing on HTML documents as well, but it's far from a robust, implementation-independent test suite, which sure would be nice to have right now. I think ideally we could provide some HTML documents and some format of input (maybe just JSON?) indicating the directive as input and the expected highlighted text as outputs, and from there it should be pretty straightforward for various implementations to test against the suite.

Re: highlighting, based on our need for compatibility (particularly with iOS WebKit) we ended up settling on <mark> tags. CSS Highlights would be great, but don't seem close to being ready, and my first guess is that a polyfill would probably be built on <mark> anyway. Getting this right across nodes is actually really tricky, so you may find the markRange function in our lib useful if you decide to use <mark>. There's some room for improvement (it probably uses more new nodes than it needs to) but so far I've found it to be pretty robust.

Re: collaborating, for boring lawyer-type reasons I can't pull your code into our lib myself, but I'd be thrilled to take a look at any pull requests you'd be interested in sending! Alternatively, if you want to add a dependency on our work in your repo, the bulk of the interesting logic for our lib is contained in a module for easy partial reuse, without needing to use the whole polyfill. I'm happy to expose more functions (e.g., markRange) as exports if that would be useful.

[Treora]

Regarding highlighting:

@tomayac wrote:

One limitation in your highlighting approach, apart from the spec's discouragement you mention in the comment, is that it can only highlight one item at a time.

I added a test for that to the playground. It seems to highlight multiple items fine, but I now discovered that indeed this works only in Firefox; see MDN. Bummer. So it would be good to add a <mark> based solution as a back-up. Or drawing an SVG on top of things, but that is more challenging (you’d need to react to every layout change).

@tfmar wrote:

Getting this right across nodes is actually really tricky, so you may find the markRange function in our lib useful if you decide to use <mark>.

I know it’s tricky, I once wrote a module called dom-highlight-range that is similar to your implementation! :) (I recently integrated it into Apache Annotator, and still have to figure out how/whether to keep the two places in sync)

[tilgovi]

#TIL about the CSS Highlights API proposal. This looks like exactly what we'd need. Do you know if this is still being actively pursued? The spec draft as well as all issues are from December 2019…

I went back through the mailing list and it seems like the last discussion was during a face to face meeting in January, where it was resolved to publish the first public working draft, which is what I linked.
https://lists.w3.org/Archives/Public/www-style/2020Feb/0012.html

@frivoal I hope you don't mind me tagging you in here. Anything else we should know about the highlights proposal?

[bokand]

Doing some issue hygiene. I'm going to close this since there's no action to take here - thanks for sharing! (feel free to reply if there is something TODO).