Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support TextPositionSelector (in the dom package) #75

Closed
Treora opened this issue May 3, 2020 · 3 comments
Closed

Support TextPositionSelector (in the dom package) #75

Treora opened this issue May 3, 2020 · 3 comments
Assignees
Labels
let’s code Things waiting to be implemented

Comments

@Treora
Copy link
Contributor

Treora commented May 3, 2020

Following its specification.

Altough it looks simple, there may be challenges in ensuring we count characters correctly. From the spec (in the TextQuoteSelector section, but that is then referred to by the TextPositionSelector section):

The selection of the text MUST be in terms of unicode code points (the "character number"), not in terms of code units (that number expressed using a selected data type). Selections SHOULD NOT start or end in the middle of a grapheme cluster. The selection MUST be based on the logical order of the text, rather than the visual order, especially for bidirectional text. For more information about the character model of text used on the web, see charmod.

The text MUST be normalized before recording in the Annotation. Thus HTML/XML tags SHOULD be removed, and character entities SHOULD be replaced with the character that they encode.

The referenced ‘charmod’ (Character Model for the WWW) has a section on string indexing that may be relevant.

What still confuses me a little is what constitutes the exact text of a DOM. Given that normalisation should (why not must?) remove html tags, I suppose this assumes we deal with the source html.

What then to do with comments: are those text, or are their <!-- and --> parts to be removed? In the latter case, would the document’s total text equal the textContent of all children of the Document? (one may think document.documentElement.textContent, but that excludes whitespace and comments outside the <html> element)

Possibly more problematic, can one even access the source html accurately enough through the DOM? Might a source parser have modified whitespace, thus leading to miscounts? I am not even talking about executed scripts that may modify the DOM too, I suppose we have to disregard that scenario.

Of course there are implementations already whose approach and behaviour we could copy, but it may be good to do the exercise of implementing based on the spec to ensure that it matches up, also to help detect discrepancies between implementations and spots where the spec may need to be improved/updated.

Any differences in implementations would likely result in misanchored annotations, so doing this imprecisely seems of little value; unless the use is explicitly limited to only apply to e.g. selector refinement within text nodes, which could be a strategy to take.

@tilgovi (or others): what are your thoughts about this, and about the implementation as it is done in dom-anchor-text-position, in Hypothesis, or elsewhere?

@tilgovi
Copy link
Contributor

tilgovi commented Aug 16, 2020

The text MUST be normalized before recording the annotation. Thus HTML/XML tags SHOULD be removed and character entities SHOULD be replaced with the character that they encode.

As long as we stick to textContent or innerText, we are covered here. The tags are not part of this and entities are already replaced.

Possibly more problematic, can one even access the source html accurately enough through the DOM? Might a source parser have modified whitespace, thus leading to miscounts? I am not even talking about executed scripts that may modify the DOM too, I suppose we have to disregard that scenario.

We should be fine, at least for text nodes. The CSS white space properties make it important that parsers preserve the text nodes as is.

I think the spec is still somewhat vague and open to interpretation. I am partial to using innerText because it's the closest to the actual presentation. Regardless of what we choose, we have some work that supports any decision and helps us handle characters with multiple code units.

Iterating over a string in JavaScript yields strings representing the code points (each iteration may yield a string with more than one code units). As a result, one can also do [...string] and get an array of the code points. If we write a generic text position selector in terms of iteration over code points then we can compose it with anything that generates such an iterator from any other source, like generating innerText from a DOM Node or Range. The simplest thing is to call it with [...string].

However, I think we should consider going a step further and writing a text selector that consumes an iterator that yields chunks rather than receiving a full text with the initial call. This interface would be useful for streaming scenarios where the whole text may not be available or may be extremely large. The chunks themselves could be arrays or strings, and if we decide that they are strings we may wish to iterate over their code points.

@Treora Treora self-assigned this Oct 8, 2020
@Treora
Copy link
Contributor Author

Treora commented Oct 8, 2020

I started implementing this in the text-position branch. I started by simply copying the code and tests from text quote selection, and adapting them as needed.

Treora added a commit that referenced this issue Dec 24, 2020
Implement TextPositionSelector, create Chunking abstraction.

Fixes #85, #75.
@Treora
Copy link
Contributor Author

Treora commented Dec 24, 2020

Implemented in #98.

@Treora Treora closed this as completed Dec 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
let’s code Things waiting to be implemented
Projects
None yet
Development

No branches or pull requests

2 participants