Skip to content

Add normalize whitespace built-in filter#1260

Merged
clementbiron merged 6 commits into
mainfrom
add-normalize-whitespace-filter
Jul 7, 2026
Merged

Add normalize whitespace built-in filter#1260
clementbiron merged 6 commits into
mainfrom
add-normalize-whitespace-filter

Conversation

@clementbiron

Copy link
Copy Markdown
Member

Some documents use Unicode space separators (non-breaking space U+00A0, narrow no-break space U+202F, ideographic space U+3000, ...) instead of regular spaces. These characters are visually identical to a normal space but differ byte-for-byte, which produces blink when tracking terms.

This changeset adds a normalizeWhitespace built-in filter that replaces every Unicode space separator (\p{Zs}) with a regular space in the text nodes of the page.

@clementbiron
clementbiron requested a review from Ndpnt July 2, 2026 12:24

@MattiSG MattiSG left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exciting to see built-in filters expanding!

Comment thread src/archivist/extract/exposedFilters.test.js Outdated
Comment thread src/archivist/extract/exposedFilters.test.js Outdated
Comment thread src/archivist/extract/exposedFilters.test.js Outdated
Comment thread src/archivist/extract/exposedFilters.js Outdated
@clementbiron
clementbiron requested a review from MattiSG July 6, 2026 09:18
@clementbiron
clementbiron force-pushed the add-normalize-whitespace-filter branch from 6c94b29 to 0255556 Compare July 7, 2026 09:10
@clementbiron clementbiron linked an issue Jul 7, 2026 that may be closed by this pull request
@clementbiron
clementbiron force-pushed the add-normalize-whitespace-filter branch from 2616eae to 6c583ca Compare July 7, 2026 12:23
@clementbiron
clementbiron force-pushed the add-normalize-whitespace-filter branch from 6c583ca to a2f0819 Compare July 7, 2026 12:47
@clementbiron
clementbiron merged commit 97b7b41 into main Jul 7, 2026
8 checks passed
@clementbiron
clementbiron deleted the add-normalize-whitespace-filter branch July 7, 2026 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Harmonize whitespaces

2 participants