fix(search): sort a document that has no title in the reader’s language - #792
Merged
Conversation
- fall back, per locale, to the document’s first declared-locale value when it carries no value in that language, so every sort key a query can name is populated whenever the field holds anything at all - leave the search companions per-locale: a query fans out over every locale key at once, so only the sort – which names a single key – needs the fallback - document the rule and the reason under Locales in the search reference
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sorting a collection by a localized text field returned it unsorted whenever the
documents were not titled in the reader’s language.
A
sortabletext field wrote a sort key only for the languages a documentactually carried: a work with an untagged title got
name_sort_undand nothingelse, one titled in English got
name_sort_enand nothing else. A sort, unlikea search, names a single key – the query compiler picks the active locale’s.
So
orderBy: {field: NAME, direction: DESC}from a Dutch client compiled toname_sort_nl:desc, on which every document without a Dutch title carries theempty string. They all tie, and Typesense hands them back in relevance order –
which is exactly what an unsorted result looks like.
Each locale’s sort key now falls back to the document’s first value in
localesorder, so a sort key is present whenever the field holds any value in a declared
locale. A collection of untagged titles sorts by title in a Dutch request rather
than not at all; a work titled only in English sorts under its English title
next to the Dutch ones. Ordering across languages is approximate by nature, and
a total order over the titles a reader actually sees beats a partial one over
the tagged few.
The search companions deliberately keep no fallback: a query fans out over every
locale key at once, so a document is already found whichever language the reader
asks in, and copying the value would only inflate the index.
Reported downstream at https://codeberg.org/limburg/lol/issues/158, on a
register whose
CreativeWorktitles are predominantly untagged.Changes
packages/search/src/project.ts–applyTextcomputes the values perdeclared locale once, and fills each locale’s sort key from the first
non-empty bucket when its own is empty. Search keys are unchanged.
packages/search/test/project.test.ts– a document titled only in Englishsorts under that title in every declared locale, while its search keys stay
per-locale.
packages/search-pipeline/test/registry-extraction.integration.test.ts– theprojected dataset now also carries
label_sort_und, the fallback for arequest that states no language.
docs/reference/search.md– the Locales section states the fallback, why asort needs one where a search does not, and that a sort field is absent only
when the field is.