fix(search): keep a limit the caller wrote into the ask query - #161
Merged
Conversation
- semantic_search appended |limit= unconditionally, and SMW honours the last limit, so '[[Category:Item]]|limit=2' was sent as ...|limit=1000 - get_query_limit() reads the limit a query sets itself, ignoring conditions, printouts and printout parameters - the default is appended only to a query that sets no limit - the truncation warning now compares against the limit in force - 'limit=0' no longer warns about truncation
Contributor
Release previewMerging this PR would release v2.2.1 (current: Changelog preview (truncated)## v2.2.1 (2026-09-02)
### Bug Fixes
- **search**: Keep a limit the caller wrote into the ask query
([`0c75699`](https://github.com/OpenSemanticLab/osw-python/commit/0c756993bcf9a9114c3072ee03037757aaa5fcf5))
- **search**: Treat a SearchParam limit of None as no limit
([`6e94187`](https://github.com/OpenSemanticLab/osw-python/commit/6e94187c455942afdec2746a498c44c6fd5953e0))
Preview via python-semantic-release and conventional commits. |
- 'limit=None' appended the literal '|limit=None' and then raised TypeError on the truncation check - None now sends no limit parameter and leaves the wiki to apply its own - no truncation warning in that case, since no limit is in force
This was referenced Sep 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Noted as out of scope while probing #150.
The bug
semantic_searchappended the default limit unconditionally:SMW honours the last limit in the query string, so
[[Category:Item]]|limit=2went out as[[Category:Item]]|limit=2|limit=1000and the caller's limit was silently ignored. Verified live against a wiki: it returned 1000 results, not 2.Changes
get_query_limit(query): returns the limit an SMW ask query sets itself,Noneif it sets none.semantic_search: appendSearchParam.limitonly to a query that does not set its own. A query that does is sent unchanged.SearchParam.limit.limit=0no longer triggers that warning.SearchParam.limit=Nonenow means "apply no limit and leave it to the wiki", instead of raisingTypeError.Nonemeaning documented onSearchParam.limitand in thesemantic_searchdocstring.get_query_limit, 9 tests forsemantic_search, all offline.Why the truncation warning had to change with it
#150warns whenn >= query.limit. Once a caller's|limit=2is honoured, results are truncated at 2 whilequery.limitis still the default 1000, so2 >= 1000is false and the warning would never fire on exactly the queries most likely to hit their limit. The effective limit is now a local, used for both the append decision and the warning.limit=0follows from the same local:0 >= 0would warn "returned 0 results, which meets the requested limit of 0. Results are truncated" on every count-style query. A zero limit asks for no results, so meeting it says nothing about truncation.SearchParam.limit = None
limitis typedOptional[int], butNonewas never usable: it appended the literal|limit=Noneand then raisedTypeErroronn >= None.src/osw/data/import_utility.py:809-823works around it by building aSearchParamwithout the field at all when its ownlimitargument isNone.Nonenow means what the type suggests: send no limit parameter and leave the wiki to apply its own default. No limit is then in force, so no truncation warning is emitted, since the result count says nothing about completeness.Nothing can regress: every previous
limit=Nonecall raised.The
import_utilityworkaround is left in place. It is not equivalent to passingNonethrough: omitting the field yields the1000default, whileNonewould now yield SMW's own, much smaller default. Collapsing the branch would change how many results that caller gets.Parsing the limit out of a query
|is not a plain separator in SMW:||is the disjunction operator inside[[...]]conditions, so[[Has p::a||limit=2]]would split into a boguslimit=2]]segment. Conditions are removed before the parameters are split, and the parameter pattern is anchored:\[\[.*?\]\]can only remove a prefix of a nested condition, never text past a]], so it cannot swallow a following|limit=.^limitkeeps the printout|?limitand the printout parameter|?Has subobject|+limit=3from being read as the query limit. Both limit something other than the query.limit=with a non-numeric value returnsNone, so the default is appended after it....|limit=all|limit=1000is a well-formed parameter list and SMW takes the last, which is the behaviour before this change.Behaviour change
limit=now wins overSearchParam.limit. Previously theSearchParamvalue always won, which is the bug.SearchParam.limit.SearchParam.limit=Nonesends no limit instead of raising. No caller could have relied on the old behaviour.Not covered
The truncation warning still only catches truncation at the requested limit. SMW additionally caps every request server-side at
$smwgQMaxLimit, so a request above that cap is silently reduced and returns fewer results than asked for without any warning. Tracked in #162, which also sketches reading SMW's ownquery.meta.hasFurtherResultsflag instead of inferring truncation from the result count.Verification
Offline suite passes (156 passed, 1 skipped). No other call site in the repo builds an SMW ask string containing a limit, so nothing else changes behaviour.