fix: handle empty SMW ask results in semantic_search - #150
Conversation
- SMW serialises an empty ask result set as a JSON array, not an object, so any zero-result query raised AttributeError instead of returning [] - normalise the payload once via _ask_results_as_dict() - warn when a query hits its limit, which silently truncated results - warn when entries are dropped by the exists != "1" filter - refs #145, #111
Release previewMerging this PR would release v2.0.3 (current: Changelog preview (truncated)## v2.0.3 (2026-08-31)
### Bug Fixes
- Handle empty SMW ask results in semantic_search
([`e00ca8b`](https://github.com/OpenSemanticLab/osw-python/commit/e00ca8be80eb1575a520b609d5d988604ad59e24))
### Testing
- Rename oold.py to oold_test.py so its tests are collected
([`20072a9`](https://github.com/OpenSemanticLab/osw-python/commit/20072a9249cd97126a222c62a70f84e0433343ef))
Preview via python-semantic-release and conventional commits. |
|
The out-of-scope item noted here, That PR also adjusts the truncation warning added here: it compared One limitation of the warning is left open and tracked in #162: SMW caps requests server-side at |
Fixes the crash behind #145 and explains the "random results" in #111.
Root causes
Three separate defects, all reproduced live against
healthbatt.projects01.open-semantic-lab.org.1. Zero-result queries raise
AttributeError(fixed here)SMW's
action=askserialises a non-empty result set as a JSON object keyed by page title, but an empty one as a JSON array.semantic_searchcalled.values()on it unconditionally (src/osw/wiki_tools.py:257), so every zero-result query raisedAttributeError: 'list' object has no attribute 'values'instead of returning[].Affects single queries, sequential batches and parallel batches.
return_json=Truehappened to survive because it returns before that line.OSW.query_instances()(src/osw/core.py:2007) is affected too, so querying a category with no instances raised instead of returning an empty list.2. Silent truncation at
limit(surfaced here)SearchParam.limitdefaults to 1000 and there is no pagination, so a larger match set is silently reduced to an arbitrary subset. Measured on healthbatt:[[Category:Item]]matches 2637 pages,semantic_searchreturned 1000 with no indication.3.
existsflickers server-side (surfaced here, not fixable in this library)The
exists == "1"filter silently discards results. The field only ever takes'1'or'', but it is not stable: across 8 identical calls, 8 pages flipped between the two, and no page was consistently''. The flips are perfectly correlated across pages (calls 0,2,3,4,7 agree; calls 1,5,6 agree), which points at inconsistent state between backends or an SMW query-result cache rather than at real page deletions. Consecutive identical calls returned 998 or 994 titles.This is the "random results" from #111. It needs to be addressed on the OSL instance side; this PR only makes the loss visible.
Changes
_ask_results_as_dict()normalises theaskpayload to a mapping, so an empty result set yields[]instead of raisingwarnings.warnwhen the result count meets the requested limit, emitted before thereturn_jsonearly return so both return modes get itwarnings.warnwhen entries are dropped by theexists != "1"filter, naming how manyFilter semantics, the default limit, and
prefix_searchare unchanged.Verification
pytest tests/ --ignore=tests/integration-> 61 passed, 1 skippedAttributeError/DID NOT WARNreturn_jsonwith a zero-result query all return results now instead of raising; both warnings fire with real countsOut of scope, noticed while probing
single_query += f"|limit={query.limit}"is appended unconditionally, so a caller-supplied limit in the query string is silently overridden:[[Category:Item]]|limit=2is sent as[[Category:Item]]|limit=2|limit=1000and SMW honours the last one. Verified live (returned 1000, not 2). Not changed here.