[fix] engine woxikon.de - don't raise exception on empty result list

[return42]

What does this PR do?

[fix] engine woxikon.de - don't raise exception on empty result list

Suggested-by: @allendema [1]

[1] #1543 (comment)

Why is this change important?

Woxikon expects a word in German, so with query "foo" the site finds nothing and respons a 404:

httpx.HTTPStatusError: Client error '404 Not Found' for url 'https://synonyme.woxikon.de/synonyme/foo.php'

How to test this PR locally?

 !woxikon.de_synonyme foo

There should not be an 404 error any longer.

Related issues

Closes: #1543

[unixfox]

Will it raise an exception if the status code is higher than 500 though? Because that's useful for knowing if the server is down for example.

Same goes for 429 if the server rate limit SearXNG.

[return42]

Will it raise an exception if the status code is higher than 500 though?

No ..

searxng/searx/network/network.py

Lines 239 to 247 in 6f28a69

	def patch_response(response, do_raise_for_httperror):
	if isinstance(response, httpx.Response):
	# requests compatibility (response is not streamed)
	# see also https://www.python-httpx.org/compatibility/#checking-for-4xx5xx-responses
	response.ok = not response.is_error
	# raise an exception
	if do_raise_for_httperror:
	raise_for_httperror(response)

I think the intention of raise_for_httperror was to implement a handle in the response function / by example we have one in the wikipedia.py:

searxng/searx/engines/wikipedia.py

Lines 56 to 69 in 6f28a69

	if resp.status_code == 404:
	return []
	if resp.status_code == 400:
	try:
	api_result = loads(resp.text)
	except:
	pass
	else:
	if (
	api_result['type'] == 'https://mediawiki.org/wiki/HyperSwitch/errors/bad_request'
	and api_result['detail'] == 'title-invalid-characters'
	):
	return []

The drawback of a XPath engine (woxikon) is, that we can't implement a response handle.

[unixfox]

You don't get a status code like in https://github.com/searxng/searxng/pull/1682/files#diff-a34ea98196211e28f061d63a43f8056671e861d505000f965f341ea080fdbf13R61?

[return42]

Not sure I understand your question right ..

XPath engines do not have a dedicated python module (to implement a engine specific HTTP error handle) and the raise_for_httperror: false is a bolean (in the settings.yml).

Do you suggest a different implementation for XPath engines, to fetch only configured HTTP status code .. e.g.:

  - name: woxikon.de synonyme
    engine: xpath
    ignore_http_status: 
      - 404
      - xxx

@dalf what do you think about a ignore list for XPath engines?

[unixfox]

Oops I didn't see that it wasn't a dedicated python file for the engine.

Your solution with ignore_http_status seems a good IMO.

[dalf]

I agree it should be implemented in xpath.py.

By ignore_http_status you mean the error is ignored, and the result is parsed?
Yes, go.

Otherwise if xpath.py returns an empty result for these HTTP status code, some alternative names : http_status_as_no_result, no_result_for_http_status without strong conviction.

[return42]

and the result is parsed?

not parsing; the first intention was not to raise an exception on empty result lists. Some engines (like woxikon here in this PR) return a "404 Not Found" when they do not have a match for the search term. I think it is good to have an option in the configured xpath engine to fetch some HTTP status codes and return an empty result list.

http_status_as_no_result, no_result_for_http_status without strong conviction.

True, the name could be improved, the ones you suggested I still do not like so much, but my suggestions are also not better: no_match_for_http status

[dalf]

I've added a commit so unexpected HTTP statuses are not hidden.

[return42]

I've added a commit so unexpected HTTP statuses are not hidden.

Thanks a lot .. I rebased and going to merge after checks from CI have been passed.