Bug: duckduckgo engine

[samsapti]

Version of SearXNG, commit number if you are using on master branch and stipulate if you forked SearXNG
Repository: https://github.com/searxng/searxng
Branch: master
Version: 2023.5.10+cb1c3741
Instance: https://search.sapti.me

How did you install SearXNG?

Via Docker.

What happened?

This is a new one. DuckDuckGo is the new Qwant (#2011). It reports server API error, and it has been pretty consistent over the last couple of hours.

How To Reproduce

1. Go to SearXNG
2. Search for something
3. Observer error message

Expected behavior

DuckDuckGo should return results.

Screenshots & Logs

From docker logs searxng after performing a search:

2023-05-12 01:54:41,579 WARNING:searx.engines.duckduckgo: ErrorContext('searx/engines/duckduckgo.py', 101, "raise SearxEngineAPIException('Request failed')", 'searx.exceptions.SearxEngineAPIException', None, ('Request failed',)) False
2023-05-12 01:54:41,579 ERROR:searx.engines.duckduckgo: exception : Request failed
Traceback (most recent call last):
  File "/usr/local/searxng/searx/search/processors/online.py", line 165, in search
    search_results = self._search_basic(query, params)
  File "/usr/local/searxng/searx/search/processors/online.py", line 139, in _search_basic
    self.engine.request(query, params)
  File "/usr/local/searxng/searx/engines/duckduckgo.py", line 241, in request
    params['data']['vqd'] = get_vqd(query, params["headers"])
  File "/usr/local/searxng/searx/engines/duckduckgo.py", line 101, in get_vqd
    raise SearxEngineAPIException('Request failed')
searx.exceptions.SearxEngineAPIException: Request failed

Additional context

My settings.yml have the following suspended_times values:

suspended_times:
  SearxEngineAccessDenied: 0
  SearxEngineCaptcha: 600
  SearxEngineTooManyRequests: 600
  cf_SearxEngineCaptcha: 600
  cf_SearxEngineAccessDenied: 1200
  recaptcha_SearxEngineCaptcha: 600

Technical report

Error

• Error: searx.exceptions.SearxEngineAPIException
• Percentage: 25
• Parameters: ('Request failed',)
• File name: searx/engines/duckduckgo.py:101
• Function: get_vqd
• Code: raise SearxEngineAPIException('Request failed')

Error

• Error: searx.exceptions.SearxEngineAccessDeniedException
• Percentage: 5
• Parameters: ('HTTP error 403',)
• File name: searx/search/processors/online.py:118
• Function: _send_http_request
• Code: response = req(params['url'], **request_args)

Error

• Error: timeout
• Percentage: 0
• Parameters: ()
• File name: searx/search/processors/abstract.py:102
• Function: handle_exception
• Code: count_error(self.engine_name, exception_or_message)

Error

• Error: httpx.ReadTimeout
• Percentage: 0
• Parameters: (None, None, 'lite.duckduckgo.com')
• File name: searx/search/processors/online.py:118
• Function: _send_http_request
• Code: response = req(params['url'], **request_args)

Error

• Error: httpx.TimeoutException
• Percentage: 0
• Parameters: (None, None, None)
• File name: searx/search/processors/online.py:118
• Function: _send_http_request
• Code: response = req(params['url'], **request_args)

[KalyaSc]

Hi, have the exact same error on a new searxng installation with docker. Everything works except DDG.

I went a bit through the python scripts and the error is raised

if content.find('vqd='') == -1

Whatever it means, that's the furthest I get. Out of my scope and my knowledge. Docker is maybe the common factor here, because the public searxng instances are doing great !

[return42]

I went a bit through the python scripts and the error is raised

thanks 👍

Docker is maybe the common factor here, because the public searxng instances are doing great !

I don't think so, IMO its a question of time all servers will start to fail .. I will look at this issue ASAP .. but for know I have to stay at work :-( .. sorry for the delay ..

[samsapti]

Still ongoing, even after restarting SearXNG.

[return42]

Send #2430 .. I have not yet had time for serious testing --> testers are welcome 😄

[KalyaSc]

Hey ! That did the trick ! Thank you ! 👍

edit: logs are also clean and no errors

{ I have no idea how such small/"random" changes can make a difference... seams like magic ! }

[samsapti]

@return42 that fixed it. Thanks!