Bug: google engine

[NoExplorer]

Version of SearXNG, commit number if you are using on master branch and stipulate if you forked SearXNG
Repository: https://github.com/searxng/searxng
Branch: master
Version: 2023.6.12

How did you install SearXNG?

I am using SearXNG through a public instance https://searx.be

What happened?

Search engine Google returns no results with the error Suspended: access denied when trying to search something. Accessing Google with its own link (https://www.google.com) or its alternatives (https://www.google.fi and/or https://www.google.se) works just fine, on any browser and device I've tried so far.

How To Reproduce

1. Visit the instance's page https://searx.be
2. Type something in the search box (anything) and click the Search button (Magnifying glass icon)

Expected behavior

Search engine returns results for the requested search query, without an error.

Screenshots & Logs

Additional context

If the browser could be an issue, I will list its configuration here:
Browser: LibreWolf 113.0.2-1
Add-Ons active: uBlock Origin , ClearURLs, NoScript Security Suite, Decentraleyes

I am getting errors for many other engines as well, for example:
DuckDuckGo, which returns no search results with error HTTP connection error
qwant, which returns no search results with error server API error
StartPage, which returns no search results with error Suspended: CAPTCHA (Solvable by keeping a tab of StartPage open alongside SearXNG.)

I've tried checking if it is a network error, but with every restart I did to my router I got no progress done.

Some other devices (i.e my Android phone) face the same issue as described above on the same network.

I hope I provided enough details and info, if not, do not hesitate to tell me.

Technical report

Error

• Error: httpx.TimeoutException
• Percentage: 0
• Parameters: (None, None, None)
• File name: searx/search/processors/online.py:118
• Function: _send_http_request
• Code: response = req(params['url'], **request_args)

Error

• Error: timeout
• Percentage: 0
• Parameters: ()
• File name: searx/search/processors/abstract.py:102
• Function: handle_exception
• Code: count_error(self.engine_name, exception_or_message)

Error

• Error: httpx.ReadTimeout
• Percentage: 0
• Parameters: (None, None, 'www.google.com')
• File name: searx/search/processors/online.py:118
• Function: _send_http_request
• Code: response = req(params['url'], **request_args)

Error

• Error: searx.exceptions.SearxEngineAccessDeniedException
• Percentage: 0
• Parameters: ('HTTP error 403',)
• File name: searx/search/processors/online.py:118
• Function: _send_http_request
• Code: response = req(params['url'], **request_args)

[return42]

Version: 2023.6.12

its not outdated

Percentage: 0

may be it was an temporary issue. If this happens again to you use another engine from https://searx.space/

@unixfox I close this issue / if you think different, please reopen.

[return42]

@NoExplorer what default language is set in your browser / what language is this ..

[return42]

@xbdmHQ please don't make any claims you haven't verified. Google works and the error rate is less than one percent.

searx.be is the biggest instance I know, it is maintained by @unifox for many years and he brings his experience and know-how to this project for many years.

[NoExplorer]

@NoExplorer what default language is set in your browser / what language is this ..

On LibreWolf, the default language is my system's language, English. Google might've pulled out Greek from the IP address or something

[NoExplorer]

google engine is broken on that search, seems to me the guy tried to change something and forgot how to fix it lol because the google engine works on most of the others including mines

I don't really change my settings for SearXNG frequently. I usually set it up and never touch it. If that's what you're talking about.

[unixfox]

I did receive two days in a row a wave of bots sending huge amount of requests at the same time and this got my 4 servers blocked by google.

I'm still investigating the issue because I'm 100% certain Google is doing some fingerprinting because the same HTTP request was working in a browser and turning on HTTP2 for the google engine fixed the issue.

Anyway @return42 it comes down to the same shared problem with the other engines, there is a need to reduce the bots requests (which you are working on :) ).

@NoExplorer this issue is only related to searx.be servers, you can't fix on your side. Like return42 said you are free to choose another instance if something doesn't work on an instance: https://searx.space

[NoExplorer]

I see, if that's the case I'll probably switch to another instance and wait for an update to this. Thank you for the help everyone!

Closing issue for now.

[return42]

Anyway @return42 it comes down to the same shared problem with the other engines, there is a need to reduce the bots requests (which you are working on :) ).

@unixfox: give Method ip_limit a try, all you need is a /etc/searxng/limiter.toml ..

[botdetection.ip_limit]
link_token = true

---

The method must run for a while to detect the bots ... the longer it runs the better the defense against the bots. On my instance this method is active since about 12 days .. my log is only 8 days and in this 8 days this method blocks ...

$ sudo -H journalctl -u "uwsgi@searxng" | grep -o 'BLOCK: .* SUSPICIOUS_IP_WINDOW' | wc -l 
12093

.. request from ..

$  sudo -H journalctl -u "uwsgi@searxng" | grep -o 'BLOCK: .* SUSPICIOUS_IP_WINDOW' | sort -u | wc -l
606

unique IPs.

• [mod] limiter: reduce request rates for requests without a ping #2357