Use searx.space to see if SearXNG instances are online

[return42]

The implementation of farside instances do a search request on each SearXNG instance to see if the instance is online and does it work.

From the view of a SearXNG instance, the sum of the farside instance behave like a bot net that has been gone wild / doing the same query:

farside/config/config.exs

Lines 16 to 18 in 6048d3c

	queries: [
	"weather",
	"time"

.. again and again, 24/7 from different IPs. In SearXNG we tend to block wild bots since they will have a negative impact on the engines of the SearXNG instance when we pass trough these requests.

My suggestion:

To avoid this flood of requests on each SearXNG instance, the farside instances can use the list from the SearXNG team to see what instances are online --> https://searx.space/

May the farside team has more / other suggestions to improve the situation / Thank you for your understanding.

[benbusby]

Understandable. I'll just remove the periodic polling for SearXNG and update the nightly build to factor in instance availability when generating the instance list.

[unixfox]

We can also increase the frequency of checking the health of each instance. @dalf proposed to do that, I'm not what's the current frequency (one per hour)?