Update Caddyfile to Add X-Real-IP to fix bot detection

[ChillyKitty]

Bot detection complains about not having X-Real-IP which is required https://docs.searxng.org/src/searx.botdetection.html#id4 this adds the header and fixes the problem for me.

[return42]

@ChillyKitty thanks a lot for adding X-Real-IP to the caddy config. I am not a docker expert and it would take too long to build my own testbed. So I'll just ask you directly, just to be sure:

a.) In the bot detection, the remote IP of the request is taken from (first match):

• X-Forwarded-For
• X-Real-IP
• flask.Request.remote_addr

Its documented here in the searx.botdetection.get_real_ip function.

b.) From the Caddy docs: .. sets or augments the X-Forwarded-For header field.

Can you confirm that X-Real-IP and X-Forwarded-For hold the same value (IP).

[return42]

related:

• ERROR:searx.botdetection: X-Real-IP header is not set! #174

[dalf]

the remote IP of the request is taken from (first match):

as far I can understand from the code, both X-Forwarded-For and X-Real-IP are required.

[return42]

as far I can understand from the code, both X-Forwarded-For and X-Real-IP are required.

not really

 request_ip = forwarded_for or real_ip or remote_addr or '0.0.0.0'

but the code write log messages:

• error: when X-Forwarded-For and/or X-Real-IP is missed
• warning: when the headers have contradictory values.

Reference: get_real_ip(request: flask.Request)

[dalf]

This log makes X-Real-IP mandatory de facto:
https://github.com/searxng/searxng/blob/4b42644579581699ad1d4b7974610933277dd021/searx/botdetection/_helpers.py#L104-L105

This other log makes X-Forwarded-For mandatory de facto:
https://github.com/searxng/searxng/blob/4b42644579581699ad1d4b7974610933277dd021/searx/botdetection/_helpers.py#L95-L96

By de facto, I mean the user has to set these two headers to avoid tons of errors.

[return42]

By de facto, I mean the user has to set these two headers to avoid tons of errors.

Ah, OK .. sorry my slow comprehension .. now I understand what you mean / in my understanding of required I thought about the limiter works or not .. was not thinking about the massive error context 🤦

We can reduce the tons of error logs similar to what you suggested in #174 (reply in thread) .. I send PR

• [fix] reduce error logs related to X-Forwarded-For & X-Real-IP headers searxng#2681

.. does the PR mitigate the problem? / .. lets have further discussions in the PR.