HTTP headers #715
Comments
IMHO it would be better to have the web servers create the headers if you want to have additional headers (like key pinning stuff). But because Apache/mod_uwsgi does not work with header add I hard code my headers in webapp.py by now which is not very easy because I need to edit webapp.py with every git pull. Is there an mod_proxy_uwsgi description for searx? mod_proxy_uwsgi makes web server side headers possible. |
A solution about hard coded headers in webapp is settings.yml. But it would be messy to have header inside settings.yml and inside the HTTP server. |
after reverse proxying my uwsgi <-> apache connection by http all my headers are in apache now. So updating is easier now because there is no webapp.py patching anymore. Headers in apache is also good if you are intercepting by filtron (if it works) because then you do not have problems with loosing headers. |
It would be good to have decent defaults for http headers, but I'm not sure if the searx webapp is the right place for these headers. What if someone wants to change it for some reason? Should we add a configuration section for HTTP headers? Also, there is a possibility that adding new default headers may break someones existing configuration (CSP). As a backward compatible solution, I'd suggest to do not turn these headers on by default, but allow admins to turn it on from What do you think? |
I'm afraid that off by default means "it doesn't exist" most of the time: if these headers would break something we would have feedback, but there are mostly seat belts (except the Cache-Control), so most of the time even turn off everything will be look good (same for a wrong TLS configuration). But as you said, if some headers are enabled by default, it may conflict with some existing configurations. The only way I see is a two phase activation:
Can be related to searx/searx-instances#70 :
Does it worth it ? I'm not sure where to put the cursor. Looking at https://github.com/searx/searx-docker/blob/master/Caddyfile (not updated since last year, may be it requires some updates ? may be some values are broken ?). CSPdocumentation: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy value in searx-docker: https://github.com/searx/searx-docker/blob/7ad4d4903d0ce41de4094d5fe0fecda156b41eaa/Caddyfile#L83 The value of this header depends on the searx code (tile.openstreetmap.org hostname etc...) I think a default value would be good : currently we can't change this value without issue. X-XSS-ProtectionThe header seems deprecated (replaced by CSP) : https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-XSS-Protection To keep the backward compatibility for some old browsers, we can keep this header. X-Frame-OptionsThe header seems deprecated (replaced by CSP) : https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options To keep the backward compatibility for some old browsers, we can keep this header. X-Content-Type-Optionshttps://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Content-Type-Options Not deprecated, seems good whatever the usage context Feature-Policyhttps://developer.mozilla.org/en-US/docs/Web/HTTP/Feature_Policy It is a seat belt: searx doesn't use these features, if for some reason there is an XSS, at least this header will block usage of these features. Here, my take is HTTP/2 allows to send a reference to some previous headers so it doesn't cost anything to add this header. Referrer-Policyhttps://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referrer-Policy Seems good, it is redundant with the templates : searx/searx/templates/oscar/base.html Line 10 in db703a0
X-Robots-TagAccording to https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers : Cache-ControlRelated to
I think a default value would be good at least for the static files ? I'm not sure what to do for the index and preferences page ( searx-docker set the value of "no-cache, no-store", but some instances doesn't set this value. |
Good points. What about adding the headers that most probably can't cause backward incompatibility and creating a warning for adding the rest in the future? |
Except Cache-Control for the /index and /preferences pages, I would declare all these headers in settings.yml , section The main issue is the duplicate headers: the reverse proxy most probably will add or replace some headers too. Replace should not be a problem (but it produces an expect configuration), add may create a wrong / unsecure configuration.
Adding headers in settings.yml is related to #1209 : settings.yml won't be updated blindly if it contains some specific configuration (like the secret key, the engine tokens, etc...) |
[enh] add default http headers - closes #715
searx includes contents from others websites. The contents are escaped, trimed and all, but there is always a risk. The security could be improve using HTTP headers.
Some headers that could be added :
X-Frame-Options: SAMEORIGIN
when there is the image proxy (break the included videos, in this case a more precise).
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options
X-Content-Security-Policy "default-src 'self'; img-src 'self'; style-src 'self' 'unsafe-inline'; font-src 'self'; script-src 'self' 'unsafe-inline'; connect-src 'self';"
to be adjust when there are images, videos
https://content-security-policy.com/
https://report-uri.io/
X-Content-Type-Options: nosniff
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Content-Type-Options
X-XSS-Protection "1; mode=block"
https://blogs.msdn.microsoft.com/ieinternals/2011/01/31/controlling-the-xss-filter/
X-Download-Options: noopen
https://msdn.microsoft.com/fr-fr/library/jj542450(v=vs.85).aspx#noOpenDirective
more or less out of topic
X-Robots-Tag: noindex, nofollow
http://robots-txt.com/x-robots-tag/
avoid to be indexed and then spam (useless for that purpose ?)
HSTS header ( see #62 )
https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security
https://hstspreload.appspot.com/
Should nginx / apache send the headers or should the python code should send them ? in case of X-Content-Security-Policy and X-Frame-Options the values can change according to the search options.
The text was updated successfully, but these errors were encountered: