New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HTTP/S Proxy support with Yacy Server #36
Comments
This would be useful for me as well. I am running yacy behind an nginx proxy and cannot fully contribute as a senior node. |
Hmmm, when I first setup my server the above was true. A few hours later it was showing itself to be running in senior mode, so I guess it's working fine behind the nginx proxy. |
@maxolasersquad correct me if I am wrong, but as I understand it, @sudheesh001 was talking about proxying outgoing http connections from YaCy (acting as a client, when crawling for example), whereas you are describing your YaCy peer as running behind nginx acting as a reverse proxy/gateway (receiving incoming connections first and transmitting them to your Yacy peer). |
I think you are correct. When my server was running in junior mode I assumed it was because of the nginx proxy and went searching the issues. I originally thought this was it. Since my server began running in senior mode after a while its clear that yacy had no issue running behind an nginx server. |
Absolutely @luccioman . In this issue, I was basically referring to the ability for the yacy clients to communicate when behind a proxy by enabling the http_proxy and https_proxy environment variables. Thank you @maxolasersquad for taking your time to deploy the server behind an nginx proxy. Please feel free to share the nginx configuration as an update to the documentation for running Yacy peers for others in the community who might want to setup Yacy behind an Nginx proxy. 👍 |
Hello! I have also installed YaCy 1.92 docker version listening at port 8090 behind of my nginx, public https port 443 (http port 80 redirects to https), and looking for the right way to configure YaCy. In If I change the ports at Enabling Any guidelines since the wiki is still offline? getting Thank you in advance! |
@ehehdada the fact that you can not modify the listening port to something else than 8090 is indeed a limitation specific to the YaCy Docker image which is configured to expose only the 8090 and 8443 ports. That's right that it should be better documented. |
I am trying to expose the yacy server through an nginx proxy. Basically my whole web servers front end is nginx and it routes the requests for single (sub)domains to different docker containers. This works fine for nextcloud and others but while yacy works it complains about not being able to connect to other yacy servers. When I set the actual "world" port numbers 80/443 then yacy won't start.. |
No dice, I had to make a hard route from the internet to the yacy docker container. If anyone can tell me how to configure yacy to work behind a nginx proxy I'm all ears. |
Wish I could, I had to do the same thing. |
It should work this way:
However, this does not re-route the port number. Right now the demo peer at https://yacy.searchlab.eu/ appears as senior but is doing it differently
The same should be possible for YaCy in docker containers, just expose their port to the docker host and then set the staticIP inside the container to the host IP address. |
Static IP is set. As said, when routing the ports to any other outside ports then yacy can't federate, when I expose the default ports to the world it works... Gonna re-check my firewalls etc. |
Most other web applications I've admistered offered a setting like "public URL" or something like that where I could set the full public URL (protocol, host and port), where YaCy only offers that staticIP setting. It's really weird that YaCy only lets you change the domain part and expects the port to always be the same one it listens on. |
solved that: server {
listen 80;
return 301 https://$host$request_uri;
}
server {
listen 443;
server_name my.domain.name;
ssl_certificate /etc/letsencrypt/live/my.domain.name/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/my.domain.name/privkey.pem; # managed by Certbot
ssl on;
ssl_session_cache builtin:1000 shared:SSL:10m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
ssl_prefer_server_ciphers on;
access_log /var/log/nginx/yacy.access.log;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Fix the “It appears that your reverse proxy set up is broken" error.
proxy_pass http://10.8.0.5:8090;
proxy_read_timeout 90;
proxy_redirect http://10.8.0.5:8090 https://my.domain.name;
}
}
server {
listen 8090;
server_name my.domain.name;
access_log /var/log/nginx/yacy.access.log;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Fix the “It appears that your reverse proxy set up is broken" error.
proxy_pass http://10.8.0.5:8090;
proxy_read_timeout 90;
}
}
yacy: Sytem Administration->Server access settings:
Use case & Accounts:
UPD (2023-03-09): this still works, but it takes 10-15 minutes for the yacy web interface to show the senior status. |
From what I see, #440 does nothing for the issue, specifically in a container. I place in port 443 (https only), but it keeps expecting from port 8090. |
you cannot bind to 443 port under unix/linux, unless you're root. that's just a remark, not sure, if it's related to your problem. nginx config was discussed in the forum as well |
Nope, YaCy was configured to use port 8090. The public port was 443 under
They didn't "solve" the issue. They simply exposed port 8090 to the outside. |
The port setting just changes the port in the config, you have to manually bind a proxy (like nginx) to that port and forward/pass traffic to 8090.
What it does is allow you to have a proxy (like nginx) handling the public port with TLS and then forward to 8090 (the port yacy is listening on) internally. This is regardless of the deployment method, you can also do this using containers. As can be seen in the help text:
So if you want to have NGINX handling your TLS stuff and users expect to type "https://somedoiman.com" into the browser and see yacy, you'd have to make NGINX do a |
I am aware of this, and this is what I've done. Apache Proxy directs port 443 -> local host:23243 23243 is the port exposed by the container, internally that maps to 8090. And it didn't work. |
I get the same impression that it's not possible to properly define an external accessible IP and port for all four variations - IPv4, IPv6, HTTP and HTTPS. |
Can someone give me a hint, I probably missed it in the discussion. How can I crawl with YaCy when I am behind a proxy? The environment variables I can't find the proxy setting (Connect through proxy, I don't want to use the YaCy proxy itself) in the admin settings anywhere. |
Thanks to https://wiki.yacy.net/index.php/De:YaCy-Tor#Konfigurationsdateien_.C3.A4ndern I found the respective settings ( |
Hi, huricanefrog, and thanks for feedback! I'll include that in the FAQ. How can I crawl with YaCy when I am behind a proxy? Set up proxy settings in configuration file
would be sufficient? |
@okybaca |
great, thanks!
i'll include the UI settings too.
|
added to faq in yacy/yacy_net_homepage#30 |
might be the environment variables already present in the system, there should be a way to tell Yacy to use internet behind a proxy environment or respect the http_proxy settings.
The text was updated successfully, but these errors were encountered: