optimize serving balloon-mapping and /home pages for high traffic

[jywarren]

Currently researching how to use NGINX: http://www.howtoforge.com/how-to-speed-up-drupal-7.7-with-boost-and-nginx-debian-squeeze-p3

To get actual IP addresses instead of the reverse proxy, we'll need to install rpaf: http://olex.openlogic.com/wazi/2011/add-a-nginx-reverse-proxy-to-your-lamp-setup/

[jywarren]

Also remove www.pub... and http://publiclab redundancy (see howtoforge article above, p2)

[rjcorwin]

That install doesn't look too hairy. I'm interested to see how it goes.
Are you working on it right now?

RJ Steinert
http://rjsteinert.com

On Wed, Apr 18, 2012 at 4:56 PM, Jeffrey Warren <
reply@reply.github.com

wrote:

Currently researching how to use NGINX:
http://www.howtoforge.com/how-to-speed-up-drupal-7.7-with-boost-and-nginx-debian-squeeze-p3

To get actual IP addresses instead of the reverse proxy, we'll need to
install rpaf:
http://olex.openlogic.com/wazi/2011/add-a-nginx-reverse-proxy-to-your-lamp-setup/

---

Reply to this email directly or view it on GitHub:
#147

[jywarren]

yep; i'm in the public chat room if you're around: publiclaboratory.org/chat

On Wed, Apr 18, 2012 at 5:11 PM, R.J. Steinert <
reply@reply.github.com

wrote:

That install doesn't look too hairy. I'm interested to see how it goes.
Are you working on it right now?

RJ Steinert
http://rjsteinert.com

On Wed, Apr 18, 2012 at 4:56 PM, Jeffrey Warren <
reply@reply.github.com

wrote:

Currently researching how to use NGINX:

http://www.howtoforge.com/how-to-speed-up-drupal-7.7-with-boost-and-nginx-debian-squeeze-p3

To get actual IP addresses instead of the reverse proxy, we'll need to
install rpaf:

http://olex.openlogic.com/wazi/2011/add-a-nginx-reverse-proxy-to-your-lamp-setup/

---

Reply to this email directly or view it on GitHub:
#147

---

Reply to this email directly or view it on GitHub:
#147 (comment)

[jywarren]

ok, i'm currently blocking hotlinked files from off domain, 'cause that's in the example nginx config. Reminder to self to re-enable that.

[jywarren]

ok running nginx at publiclaboratory.org:8088 but got:

2012/04/18 17:51:33 [error] 5660#0: *1 no live upstreams while connecting to upstream, client: 208.54.36.199, server: publiclaboratory.org, request: "GET /index.php HTTP/1.1", upstream: "http://localhost/index.php", host: "publiclaboratory.org:8088"
2012/04/18 17:52:38 [error] 5660#0: *9 connect() failed (111: Connection refused) while connecting to upstream, client: 208.54.36.199, server: publiclaboratory.org, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:80/", host: "publiclaboratory.org:8088"

maybe i have localhost access off for apache? working...

[jywarren]

i got nginx working and redirecting, but when i try to swap it so nginx is handling port 80 and apache on port 8088, it says port already in use... i think this means i'd have to swap over ALL apache sites to 8088, individually in the /etc/apache2/sites-available/* directory to all say 8088, and nginx would sit in front of them.

or I could try to make the /etc/apache2/sites-available/default file deal with everything except publiclaboratory.org:80

either way it'll take a little work to get going and I'm not really wanting to try it on the production server. Load times have dropped again as traffic has gone down so I want to test this out on a smaller service like SpectralWorkbench.org before really trying it on publiclaboratory.org.

[jywarren]

alternative: proxy everything through NGINX and run apache on port 8088

this means figuring out how to proxy each site to its respective hostname in /etc/nginx/sites-available/*?

multiple server entries seems dumb, can we wildcard? (http://www.besttechie.net/wp-content/uploads/nginx-example.conf)

[jywarren]

also someone says (http://wp-performance.com/2010/10/nginx-reverse-proxy-cache-wordpress-apache/):

#Probably not needed, as the proxy will pass back the host in "proxy_set_header"
server_name www.yoursite.com yoursite.com;
access_log /var/log/nginx/yoursite.proxied.log;

or wildcard:

http://nginx.org/en/docs/http/server_names.html

server {
    listen       80;
    server_name  *.*;
    ...
}

[rjcorwin]

Interesting stuff.

proxy everything through NGINX and run apache on port 8088

I wonder if this would end up being difficult having to tune
/etc/nginx/sites-available/default
for every site on the server...

if ($http_cookie ~ "DRUPAL_UID" )

Would have to be expanded. I wonder what else...

RJ Steinert
http://rjsteinert.com

On Thu, Apr 19, 2012 at 9:23 AM, Jeffrey Warren <
reply@reply.github.com

wrote:

also someone says (
http://wp-performance.com/2010/10/nginx-reverse-proxy-cache-wordpress-apache/
):

#Probably not needed, as the proxy will pass back the host in
"proxy_set_header"
server_name www.yoursite.com yoursite.com;
access_log /var/log/nginx/yoursite.proxied.log;

---

Reply to this email directly or view it on GitHub:
#147 (comment)

[jywarren]

i'm trying it now; still can't seem to allow connections to Apache via 127.0.0.1 or localhost... curl blablabla 127.0.0.1:80 fails

also:

2012/04/19 16:41:08 [error] 30686#0: *9 connect() failed (111: Connection refused) while connecting to upstream, client: 18.248.6.170, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:80/", host: "spectralworkbench.org:8088"
2012/04/19 16:41:09 [error] 30686#0: *9 open() "/opt/nginx/html/favicon.ico" failed (2: No such file or directory), client: 18.248.6.170, server: , request: "GET /favicon.ico HTTP/1.1", host: "spectralworkbench.org:8088"

when trying to get to spectralworkbench.org:8088

[jywarren]

almost there, getting to apache but need to get the virtual host config to actually direct it to the right web app. This has a lot of helpful stuff: http://www.dikant.de/2008/07/10/nginx-as-a-reverse-proxy-for-apache/

[jywarren]

finally, reverse proxying directly to rails apps: http://blog.phusion.nl/2010/09/21/phusion-passenger-running-multiple-ruby-versions/

[jywarren]

close to ready to go, but I want to do it from a hardline (i'm on a train) like in the Matrix. The nginx default entries look like this:

server {
        listen 80;
        server_name archive.publiclaboratory.org;
        location / {
                proxy_pass http://archive.publiclaboratory.org:8088;
                proxy_set_header Host $host;
        }
}

server {
        listen 80;
        server_name mapknitter.org;
        location / {
                proxy_pass http://mapknitter.org:8088;
                proxy_set_header Host $host;
        }
}

1. backup whole /etc/apache2/sites-available directory and /etc/nginx/sites-available/default to .bkp
2. change all instances of 8088 > 80 and 80 > 8088 in /etc/nginx/sites-available/default
3. change all instances of 80 > 8088 and 8088 > 80 in /etc/apache2/sites-available/*
4. take down apache & nginx, restart both
5. check each domain

I'd like to benchmark before and after, but that might be hard on a production machine. If we don't have more crazy server issues (we're on slashdot front page now and doing OK, i guess!) I can let the current system run for a couple days to let it even out, then do this swich. I'd hope to see a distinct drop in response time. In very rough tests, I've seen Nginx serve an HTML page between 60-100% faster than Apache. Nice.