New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nginx timeout : timeout issues #122

Closed
jmls opened this Issue Mar 13, 2015 · 13 comments

Comments

Projects
None yet
5 participants
@jmls
Copy link

jmls commented Mar 13, 2015

I have having timeout problems with nginx. After looking through the config I see a problem

my docker container was started up with -e VIRTUAL_HOST=foo.bar.com
docker ps gives me 971c26986bb3 [snip] 0.0.0.0:49156->3000/tcp my_app
docker inspect 971c26986bb3 gives me

"NetworkSettings": {
"Bridge": "docker0",
"Gateway": "172.17.42.1",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "172.17.0.78",

curl -H 'Host: foo.bar.com' http://localhost gives me a timeout
curl -H 'Host: foo1.bar.com' http://localhost gives me a 503 Service Temporarily Unavailable (as expected)

curl -H 'Host: foo.bar.com' http://localhost:49156 gives me the expected data (so proving that the server is running on 49156->3000)

so, what am I doing wrong ?

# If we receive X-Forwarded-Proto, pass it through; otherwise, pass along the
# scheme used to connect to this server
map $http_x_forwarded_proto $proxy_x_forwarded_proto {
  default $http_x_forwarded_proto;
  ''      $scheme;
}

# If we receive Upgrade, set Connection to "upgrade"; otherwise, delete any
# Connection header that may have been passed to this server
map $http_upgrade $proxy_connection {
  default upgrade;
  ''      '';
}

gzip_types text/plain text/css application/javascript application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

log_format vhost '$host $remote_addr - $remote_user [$time_local] '
                 '"$request" $status $body_bytes_sent '
                 '"$http_referer" "$http_user_agent"';

access_log /proc/self/fd/1 vhost;
error_log /proc/self/fd/2;

# HTTP 1.1 support
proxy_http_version 1.1;
proxy_buffering off;
proxy_set_header Host $http_host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $proxy_connection;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $proxy_x_forwarded_proto;

server {
        listen 80 default_server;
        server_name _; # This is just an invalid value which will never trigger on a real hostname.
        return 503;
}

upstream foo.bar.com {
                   # my_app
                   server 172.17.0.78:3000;
}

server {
        server_name foo.bar.com;

        location / {
                proxy_pass http://foo.bar.com;

        }
}

@jmls jmls changed the title nginx timeout : wrong ip nginx timeout : timeout issues Mar 13, 2015

@jwilder

This comment has been minimized.

Copy link
Owner

jwilder commented Mar 13, 2015

Any errors in the nginx-proxy logs? What does docker logs show?

@jmls

This comment has been minimized.

Copy link
Author

jmls commented Mar 13, 2015

oohh. I just tried curl -H 'Host: foo.bar.com' http://localhost and got

nginx.1 | 192.168.99.222 - - [13/Mar/2015:19:46:31 +0000] "GET / HTTP/1.1" 499 0 "-" "curl/7.37.1" "- in docker log ngProxy

what's 499 when it's at home ? aww crap. it's there because I ctrl-c'd

still, at least it shows that ngproxy was "processing" the request

@jmls

This comment has been minimized.

Copy link
Author

jmls commented Mar 13, 2015

doing a little more digging. I've restarted my machine and all docker instances. still no joy. So tried docker exec -t -i ngproxy /bin/bash and from the command line catted /etc/nginx/conf.d/default.conf

confirmed that foo.bar.com is present. ip address is 172.17.0.4

from the same command line (ie within nginx) tried wget 172.17.0.4:3000 timeout.
tried to ping 172.17.0.4, 100% packet loss

logged out of nginx.

from my normal command line I tried the same wget - and it worked, downloaded the website . ping also works just fine

so, it certainly seems as if there is a problem with nginx / docker

@jmls

This comment has been minimized.

Copy link
Author

jmls commented Mar 19, 2015

have you had any thoughts on this ? I still seem to be having the problems ... ;(

@jwilder

This comment has been minimized.

Copy link
Owner

jwilder commented Mar 19, 2015

Have you tried starting up a regular ubuntu container and just wget 172.17.0.4:3000 to see if it's reachable? That might help determine whether this is an nginx-proxy issue or something specific to your host/docker setup.

@jmls

This comment has been minimized.

Copy link
Author

jmls commented Mar 19, 2015

root@7aef07ef495b:/# curl 172.17.0.4:3000
curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection timed out

@jwilder

This comment has been minimized.

Copy link
Owner

jwilder commented Mar 19, 2015

I think there is something with your host setup. Maybe firewall rules or docker is running in a way that prevents inter-container communication.

@jmls

This comment has been minimized.

Copy link
Author

jmls commented Mar 19, 2015

something is not right - that's for sure. However, any docker instance that starts with a link (--link="nodegloo-postgres:pgdb" for example) works just fine ...

@jmls jmls closed this Mar 19, 2015

@jmls jmls reopened this Mar 19, 2015

@jmls jmls closed this Mar 19, 2015

@jmls

This comment has been minimized.

Copy link
Author

jmls commented Mar 19, 2015

wrong button. sorry ;)

@jmls jmls reopened this Mar 19, 2015

@exadeci

This comment has been minimized.

Copy link

exadeci commented Jul 28, 2015

I have a similar issue:

proxy:
  image: jwilder/nginx-proxy:latest
  ports:
    - "80:80"
  volumes:
    - "/var/run/docker.sock:/tmp/docker.sock"

db:
  image: mysql:5.7.7
  volumes_from:
    - dbdata
  environment:
    MYSQL_ROOT_PASSWORD
  ports:
    - "3306:3306"

dbdata:
  image: mysql:5.7.7
  command: /bin/echo mysql Data container

redis:
  image: redis

rails:
  build: .
  command: bundle exec unicorn -E production -c config/unicorn.rb
  restart: always
  links:
    - db
    - redis
  volumes:
    - .:/test
  environment:
    VIRTUAL_HOST: my.docker
    RAILS_ENV: production

worker:
  build: .
  command: bundle exec sidekiq -e production -c 25
  environment:
    RAILS_ENV: production
    SECRET_KEY_BASE
  links:
    - db
    - redis

And nginx dockergen logs:

proxy_1  | dockergen.1 | 2015/07/28 12:16:53 Generated '/etc/nginx/conf.d/default.conf' from 1 containers
proxy_1  | dockergen.1 | 2015/07/28 12:16:53 Watching docker events
proxy_1  | dockergen.1 | 2015/07/28 12:16:54 Received event start for container 2e3ce4c359a9

And then loops with:

proxy_1  | dockergen.1 | 2015/07/28 12:16:54 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload'
proxy_1  | dockergen.1 | 2015/07/28 12:16:54 Received event die for container 2e3ce4c359a9
@md5

This comment has been minimized.

Copy link
Contributor

md5 commented Jul 28, 2015

@exadeci What does the Dockerfile in your local directory look like? I suspect it isn't exposing any ports.

@exadeci

This comment has been minimized.

Copy link

exadeci commented Jul 28, 2015

And...I missed that 😓. I exposed it on the docker-compose and now it works. Thank you

@jwilder jwilder closed this Jul 28, 2015

@simison

This comment has been minimized.

Copy link

simison commented Feb 18, 2016

For anyone who might stumble upon this thread via search engine —

I had this issue. Stranded docker-compose run was causing this in my case.

I was getting flooded by these lines:

nginx_1   | dockergen.1 | 2016/02/18 08:38:31 Received event die for container d2c1fc301608
nginx_1   | dockergen.1 | 2016/02/18 08:38:31 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload'
nginx_1   | dockergen.1 | 2016/02/18 08:38:31 Received event start for container d2c1fc301608
nginx_1   | dockergen.1 | 2016/02/18 08:38:31 Contents of /etc/nginx/conf.d/default.conf did not change. Skipping notification 'nginx -s reload'

Running:

docker-compose ps

...gave:

demo_app_1       npm start                        Exit 137                       
demo_app_run_5   npm update                       Up         3000/tcp, 35729/tcp 
demo_mongodb_1   /entrypoint.sh --smallfile ...   Exit 0                         
demo_nginx_1     /app/docker-entrypoint.sh  ...   Exit 2  

...and therefore running:

docker stop $(docker ps -a -q)
docker-compose ps

...gave:

demo_app_1       npm start                        Exit 137                       
demo_mongodb_1   /entrypoint.sh --smallfile ...   Exit 0                         
demo_nginx_1     /app/docker-entrypoint.sh  ...   Exit 2  

...and the problem was gone.

If you'd like to remove those containers after stopping, you can do docker rm $(docker ps --no-trunc -aq)

There must've been something wrong with my way of doing the docker-compose run in first place, but that's a different issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment