Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

404 every second time #1081

Closed
mehmandarov opened this issue Feb 28, 2018 · 10 comments
Closed

404 every second time #1081

mehmandarov opened this issue Feb 28, 2018 · 10 comments

Comments

@mehmandarov
Copy link

mehmandarov commented Feb 28, 2018

I have to containers with two services running (one on each container) on port 8080, mapped to two different pots on the host machine.
I am getting a HTTP 404 error after each successful load (HTTP 200) for the same service:

nginx-proxy_1     | nginx.1    | localhost 172.18.0.1 - - [28/Feb/2018:21:06:35 +0000] "GET /myservice1 HTTP/1.1" 200 5 "-" "curl/7.54.0"
nginx-proxy_1     | nginx.1    | localhost 172.18.0.1 - - [28/Feb/2018:21:06:35 +0000] "GET /myservice1 HTTP/1.1" 404 68 "-" "curl/7.54.0"
nginx-proxy_1     | nginx.1    | localhost 172.18.0.1 - - [28/Feb/2018:21:06:35 +0000] "GET /myservice1 HTTP/1.1" 200 5 "-" "curl/7.54.0"
nginx-proxy_1     | nginx.1    | localhost 172.18.0.1 - - [28/Feb/2018:21:06:36 +0000] "GET /myservice1 HTTP/1.1" 404 68 "-" "curl/7.54.0"
nginx-proxy_1     | nginx.1    | localhost 172.18.0.1 - - [28/Feb/2018:21:06:36 +0000] "GET /myservice1 HTTP/1.1" 200 5 "-" "curl/7.54.0"
nginx-proxy_1     | nginx.1    | localhost 172.18.0.1 - - [28/Feb/2018:21:06:36 +0000] "GET /myservice1 HTTP/1.1" 404 68 "-" "curl/7.54.0"
nginx-proxy_1     | nginx.1    | localhost 172.18.0.1 - - [28/Feb/2018:21:06:36 +0000] "GET /myservice1 HTTP/1.1" 200 5 "-" "curl/7.54.0"

But, this does not happen if I call two different services (HTTP 200, but 404 if I repeat the call):

nginx-proxy_1     | nginx.1    | localhost 172.18.0.1 - - [28/Feb/2018:21:13:00 +0000] "GET /myservice2 HTTP/1.1" 200 23 "-" "curl/7.54.0"
nginx-proxy_1     | nginx.1    | localhost 172.18.0.1 - - [28/Feb/2018:21:13:02 +0000] "GET /myservice1 HTTP/1.1" 200 5 "-" "curl/7.54.0"
nginx-proxy_1     | nginx.1    | localhost 172.18.0.1 - - [28/Feb/2018:21:13:05 +0000] "GET /myservice2 HTTP/1.1" 200 23 "-" "curl/7.54.0"
nginx-proxy_1     | nginx.1    | localhost 172.18.0.1 - - [28/Feb/2018:21:13:07 +0000] "GET /myservice1 HTTP/1.1" 200 5 "-" "curl/7.54.0"
nginx-proxy_1     | nginx.1    | localhost 172.18.0.1 - - [28/Feb/2018:21:13:08 +0000] "GET /myservice2 HTTP/1.1" 200 23 "-" "curl/7.54.0"
nginx-proxy_1     | nginx.1    | localhost 172.18.0.1 - - [28/Feb/2018:21:13:10 +0000] "GET /myservice1 HTTP/1.1" 200 5 "-" "curl/7.54.0"
nginx-proxy_1     | nginx.1    | localhost 172.18.0.1 - - [28/Feb/2018:21:13:13 +0000] "GET /myservice1 HTTP/1.1" 404 68 "-" "curl/7.54.0"

This happens when I use curl. When I use browser, it is the opposite – I get 200 every time I reload the same service, but if I change the service I get 404 for the first time, and 200 for all the consecutive reloads.

I would appreciate if someone could explain this issue? Thanks in advance!

@jwilder
Copy link
Collaborator

jwilder commented Mar 24, 2018

Hi, I'd like to keep Github issues for actionable bug reports or feature requests. Please ask questions in the Q&A Group.

Thanks!

@jwilder jwilder closed this as completed Mar 24, 2018
@Lewiscowles1986
Copy link

I'm experiencing this bug on work software using this.

Ubuntu 18.04 (fully patched) 64-bit (output of uname -a)

Linux lewis-HP-Pavilion-Laptop-15-cs0xxx 4.15.0-38-generic #41-Ubuntu SMP Wed Oct 10 10:59:38 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Docker version (from docker -v)

Docker version 18.06.1-ce, build e68fc7a

Docker-compose version (from docker-compose -v)

docker-compose version 1.21.2, build a133471

nginx.1    | kalo.test 172.17.0.1 - - [30/Oct/2018:13:51:10 +0000] "GET /vendor.js HTTP/1.1" 404 151 "http://[censored].test/me/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:63.0) Gecko/20100101 Firefox/63.0"

@mehmandarov
Copy link
Author

@Lewiscowles1986 This is probably because you are having two different services behind the proxy. Or the code is not deployed on one of the services (if the services behind proxy are the same).

@Lewiscowles1986
Copy link

I'm so sorry I forgot to update this. Yes it was a documentation issue. I'd been telling new starters at work but must have forgotten posting the issue. There was a test and local version of a container running asking for the same hostname and port and that was causing issues.

@dman776
Copy link

dman776 commented Feb 28, 2019

I am experiencing this same issue. I have a python http service behind the proxy AND a few APACHE2 web servers. When making requests to apache, everything works as expected. However, on the Python server container, every other request returns a 404.

The logs from nginx-proxy container show the 200/404, 200/404 pattern.
The logs from the python container, only show the successful 200 request-responses.

uname -a:
Linux otis 4.4.0-142-generic #168-Ubuntu SMP Wed Jan 16 21:00:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

docker -v:
Docker version 18.09.2, build 6247962

docker-compose.yml.txt

@Lewiscowles1986
Copy link

Are there other ports bound to the same as the python application?

@dman776
Copy link

dman776 commented Mar 4, 2019

no. the python server only exposes port 8080

docker-compose logs nginx-proxy shows the alternating errors: (virtual host and IP addresses masked in this output)

nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:07:44 +0000] "GET /sysinfo HTTP/1.1" 404 205 "-" "python-requests/2.19.1"
nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:07:48 +0000] "PUT /sysinfo HTTP/1.1" 200 0 "-" "python-requests/2.19.1"
nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:08:04 +0000] "PUT /sysinfo HTTP/1.1" 405 228 "-" "python-requests/2.19.1"
nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:08:08 +0000] "PUT /sysinfo HTTP/1.1" 200 0 "-" "python-requests/2.19.1"
nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:08:17 +0000] "PUT /sysinfo HTTP/1.1" 405 228 "-" "python-requests/2.19.1"
nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:08:21 +0000] "PUT /sysinfo HTTP/1.1" 200 0 "-" "python-requests/2.19.1"
nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:08:24 +0000] "PUT /sysinfo HTTP/1.1" 405 228 "-" "python-requests/2.19.1"
nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:08:27 +0000] "PUT /sysinfo HTTP/1.1" 200 0 "-" "python-requests/2.19.1"
nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:08:32 +0000] "PUT /sysinfo HTTP/1.1" 405 228 "-" "python-requests/2.19.1"
nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:08:34 +0000] "PUT /sysinfo HTTP/1.1" 200 0 "-" "python-requests/2.19.1"
nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:08:41 +0000] "GET /sysinfo HTTP/1.1" 404 205 "-" "python-requests/2.19.1"
nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:08:44 +0000] "PUT /sysinfo HTTP/1.1" 200 0 "-" "python-requests/2.19.1"
nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:08:51 +0000] "PUT /sysinfo HTTP/1.1" 405 228 "-" "python-requests/2.19.1"
nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:08:53 +0000] "PUT /sysinfo HTTP/1.1" 499 0 "-" "python-requests/2.19.1"
nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:08:57 +0000] "GET /sysinfo HTTP/1.1" 404 205 "-" "python-requests/2.19.1"
nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:09:07 +0000] "PUT /sysinfo HTTP/1.1" 200 0 "-" "python-requests/2.19.1"
nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:09:16 +0000] "PUT /sysinfo HTTP/1.1" 405 228 "-" "python-requests/2.19.1"
nginx-proxy    | nginx.1    | xxxx 1.2.3.4 - - [04/Mar/2019:23:09:24 +0000] "PUT /sysinfo HTTP/1.1" 200 0 "-" "python-requests/2.19.1"

@dman776
Copy link

dman776 commented Apr 4, 2019

I found something interesting:
in nginx-proxy (container) /etc/nginx/conf.d/default.conf

I see:

# data.dnsname.org
upstream data.dnsname.org {
    ## Can be connected with "proxy" network
    # sysinfodb
    server 172.18.0.7:8080;
    ## Can be connected with "proxy" network
    # data
    server 172.18.0.5:80;

It appears that nginx is round-robin cycling thru these two servers. Only the first one is actually serving files. the second one fails with a 404. Why is the second entry there? How do I remove it?

@dman776
Copy link

dman776 commented Apr 4, 2019

if I MANUALLY edit the default.conf file and remove the second server entry, everything works as expected.

@SanariSan
Copy link

Got such problem with 404/200 alternation.

Now it's resolved and I assume that my problem was in making docker network connect on nginx-proxy, acme-companion and every other container after their launch.

For nginx-proxy/acme I did connect just because launching those is a part of another project, so I had no need to create separate network from the start.

As for other projects, I wanted to not mix external connection with inner cross-container routing. Since docker can't start with multiple --net I had to choose one from start (and it's obviously not external network).

Today bumped into 404/200 with docker compose, where I had default network for inner communication and
custom inbound network for https routing to server port.

Have made out lots tweaking, but ended up actually fixing that after launching nginx-proxy / acme-companion with pre-defined network. Here's how it looks:

Create network with:

docker network create inbound

How I launch nginx-proxy/acme (from control script here):

    command="/usr/bin/docker run \
    --rm \
    --detach \
    --name nginx-proxy \
    --publish 80:80 \
    --publish 443:443 \
    --net inbound \
    --volume $(pwd)/log:/log \
    --volume certs:/etc/nginx/certs \
    --volume vhost:/etc/nginx/vhost.d \
    --volume html:/usr/share/nginx/html \
    --volume /var/run/docker.sock:/tmp/docker.sock:ro \
    nginxproxy/nginx-proxy:1.0"

    command="/usr/bin/docker run \
    --rm \
    --detach \
    --name nginx-proxy-acme \
    --net inbound \
    --volumes-from nginx-proxy \
    --volume /var/run/docker.sock:/var/run/docker.sock:ro \
    --volume acme:/etc/acme.sh \
    --env \"DEFAULT_EMAIL=${CERTIFICATE_EMAIL}\" \
    nginxproxy/acme-companion:2.2"

Part of docker-compose from recent project:

...
networks:
    inbound:
        name: inbound
        external: true
services:
    app:
        image: ...
        networks:
            - inbound
            - default
     postgres:
        image: ...
        networks:
            - default
...


When starting container with ❗docker, not ❗docker-compose
You should initially put --net nginx-proxy-network (inboundin my case)
And then throw docker connect network your-internal-net container (make sure container can retry connections if it relies on internal net)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants