Connection refused while connecting to upstream

[Hermsi1337]

Hey Folks,

I'm a bit of a blockhead just now. I try to setup invoice ninja along with an nginx-proxy from jwilder with docker-compose. No matter what I do, I get the following error:

[error] 6#6: *1 connect() failed (111: Connection refused) while connecting to upstream, client: XXX.XX.X.X, server: application, request: "GET / HTTP/1.1", upstream: "fastcgi://XXX.XX.X.X:9000", host: "domain.tld"

Do you guys have an Idea?

This is my docker-compose.yml:

version: '3'

volumes:
  invoice-db-data:
     driver: local-persist
     driver_opts:
       mountpoint: ${CONTAINERVOLUME}/var/lib/mysql
  invoice-app-data:
     driver: local-persist
     driver_opts:
       mountpoint: ${CONTAINERVOLUME}/var/www/app

services:
  database:
     image: mysql
     restart: unless-stopped
     volumes:
       - invoice-db-data:/var/lib/mysql
     environment:
       - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
       - MYSQL_DATABASE=${MYSQL_DATABASE}
       - MYSQL_USER=${MYSQL_USER}
       - MYSQL_PASSWORD=${MYSQL_PASSWORD}

  webserver:
    image: nginx
    restart: unless-stopped
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      - invoice-app-data:/var/www/app
    environment:
      - VIRTUAL_HOST=${VIRTUAL_HOST}
      - VIRTUAL_PROTO=http
      - VIRTUAL_PORT=80
      - LETSENCRYPT_HOST=${LETSENCRYPT_HOST}
      - LETSENCRYPT_EMAIL=${LETSENCRYPT_EMAIL}
    ports:
      - ${PROXYPORT}:80
    networks:
      - nginxproxy
      - default

  application:
    image: invoiceninja/invoiceninja
    restart: unless-stopped
    environment:
       - APP_DEBUG=1
       - APP_URL=http://${VIRTUAL_HOST}
       - DB_HOST=database
       - DB_DATABASE=${MYSQL_DATABASE}
       - DB_USERNAME=${MYSQL_USER}
       - DB_PASSWORD=${MYSQL_PASSWORD}
    depends_on:
      - database
      - webserver
    volumes:
      - invoice-app-data:/var/www/app

  cron:
    image: invoiceninja/invoiceninja
    restart: unless-stopped
    user: www-data
    environment:
       - APP_DEBUG=1
       - DB_HOST=database
       - DB_DATABASE=${MYSQL_DATABASE}
       - DB_USERNAME=${MYSQL_USER}
       - DB_PASSWORD=${MYSQL_PASSWORD}
    depends_on:
      - database
      - webserver
      - application
    entrypoint: |
      bash -c 'bash -s <<EOF
      trap "break;exit" SIGHUP SIGINT SIGTERM
      sleep 300s
      while /bin/true; do
        DB_USERNAME=${MYSQL_USER} DB_PASSWORD=${MYSQL_PASSWORD} /usr/local/bin/php /var/www/app/artisan ninja:send-invoices
        DB_USERNAME=${MYSQL_USER} DB_PASSWORD=${MYSQL_PASSWORD} /usr/local/bin/php /var/www/app/artisan ninja:send-reminders
        sleep 1d
      done
      EOF'

networks:
  nginxproxy:
    external:
      name: ${NGINXPROXYNETWORK}

My nginx.conf:

user www-data;

events {
  worker_connections 768;
}

http {
    upstream backend {
        server application:9000;
    }
    include /etc/nginx/mime.types;
    default_type application/octet-stream;
    gzip on;
    gzip_disable "msie6";

    server {
        listen      80 default;
        server_name application;

        root /var/www/app/public;

        index index.php;

        charset utf-8;

        location / {
            try_files $uri $uri/ /index.php?$query_string;
        }

        location = /favicon.ico { access_log off; log_not_found off; }
        location = /robots.txt  { access_log off; log_not_found off; }

        sendfile off;

        location ~ \.php$ {
            fastcgi_split_path_info ^(.+\.php)(/.+)$;
            fastcgi_pass backend;
            fastcgi_index index.php;
            include fastcgi_params;
            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
            fastcgi_intercept_errors off;
            fastcgi_buffer_size 16k;
            fastcgi_buffers 4 16k;
        }

        location ~ /\.ht {
            deny all;
        }
    }
}

[ghhv]

Hi,
I wish I'd seen this 6 days ago. I too have had this problem trying to move two Ninja installations to a new server under Docker. I couldn't even get a fresh install happening.

After going crazy thinking it was an Nginx/Proxy/Let's Encrypt issue like I'm sure you did, I discovered tonight that the Docker container with the database is not being seeded and nothing works. There's nothing in any logs anywhere to indicate such an error.

If you want to prove the same thing to yourself, connect to your DB container via
docker exec -it container_db bash
connect to mysql with
mysql -u username -p ninja it will prompt for password.. change username and database name to suit.

type show tables; and you will probably find like I did, that it's empty.

The other option which I didn't try yet is dropping back to a previous image version other than latest.

Mega-frustrating.

[Hermsi1337]

Hey @ghhv

many thanks for your thoughts. But unfortunately the database-container is running very well.
I guess that my problem is located at the network-layer. As soon as I remove the nginx-proxy-network from webserver-networks, everything's working smoothly.

TBH:
I have absolutely no idea how to solve this one. For me, invoice-ninja is the only image that runs into this error. I will investigate this further and hopefully I will find a solution.

[ghhv]

Doh.. that's a shame.. so weird on my end.

As for your Docker-Compose.YML file, you're missing the - VIRTUAL_NETWORK= which should be set to ${NGINXPROXYNETWORK} in the WEB container.

Hopefully that fixes it.

Cheers,

[ghhv]

I also have the web port set to something different so that the external proxy port 80 redirects to it. Which will be needed for multiple end points on the same Docker server.

ports:
- 8000:8083
environment:
- VIRTUAL_NETWORK=xxx
- VIRTUAL_PORT=8083

The error you're getting normally means something is not listening at the port expected. See how those changes go.

[hillelcoren]

Maybe adding TRUSTED_PROXIES to the .env file will help.

http://docs.invoiceninja.com/en/latest/configure.html#using-a-proxy

[lalop]

@Hermsi1337 do you solved your issue ? Can we close this one ?

[Hermsi1337]

Oh sorry @lalop !!
Yeah, I finally solved it. There was another container named application in my nginx-proxy-network.
I fixed it by adding an alias to the the fpm-php container for default network.

Sorry for not closing this issue!