SSL issue / port appended after upgrade to 2.9.14

[pseudomix]

Hello incredible NPM Team !

Checklist

• Have you pulled and found the error with jc21/nginx-proxy-manager:latest docker image?
  • Yes
• Are you sure you're not using someone else's docker image?
  • Yes
• Have you searched for similar issues (both open and closed)?
  • Yes

Describe the bug
After the update to 2.9.14 some proxy hosts are not working any more.
It seems that NPM doesn't do correctly the reverse proxy as i see in the browser address that he tries to access the internal port.
Reverting to the previous release fix the issue.

Nginx Proxy Manager Version
2.9.14

To Reproduce
Steps to reproduce the behavior:

1. Create a Proxy-Host with Linuxserver.io/Heimdall behind.
2. Apply HTTPS and Force HTTPS
3. Try to access the site:
   Browser reports bad SSL (SSL_ERROR_RX_RECORD_TOO_LONG) or in the best case the page appears but without any CSS Style

FYI: My NPM is behind Cloudflare DNS

Expected behavior
Site should appear correctly like before

Operating System
Docker Image on Debian 11.2

Reverting to 2.9.13 solves the issue. I'm not an expert with nginx.
Tried to recreate a simple Proxy-Host from scratch but same issue.
Hope this will help someone.

Thanks in advance for your help

[pseudomix]

Hello,

I'm still able to login into my NPM interface and i can't see any error on my Container Log nor Nginx logs.
I think it's not the same issue as @ch4ox mentionned but maybe i'm wrong.

Thanks for your help

[chaptergy]

Have you tried disabling Cloudflare to check whether this only happens in conjunction with cloudflare or with just npm as well?

[vilvic]

We're seeing the same issue and have had to rollback to a previous version. Redirects seem to pick up the internal port

[Nesci28]

Same problem is happening here. Had to rollback to 2.9.13.

[PaPaTheGMan]

Same problem, rollback to 2.9.13 fixes issue.

[pseudomix]

Hello,

@chaptergy : i have tried to diable Cloudflare Proxying and still the same.
To illustrate @vilvic in my case i access https://someapp.mydom.com and i'm automticaly redirected to https://someapp.mydom.com:12784 where 12784 is the internal port of a docker containers exposed to my Lan.

As it does not do this issue with most of my apps i have checked if it's related to the App Behind (linuxserver/heimdall).
I have tried to pull an older version but same issue.
I have tried with a fresh image out of the box but same issue too.

Can somenone tell me what logs or file to check to help solve this issue?
Thanks for your help.

[Killator]

Hello,

Same issue for me... Context:

"Classic DNS" > Home router > rPI 4 (Debian Bulleyes up-to-date) > NPM (HTTP/80&443) > Heimdall (HTTP/82 - docker - network conf. bridge).

Update on 2.9.14 > Page KO
Rollback on 2.9.13 > Page OK

Quick investigation on 2.9.14:

• Access to the root page is OK
• Every other requests are KO, because mentioning the internal port of the docker container ":82" (as @vilvic explain above)

Don't hesitate to mention me if you need more info ! :-)

[chaptergy]

Unfortunately I'm currently not able to replicate the issue on my dev instance, but based on the number of thumbs this seems to be a widespread issue. (Btw, very good etiquette of using the reactions instead of commenting without additional useful information, thanks!)

One of the things changed with nginx configs is the addition of the X-Forwarded-Host and X-Forwarded-Port headers. But I can't really explain how they cause this. I have found that setting the Host header to the domain with an IP causes a redirect to there, but I'm not yet seeing why the host header would be changed. Maybe the applications you are proxying use this new X-Forwarded-... header and somehow mix this information?

@pseudomix @Killator Would one of you guys be able to edit the nginx config manually (located at /data/nginx/proxy_host/<id>.conf, where <id> is the actual id of the host) and remove these two lines everywhere they are used, to see if they are actually connected to the issue?

proxy_set_header X-Forwarded-Host   $host;
proxy_set_header X-Forwarded-Port   $port;

[purcilas]

I tried rolling back to the 2.9.13 docker image but still broken. Any additional steps I need to take?

[pseudomix]

Hi @chaptergy ,

I checked my host config file but not found any occurrence of the related lines sorry.

[Killator]

Hi @chaptergy,

I confirm, there is no proxy_set_header in my config file either...

# ------------------------------------------------------------
# mysite.fr
# ------------------------------------------------------------
server {
  set $forward_scheme http;
  set $server         "192.168.168.3";
  set $port           82;

  listen 80;
#listen [::]:80;

listen 443 ssl http2;
#listen [::]:443;


  server_name mysite.fr;


  # Let's Encrypt SSL
  include conf.d/include/letsencrypt-acme-challenge.conf;
  include conf.d/include/ssl-ciphers.conf;
  ssl_certificate /etc/letsencrypt/live/npm-20/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/npm-20/privkey.pem;

    # Force SSL
    include conf.d/include/force-ssl.conf;

  access_log /data/logs/proxy-host-25_access.log proxy;
  error_log /data/logs/proxy-host-25_error.log warn;

  location / {
    # Proxy!
    include conf.d/include/proxy.conf;
  }

  # Custom
  include /data/nginx/custom/server_proxy[.]conf;
}

conf.d/include/proxy.conf > Seem to be inside the container
/data/nginx/custom/server_proxy[.]conf > The custom directory don't exist in nginx... (same thing with 'find' command on *.conf)

[virse]

I can confirm pseudomix,

the mentioned related lines are also not in my config.

greets

[chaptergy]

Yeah sorry, the directive is only included instead of explicitly generated in the file, so you would have to edit the conf.d/include/proxy.conf file and remove the directives from there

[pseudomix]

Hi Chaptergy,

This did the trick!

Removing:

proxy_set_header X-Forwarded-Host   $host;
proxy_set_header X-Forwarded-Port   $port;

from /etc/nginx/conf.d/include/proxy.conf worked for me.

In the meantime i have mount this file outside of the image to don't loose the change in case of recreating the container.
Feel free to ask me other test.

Thanks

[filerun]

proxy_set_header X-Forwarded-Port   $port;

Looks like $port get's replaced with the proxied host's port (the local port) instead of the port number used in the browser (the external port).

[poltrian]

I have redirect to port 80 since the last update on Pixelfed

[atbakken]

Sorry for my ignorance as I'm not 100% familiar with nginx reverse proxy, but why would the following be used:

proxy_set_header X-Forwarded-Port $port;

in favour of using the following:

proxy_set_header X-Forwarded-Port $server_port;

Would we not want to use server_port as it's the original requested port?

[jc21]

Please try the jc21/nginx-proxy-manager:github-develop docker image, which has these headers removed and comment whether this is problem is fixed.

[pseudomix]

Hello jc21,

Problem fixed with this develop version.
Thanks for your investigations and thanks for this awesome developement.

[poltrian]

I had problems with the login on Pixelfed- It was always forwarded to port 80. With the developer version it works correctly again

[Killator]

Please try the jc21/nginx-proxy-manager:github-develop docker image, which has these headers removed and comment whether this is problem is fixed.

Working fine ! Thanks a lot ! 👍

[silent001]

I can confirm that I am having the same issue with 2.9.14. If I inspect my site (laravel site) then the server port is appended to the end of the url. This was not an issue before and I don't know how to go back to a previous version. I have pulled and used the latest jc21/nginx-proxy-manager:github-develop image and this has fixed the issue for me as well. I just pulled the latest today and noticed this issue. What is the correct course of action for people with this issue?

[AlthaTechnology]

On this it did break one of my apps. Invoice Ninja, running in docker swarm. To fix that I changed my internal apache port on that application to 443 and that fixed it, but this thread answers why I was seeing the :80 at the end of the URL.

I will add that it fixed a long standing issue I've been having with an AVideo site. When going from the main site, to the encoder site, it would delay sometimes up to 2 minutes. The usernames/credentials get passed in that transition, these additions fixed that transition.

I can add these headers onto specific proxy hosts through the custom config in the UI? Is that right?

And if anyone has any follow up information to @atbakken 's comment concerning the different way to implement a similar header that might shed some light on this for my use case. This would be great to have by default as it does seem to be an improvement.

And thanks for this project! I'm using it extensively for 100+ websites/applications in front of a multiple node docker swarm configuration.

Sincerely,

Jared

[jiangge]

Please try the jc21/nginx-proxy-manager:github-develop docker image, which has these headers removed and comment whether this is problem is fixed.

Hello jc21,

It's not been fixed yet, the docker-compose.yml is as follows:

version: '3'
services:
  app:
    image: 'jc21/nginx-proxy-manager:github-develop'
    #image: 'jc21/nginx-proxy-manager:2.9.11'
    restart: unless-stopped
    ports:
      - '80:80'
      - '81:81'
      - '443:443'
    volumes:
      - ./newdata:/data
      - ./newletsencrypt:/etc/letsencrypt
      #- ./data:/data
      #- ./letsencrypt:/etc/letsencrypt

[speculaas19]

image: 'jc21/nginx-proxy-manager:github-develop' dit the trick for me.