net::ERR_ABORTED 502 (Bad Gateway)

[ThePredators]

Version:

• listmonk: [v2.2.0]
• OS: [Ubuntu]

Description of the bug and steps to reproduce:
I manage to download & run listmonk as a service on my server but somehow i got issues when trying to access admin page.

ScreenShot:

[knadh]

Looks like listmonk binary isn't running.

[ThePredators]

Its actually running and this happens after i press login button on this page.

Proof that service is running perfectly:

[knadh]

Is there Nginx or another reverse proxy in between? The URLs in your original screenshot are throwing a 502, which means the requests aren't reaching listmonk. Whatever is trying to proxy the requests doesn't see that listmonk is running, thus is returning 502.

[ThePredators]

Is there Nginx or another reverse proxy in between? The URLs in your original screenshot are throwing a 502, which means the requests aren't reaching listmonk. Whatever is trying to proxy the requests doesn't see that listmonk is running, thus is returning 502.

There is

    if ($http_referer ~ "^http://IP_ADRESS/listmonk") {
        rewrite ^/$ /listmonk/ permanent;
        rewrite ^/(admin|api|font|public|subscription|link|campaign|webhooks)(.$
    }

    if ($http_referer ~ "^http://IP_ADRESS/admin/static/css/app") {
        rewrite ^/admin(.*) /listmonk/admin$1 last;
    }

    location /listmonk {
        rewrite ^/listmonk(.*) $1 break;
        proxy_pass http://localhost:9000;
        proxy_set_header X-Forwarded-For $proxy_protocol_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        sub_filter_types *;
        sub_filter 'http://localhost:9000/' 'http://IP_ADRESS/listmonk/';
        sub_filter '"http://localhost:9000"' '"http://IP_ADRESS/listmonk"';
        sub_filter '"/admin/"' '"/listmonk/admin/"';
        sub_filter_once off;
    }

Any other nginx conf doesn't work tried a lot based on other issues i saw.

[knadh]

I don't think trying to force listmonk to run on a sub-path (/listmonk) works. Please see #174

[ThePredators]

I don't think trying to force listmonk to run on a sub-path (/listmonk) works. Please see #174

Im running listmonk as standalone on an ubuntu server to test its capabilities. Im not using any docker. I tried without sub-path it doesn't work. Beside a have others services using sub-paths that works perfectly on the same server.

[knadh]

listmonk itself cannot produce 502s. The 502s are coming from Nginx, so there has to be something off with the Nginx config that's preventing it from reaching listmonk.

[Sebastix]

Having the same issue since a couple of days on my CentOS 7 machine. This is my nginx config file using a reverse proxy setup. I changed the port 9000 to 9080 in de listmonk setup (Docker compose).

The login screen looks fine, no errors. After I login, these errors popping up in my browser console (Brave and also in Firefox):

Failed to load resource: net::ERR_HTTP2_PROTOCOL_ERROR app.b3f9ffde.js:1          
Failed to load resource: net::ERR_HTTP2_PROTOCOL_ERROR chunk-vendors.941cc155.js:1          
Failed to load resource: net::ERR_HTTP2_PROTOCOL_ERROR main.a8a1b198.css:1          
Failed to load resource: net::ERR_HTTP2_PROTOCOL_ERROR main.33385b97.js:1          
...

I replaced my current subdomain with <subdomain>

server {
  listen 80;
  listen [::]:80;

  server_name <subdomain>;

  include snippets/letsencrypt.conf;
  return 301 https://$host$request_uri;
}

server {
  server_name <subdomain>;
  server_tokens off;
  listen 443 ssl;
  listen [::]:443 ssl;

  location / {
      proxy_pass http://127.0.0.1:9080;
      proxy_set_header  Host                $host;
      proxy_set_header  X-Real-IP           $remote_addr;
      proxy_set_header  X-Forwarded-Ssl     on;
      proxy_set_header  X-Forwarded-For     $proxy_add_x_forwarded_for;
    }

    access_log /var/log/nginx/<subdomain>.access.log;
    error_log /var/log/nginx/<subdomain>.error.log;

    ssl_certificate /etc/letsencrypt/live/<subdomain>/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/<subdomain>/privkey.pem;
    ssl_trusted_certificate /etc/letsencrypt/live/<subdomain>/chain.pem;
    include snippets/ssl.conf;
    include snippets/letsencrypt.conf;
}

Checked my nginx error log and found some permission issues:

2022/11/10 12:49:48 [crit] 26983#26983: *63 open() "/var/lib/nginx/tmp/proxy/9/02/0000000029" failed (13: Permission denied) while reading upstream, client: 92.68.203.32, server: <subdomain>, request: "GET /admin/static/js/main.33385b97.js HTTP/2.0", upstream: "http://127.0.0.1:9080/admin/static/js/main.33385b97.js", host: "<subdomain>"

After checking the permissions on /var/lib/nginx/tmp/proxy found out the group was on root instead of nginx. Changed the group to nginx fixed the issue.

chown -R nginx:nginx /var/lib/nginx/tmp/proxy

Maybe this info can help you out, or others in the future.

[MaximilianKohler]

@Sebastix I'm also on Centos 7 and having a 502 bad gateway error, but when loading the webpage -- not after logging into it.

I also had to change the port because it conflicted with php-fpm, but even after doing that, I'm still getting the 502 error. I posted my steps and config here: #1573 (comment)

I checked my nginx error log /home/nginx/domains/test1.mysite.com/log/error.log and I see a bunch of:

2023/10/31 06:42:57 [error] 62193#62193: *96 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: <IP>, server: test1.mysite.com, request: "GET / HTTP/2.0", upstream: "http://127.0.0.1:9000/", host: "test1.mysite.com"

I added these lines that you used (then restarted nginx) but it didn't make a difference:

      proxy_set_header  Host                $host;
      proxy_set_header  X-Real-IP           $remote_addr;
      proxy_set_header  X-Forwarded-Ssl     on;
      proxy_set_header  X-Forwarded-For     $proxy_add_x_forwarded_for;

I do not have a /var/lib/nginx/ folder on my system. But I guess that's unique to the specific error you got.

Oh, I see you also changed proxy_pass http://127.0.0.1:9080; to 9080 (mine was still 9000). I changed mine to 3870 and restarted nginx but I still get the same 502 error. I'll try changing everything to 9080. You only change in two spots right? The docker-compose.yml https://github.com/knadh/listmonk/blob/master/docker-compose.yml#L11 and the proxy_pass in the SSL config?

config.toml stays at address = "0.0.0.0:9000"? And docker-compose.yml changes to:

x-app-defaults: &app-defaults
  restart: unless-stopped
  image: listmonk/listmonk:latest
  ports:
    - "9080:9000"

I did that, and also tried port 9080 instead of the random 3870 I picked, and I still get the 502 error.

I did a search for the error in the log https://www.ecosia.org/search?q=104%253A%20Connection%20reset%20by%20peer)%20while%20reading%20response%20header%20from%20upstream and it sounds like there's still a conflict with php fpm.

EDIT: It was a firewall issue. I disabled my firewall and it works.

[silver0-stack]

serve -s build worked for me