bug: 'Internal Server Error' with no more error information

[kidlj]

Describe the bug

I deploy Logto in a docker container, connecting to a standalone database with a dedicated DB user. The DB seeding process is fine, and Logto startup is also fine. But when accessing the admin dashboard, it always shows 'Internal Server Error' with no more error message or logs.

I tested with the official docker-compose.yml which connects to a containered Postgresql using postgres user, and it works fine.

Expected behavior

No internal server error.

How to reproduce?

1. Create a dedicated database and user for Logto usage.

$ createdb logto
$ createuser --login --createrole --pwprompt logto

2. docker-compose.yaml

$ cat docker-compose.yaml
version: "3.9"
services:
  app:
    image: svhd/logto:1.11
    entrypoint: ["sh", "-c", "npm run cli db seed -- --swe && npm start"]
    ports:
      - 3001:3001
      - 3002:3002
    environment:
      - TRUST_PROXY_HEADER=1
      - DB_URL=postgres://logto:logto@172.21.0.12:5432/logto
      - ENDPOINT=https://auth.acme.co
      - ADMIN_ENDPOINT=https://admin.acme.co

3. docker-compose up

metaphor $ docker-compose up -d
Creating logto_compose_app_1 ... done
metaphor $ docker logs logto_compose_app_1 -f

> cli
> logto db seed --swe

- Create tables
info ✔ Create tables
info ✔ [default] Generated config oidc.privateKeys
info ✔ [default] Generated config oidc.cookieKeys
info ✔ [default] Seed OIDC config
info ✔ [admin] Generated config oidc.privateKeys
info ✔ [admin] Generated config oidc.cookieKeys
info ✔ [admin] Seed OIDC config
info ✔ Cloud Service Application successfully created for: default
info ✔ Seed data

> start
> cd packages/core && NODE_ENV=production node .

(node:45) ExperimentalWarning: Import assertions are not a stable feature of the JavaScript language. Avoid relying on their current behavior and syntax as those might change in a future version of Node.js.
(Use `node --trace-warnings ...` to show where the warning was created)
(node:45) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
warn [CACHE] No Redis client initialized, skipping
info Core app is running at http://localhost:3001/
info Core app is running at https://auth.acme.co/
info Admin app is running at http://localhost:3002/
info Admin app is running at https://admin.acme.co/
info Init tenant: default      // 500 error; no more logs
info Init tenant: admin       // 500 error; no more logs

Context

• Logto Cloud
• Self-hosted, Logto version =
  • Container (Docker image) svhd/logto:1.11
  • Raw Node.js

Screenshots

[kidlj]

Is there any startup command argument that outputs more logs? Thanks.

[xiaoyijun]

Hi @kidlj , I just tested with your setup, but when I run docker compose up with the new created user, I encountered an error:

It's strange that you can seed your db with a role with insufficient permission and I cannot reproduce your situations.

Please provide more context, or reach out in our Discord channel to see if we can help you pinpoint the issue.

[kidlj]

@xiaoyijun Thank you for the help and sorry for not providing sufficient information to reproduce. I was using PostgreSQL 13 which is OK to seed db, but from PostgreSQL 15+ on, you must grant permission on PUBLIC schema to Logto user.

GRANT ALL ON SCHEMA public TO logto;

will fix this.

ref: https://medium.com/@dmitry.romanoff/postgresql-version-15-error-permission-denied-for-schema-public-dc8285c41e00

I have tested with PostgreSQL 13 and PostgreSQL 16, both resulted in the same 'Internal Server Error'.

[xiaoyijun]

Hi @kidlj , I just tested the way you have done, and it works on my machine.
It seems that the Internal Server Error is from your gateway or some incorrect proxy settings, please check your webserver configs.

[kidlj]

It should not be a proxy error, although I used Nginx to proxy to upstream Logto server. But when I skip Nginx and access Logto server directly, it's also an 'Internal Server Error', with no accessing logs.

It may be a Docker related issue, but when I put the PostgreSQL service to a container(as the official docker-compose.yaml does), everything works fine.

[xiaoyijun]

@kidlj , I deployed Logto as per your instructions, but I still couldn't reproduce the issue you mentioned.
There is no specific 500 error information shown in the Logto logs, so it's highly possible that the network requests didn't reach the Logto service.

Maybe it's worth checking if there are other reasons.

Here are the steps I took to deploy Logto:

1. Installed and ran Postgres (v16) on the machine.

postgres          8392   0.0  0.0 409142512   7424   ??  Ss   10:55PM   0:03.72 /Library/PostgreSQL/16/bin/postgres -D /Library/PostgreSQL/16/data

2. Created a Logto user on Postgres as you instructed and granted all permissions.

$ createdb logto
$ createuser --login --createrole --pwprompt logto

GRANT ALL ON SCHEMA public TO logto;

3. Launched the Logto Instance on the machine using the compose file you provided.

version: "3.9"
services:
  app:
    image: svhd/logto:1.11
    entrypoint: ["sh", "-c", "npm run cli db seed -- --swe && npm start"]
    ports:
      - 3001:3001
      - 3002:3002
    environment:
      - TRUST_PROXY_HEADER=1
      - DB_URL=postgres://logto:logto@host.docker.internal:5432/logto
      - ENDPOINT=https://${MYCUSTOM_AUTH_DOMAIN}
      - ADMIN_ENDPOINT=https://${MYCUSTOM_ADMIN_DOMAIN}

xiaoyijun@xiaoyijunAtSilverhand logto-bug-inspect % docker compose up
[+] Running 1/1
 ⠿ Container logto-bug-inspect-app-1  Recreated                                                                                                                                                                0.1s
Attaching to logto-bug-inspect-app-1
logto-bug-inspect-app-1  | 
logto-bug-inspect-app-1  | > cli
logto-bug-inspect-app-1  | > logto db seed --swe
logto-bug-inspect-app-1  | 
logto-bug-inspect-app-1  | - Create tables
logto-bug-inspect-app-1  | info ✔ Create tables
logto-bug-inspect-app-1  | info ✔ [default] Generated config oidc.privateKeys
logto-bug-inspect-app-1  | info ✔ [default] Generated config oidc.cookieKeys
logto-bug-inspect-app-1  | info ✔ [default] Seed OIDC config
logto-bug-inspect-app-1  | info ✔ [admin] Generated config oidc.privateKeys
logto-bug-inspect-app-1  | info ✔ [admin] Generated config oidc.cookieKeys
logto-bug-inspect-app-1  | info ✔ [admin] Seed OIDC config
logto-bug-inspect-app-1  | info ✔ Cloud Service Application successfully created for: default
logto-bug-inspect-app-1  | info ✔ Seed data
logto-bug-inspect-app-1  | npm notice 
logto-bug-inspect-app-1  | npm notice New major version of npm available! 9.8.1 -> 10.2.4
logto-bug-inspect-app-1  | npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.2.4>
logto-bug-inspect-app-1  | npm notice Run `npm install -g npm@10.2.4` to update!
logto-bug-inspect-app-1  | npm notice 
logto-bug-inspect-app-1  | 
logto-bug-inspect-app-1  | > start
logto-bug-inspect-app-1  | > cd packages/core && NODE_ENV=production node .
logto-bug-inspect-app-1  | 
logto-bug-inspect-app-1  | (node:44) ExperimentalWarning: Import assertions are not a stable feature of the JavaScript language. Avoid relying on their current behavior and syntax as those might change in a future version of Node.js.
logto-bug-inspect-app-1  | (Use `node --trace-warnings ...` to show where the warning was created)
logto-bug-inspect-app-1  | (node:44) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
logto-bug-inspect-app-1  | warn [CACHE] No Redis client initialized, skipping
logto-bug-inspect-app-1  | info Core app is running at http://localhost:3001/
logto-bug-inspect-app-1  | info Core app is running at https://auth.***.***/
logto-bug-inspect-app-1  | info Admin app is running at http://localhost:3002/
logto-bug-inspect-app-1  | info Admin app is running at https://admin.***.***/
logto-bug-inspect-app-1  | info Init tenant: admin

5. Accessed the Logto Instance via a custom domain using a reverse proxy.

logto-bug-inspect-app-1  |   <-- GET /
logto-bug-inspect-app-1  |   --> GET / 302 13ms 63b
logto-bug-inspect-app-1  |   <-- GET /console/welcome
logto-bug-inspect-app-1  |   --> GET /console/welcome 200 36ms 358b

And the docker containers is as follows:

Logto works fine, please check if there are other reasons.

[kidlj]

It must be a docker-proxy related issue, since the symptom is the same as the issue(nothing but simply 500 Internal Server Error). @xiaoyijun Thanks for your help.