Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: Errors that happen during tenant initialization are not logged to console #5807

Open
2 of 4 tasks
vavido opened this issue Apr 28, 2024 · 2 comments
Open
2 of 4 tasks
Labels
enhancement Make it better

Comments

@vavido
Copy link

vavido commented Apr 28, 2024

Describe the bug

Errors that happen during tenant initialization are not reported on stdout.

Expected behavior

Whenever an error happens in the application, it should be printed to stdout to give system admins actionable information on what went wrong and how it might be fixed.

How to reproduce?

  1. Modify the database of a logto instance so that login is not possible for the logto_tenant_logto_admin user (e.g. change password)
  2. Start logto
  3. Navigate to admin console (localhost:3001)

The console only returns "Internal server error" with status code 500. No error information is printed to stdout.

Context

  • Logto Cloud
  • Self-hosted, initially happened on 1.11.0, but same behavior with 1.15.0
    • Container (Docker image)
    • Raw Node.js

Additional Info:

I discovered this bug today while trying to migrate the database used by logto to a new cluster.
After starting logto with the new Database, the console only returned "Internal Server Error" and no error message was visible in the container logs.
In the end, I had to roll back the migration because I couldn't figure out what was wrong.

Later, I cloned the repo and started adding additional error logging in various places to try and figure out what the actual error was.
In the end, adding consoleLog.error(error) to the

    const tenant = await trySafe(tenantPool.get(tenantId), (error) => {

callback in init.ts provided me with the hidden error message. With this additional line, my stdout now is

 ...
 info Init tenant: admin
error error: password authentication failed for user "logto_tenant_logto_admin"
    at Parser.parseErrorMessage (/Users/matth/Projects/selectcode/logto/node_modules/.pnpm/pg-protocol@1.6.0/node_modules/pg-protocol/dist/parser.js:287:98)
    at Parser.handlePacket (/Users/matth/Projects/selectcode/logto/node_modules/.pnpm/pg-protocol@1.6.0/node_modules/pg-protocol/dist/parser.js:126:29)
    at Parser.parse (/Users/matth/Projects/selectcode/logto/node_modules/.pnpm/pg-protocol@1.6.0/node_modules/pg-protocol/dist/parser.js:39:38)
    at TLSSocket.<anonymous> (/Users/matth/Projects/selectcode/logto/node_modules/.pnpm/pg-protocol@1.6.0/node_modules/pg-protocol/dist/index.js:11:42)
    at TLSSocket.emit (node:events:517:28)
    at addChunk (node:internal/streams/readable:368:12)
    at readableAddChunk (node:internal/streams/readable:341:9)
    at Readable.push (node:internal/streams/readable:278:10)
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23) {
  length: 120,
  severity: 'FATAL',
  code: '28P01',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'auth.c',
  line: '335',
  routine: 'auth_failed'
}

If I had the information during my initial migration, it would be simple to fix, but without the error info, i had to roll everything back after ~2h of trying various random things.

@vavido vavido added the bug Something isn't working label Apr 28, 2024
@vavido
Copy link
Author

vavido commented Apr 28, 2024

Might also be related to #4911

@charIeszhao
Copy link
Member

Thanks for the detailed information. We'll take a look!

@charIeszhao charIeszhao added enhancement Make it better and removed bug Something isn't working labels Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Make it better
Development

No branches or pull requests

2 participants