Skip to content

Connection terminated unexpectedly #3182

Open
@7hael

Description

@7hael

Describe the bug

When performing API calls to my self-hosted instance after a period of inactivity, I occasionally get the following error:

{
  "level": 50,
  "severity": "ERROR",
  "err": {
    "type": "DatabaseError",
    "message": "Failed to execute db ops",
    "stack": "Find one: Failed to execute db ops\n    at Object.findOne (/backend/src/lib/knex/index.ts:106:13)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async Object.login (/backend/src/services/identity-ua/identity-ua-service.ts:53:24)\n    at async Object.handler (/backend/src/server/routes/v1/identity-universal-auth-router.ts:48:9)",
    "name": "Find one",
    "error": {
      "type": "Error",
      "message": "select * from \"identity_universal_auths\" where \"clientId\" = $1 limit $2 - Connection terminated unexpectedly",
      "stack": "Error: select * from \"identity_universal_auths\" where \"clientId\" = $1 limit $2 - Connection terminated unexpectedly\n    at Connection.<anonymous> (/backend/node_modules/pg/lib/client.js:131:73)\n    at Object.onceWrapper (node:events:632:28)\n    at Connection.emit (node:events:518:28)\n    at Connection.emit (node:domain:489:12)\n    at Socket.<anonymous> (/backend/node_modules/pg/lib/connection.js:62:12)\n    at Socket.emit (node:events:518:28)\n    at Socket.emit (node:domain:489:12)\n    at TCP.<anonymous> (node:net:343:12)\n    at TCP.callbackTrampoline (node:internal/async_hooks:130:17)"
    }
  },
  "msg": "Failed to execute db ops"
}

Only after the first one or two error responses the API works as expected.

To Reproduce

Steps to reproduce:

  1. Deploy an infisical instance.
  2. Set up a client ID and secret for API authentication.
  3. Wait some time (I have not been able to identify a more specific rule or pattern that triggers this behavior).
  4. Attempt to authenticate using a POST request to /api/v1/auth/universal-auth/login.

Expected behavior

The API should immediatly respond with the access token.

Additional context

The error can occur for every API call, I just usually get it for the universal-auth after some period of inactivity on Infisical, but i had the same problem with different calls.

I suppose this has to do with the database connection configurations. In the instance.ts the pool parameter is not passed to knex, so the default values are used. As it is stated in the official documentation and in this knex issue (which addresses the same exact problem I am having), it is advised to change the pool's min value to 0. Is there any particular reason why the pool's min value is not being set? Wouldn't it be better to make it configurable via an environment variable?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions