Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

root: always use persistent database connections #6560

Merged
merged 2 commits into from Aug 17, 2023

Conversation

rissson
Copy link
Member

@rissson rissson commented Aug 17, 2023

Details

Ideally this would avoid re-opening a database connection for every database query, even when not using pg_bouncer.

Upside: we're avoiding creating connections for every database query.

Downside: we would always have, at most, N connections open, N being the number of workers, once every worker has been hit with a web request. Also, if something breaks, the connection is not re-created automatically, if I understand correctly.

We could also set CONN_HEALTH_CHECKS to true to mitigate that last bit, which will check the connection health once per web request.

Finally, this may help with auto-rotating secrets, in which case the connection would stay opened and healthy from when the password has been changed on postgres' side, until it reaches authentik.


Checklist

  • Local tests pass (ak test authentik/)
  • The code has been formatted (make lint-fix)

If an API change has been made

  • The API schema has been updated (make gen-build)

If changes to the frontend have been made

  • The code has been formatted (make web)
  • The translation files have been updated (make i18n-extract)

If applicable

  • The documentation has been updated
  • The documentation has been formatted (make website)

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>
@rissson rissson requested a review from a team as a code owner August 17, 2023 02:38
@rissson rissson requested review from BeryJu and removed request for a team August 17, 2023 02:38
@rissson rissson self-assigned this Aug 17, 2023
@netlify
Copy link

netlify bot commented Aug 17, 2023

Deploy Preview for authentik-storybook ready!

Name Link
🔨 Latest commit a21e5d9
🔍 Latest deploy log https://app.netlify.com/sites/authentik-storybook/deploys/64de07b8e6db440008c8e690
😎 Deploy Preview https://deploy-preview-6560--authentik-storybook.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@codecov
Copy link

codecov bot commented Aug 17, 2023

Codecov Report

Patch coverage has no change and project coverage change: -0.02% ⚠️

Comparison is base (594e031) 92.48% compared to head (a21e5d9) 92.46%.
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6560      +/-   ##
==========================================
- Coverage   92.48%   92.46%   -0.02%     
==========================================
  Files         561      561              
  Lines       27081    27080       -1     
==========================================
- Hits        25044    25037       -7     
- Misses       2037     2043       +6     
Flag Coverage Δ
e2e 51.61% <ø> (+0.01%) ⬆️
integration 26.57% <ø> (+0.01%) ⬆️
unit 89.27% <ø> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
authentik/root/settings.py 89.32% <ø> (+0.68%) ⬆️

... and 4 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 17, 2023

authentik PR Installation instructions

Instructions for docker-compose

Add the following block to your .env file:

AUTHENTIK_IMAGE=ghcr.io/goauthentik/dev-server
AUTHENTIK_TAG=gh-always-disable-persistent-db-connections-1692273370-a21e5d9
AUTHENTIK_OUTPOSTS__CONTAINER_IMAGE_BASE=ghcr.io/goauthentik/dev-%(type)s:gh-%(build_hash)s

For arm64, use these values:

AUTHENTIK_IMAGE=ghcr.io/goauthentik/dev-server
AUTHENTIK_TAG=gh-always-disable-persistent-db-connections-1692273370-a21e5d9-arm64
AUTHENTIK_OUTPOSTS__CONTAINER_IMAGE_BASE=ghcr.io/goauthentik/dev-%(type)s:gh-%(build_hash)s

Afterwards, run the upgrade commands from the latest release notes.

Instructions for Kubernetes

Add the following block to your values.yml file:

authentik:
    outposts:
        container_image_base: ghcr.io/goauthentik/dev-%(type)s:gh-%(build_hash)s
image:
    repository: ghcr.io/goauthentik/dev-server
    tag: gh-always-disable-persistent-db-connections-1692273370-a21e5d9

For arm64, use these values:

authentik:
    outposts:
        container_image_base: ghcr.io/goauthentik/dev-%(type)s:gh-%(build_hash)s
image:
    repository: ghcr.io/goauthentik/dev-server
    tag: gh-always-disable-persistent-db-connections-1692273370-a21e5d9-arm64

Afterwards, run the upgrade commands from the latest release notes.

Copy link
Member

@BeryJu BeryJu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, allthough I think we should also set CONN_HEALTH_CHECKS as you already mentioned (I'm not 100% sure about the performance impact of both of these changes, I'm estimating it'll lower the avg% for response time but could worse then p99 (especially when the password rotation does happen)

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>
@rissson
Copy link
Member Author

rissson commented Aug 17, 2023

we should also set CONN_HEALTH_CHECKS

Done.

I'm not 100% sure about the performance impact of both of these changes, I'm estimating it'll lower the avg% for response time but could worse then p99 (especially when the password rotation does happen)

I've updated the description to better differentiate web request and database queries. As the check only happens once per web request, and only if the database is queried, it shouldn't be a lot of overhead, and certainly less than re-creating a brand new connection to the database.

Also, for reference, the check made by django to see if the connection is alive is SELECT 1

@BeryJu BeryJu merged commit 1d99ec9 into main Aug 17, 2023
64 of 65 checks passed
@BeryJu BeryJu deleted the always-disable-persistent-db-connections branch August 17, 2023 17:38
kensternberg-authentik added a commit that referenced this pull request Aug 21, 2023
* main: (70 commits)
  core: hotfix group membership check (#6584)
  web: bump core-js from 3.32.0 to 3.32.1 in /web (#6581)
  web: bump tslib from 2.6.1 to 2.6.2 in /web (#6583)
  web: bump the storybook group in /web with 5 updates (#6580)
  web/flows: update flow background (#6579)
  translate: Updates for file web/xliff/en.xlf in zh_CN on branch main (#6575)
  core: rework recursive group membership (#6017)
  core: bump goauthentik.io/api/v3 from 3.2023061.11 to 3.2023061.12 (#6572)
  core: bump ruff from 0.0.284 to 0.0.285 (#6570)
  ci: bump actions/setup-node from 3.8.0 to 3.8.1
  blueprints: fix blueprint importer logging potentially sensitive data (#6567)
  web: bump API Client version (#6568)
  stages/authenticator_static: make static token size adjustable (#6565)
  root: always use persistent database connections (#6560)
  website/blog: identity fun (#6564)
  web: bump the storybook group in /web with 1 update (#6561)
  web: bump @rollup/plugin-node-resolve from 15.1.0 to 15.2.0 in /web (#6562)
  web: bump @lit-labs/task from 3.0.0 to 3.0.1 in /web (#6544)
  web: bump prettier from 3.0.1 to 3.0.2 in /web (#6549)
  web: bump the storybook group in /web with 5 updates (#6559)
  ...
BeryJu added a commit that referenced this pull request Aug 29, 2023
@BeryJu BeryJu mentioned this pull request Aug 29, 2023
7 tasks
BeryJu added a commit that referenced this pull request Aug 29, 2023
BeryJu added a commit that referenced this pull request Aug 29, 2023
Revert "root: always use persistent database connections (#6560)"

This reverts commit 1d99ec9.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants