Skip to content

feat: make platform db connection pooling configurable #2730

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open

feat: make platform db connection pooling configurable #2730

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
85f5738
feat: add database connection parameters to gitea values
merll Nov 18, 2025
2f00c85
chore: added missing default value
merll Nov 18, 2025
4315b5d
chore: removed unused schema elements
merll Nov 18, 2025
d8f8b8f
fix: location of backupSidecarResources
merll Nov 19, 2025
26baf27
fix: make use of backupSidecarResources
merll Nov 19, 2025
d480d57
feat: added postgresqlParameters to values
merll Nov 19, 2025
0e1c636
feat: added connection parameters for Harbor
merll Nov 19, 2025
ec997bc
chore: removed unused value from schema
merll Nov 19, 2025
f6e7f97
feat: added connection parameters for Keycloak
merll Nov 19, 2025
1ae34df
chore: migrated values on test fixture
merll Nov 19, 2025
961a462
Merge branch 'main' into APL-1291
merll Nov 19, 2025
3705f87
Merge branch 'main' into APL-1291
svcAPLBot Nov 20, 2025
1682949
Merge branch 'main' into APL-1291
svcAPLBot Nov 20, 2025
60f3506
Merge branch 'main' into APL-1291
svcAPLBot Nov 20, 2025
c2f3d1e
Merge branch 'main' into APL-1291
svcAPLBot Nov 20, 2025
1316cf4
Merge branch 'main' into APL-1291
svcAPLBot Nov 21, 2025
3760cc4
Merge branch 'main' into APL-1291
svcAPLBot Nov 21, 2025
938403c
Merge branch 'main' into APL-1291
svcAPLBot Nov 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions charts/otomi-db/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ labels:
prometheus: system
helm.sh/resource-policy: keep
instances: 2
postgresql:
parameters: {}
storage:
storageClass: ""
size: 5Gi
Expand Down
36 changes: 22 additions & 14 deletions helmfile.d/snippets/defaults.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,9 @@ environments:
_rawValues: {}
networkPolicies:
enabled: true
database:
maxConnections: 28
maxIdleConnections: 6
resources:
gitea:
limits:
Expand All @@ -158,20 +161,6 @@ environments:
requests:
cpu: 100m
memory: 256Mi
memcached:
limits:
cpu: 250m
memory: 256Mi
requests:
cpu: 10m
memory: 128Mi
memcachedMetrics:
limits:
cpu: 200m
memory: 128M
requests:
cpu: 10m
memory: 16M
init:
limits:
cpu: 400m
Expand Down Expand Up @@ -204,6 +193,9 @@ environments:
registry:
credentials:
username: otomi-admin
database:
maxConnections: 28
maxIdleConnections: 6
resources:
core:
requests:
Expand Down Expand Up @@ -336,6 +328,10 @@ environments:
keycloak:
adminUsername: otomi-admin
theme: otomi
database:
Copy link
Contributor

@CasLubbers CasLubbers Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would make more sense for me if these connections values are also under the database objects? Then they would end up in the same file where all the other database settings are stored.

Copy link
Contributor Author

@merll merll Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are not database settings, but application settings. When putting these in the same place, I would find it ambiguous how they relate to max_connections of the database. And applications can have different ways of connection pooling, that may also change independently from the database (note how Keycloak differs from Gitea and Harbor), while all PostgreSQL databases work the same way.
It only would make sense if we removed these and set them implicitly (e.g. set these numbers based on max_connections of the database) but I would prefer not to, as this seems like excessively opinionated template magic to me.

We could rename it however, to make it more clear. Maybe call it databaseConnection?

Copy link
Contributor

@CasLubbers CasLubbers Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I saw it wrong. I thought these values where set on the database. But that is the postgresql settings.
Indeed having databaseMaxConnection etc would help. Seeing a database object gives me the idea that you can set database things in the application settings.

maxConnections: 28
minConnections: 5
initialConnections: 5
resources:
keycloak:
requests:
Expand Down Expand Up @@ -1195,6 +1191,10 @@ environments:
requests:
cpu: "50m"
memory: 48Mi
postgresqlParameters:
max_connections: "32"
shared_buffers: 64MB
work_mem: 2MB
harbor:
imageName: null
size: 5Gi
Expand All @@ -1220,6 +1220,10 @@ environments:
requests:
cpu: "50m"
memory: 48Mi
postgresqlParameters:
max_connections: "32"
shared_buffers: 64MB
work_mem: 2MB
gitea:
imageName: null
size: 5Gi
Expand All @@ -1244,6 +1248,10 @@ environments:
requests:
cpu: "50m"
memory: 48Mi
postgresqlParameters:
max_connections: "32"
shared_buffers: 64MB
work_mem: 2MB
obj:
provider:
type: disabled
Expand Down
16 changes: 1 addition & 15 deletions tests/fixtures/env/apps/gitea.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ spec:
resources:
gitea:
limits:
cpu: '1'
cpu: "1"
memory: 1Gi
requests:
cpu: 100m
Expand All @@ -21,17 +21,3 @@ spec:
requests:
cpu: 100m
memory: 128Mi
memcached:
limits:
cpu: 250m
memory: 256Mi
requests:
cpu: 10m
memory: 128Mi
memcachedMetrics:
limits:
cpu: 200m
memory: 128M
requests:
cpu: 10m
memory: 16M
Empty file added tests/fixtures/env/teams/demo/workloadValues/wd.managed.managed.yaml
View file
Open in desktop
Empty file.
2 changes: 1 addition & 1 deletion tests/fixtures/env/teams/demo/workloadValues/wd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
a: b
{}
Empty file added tests/fixtures/env/teams/demo/workloadValues/wd1.managed.managed.yaml
View file
Open in desktop
Empty file.
2 changes: 1 addition & 1 deletion tests/fixtures/env/teams/demo/workloadValues/wd1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
a: b
{}
Empty file added tests/fixtures/env/teams/demo/workloadValues/wd2.managed.managed.yaml
View file
Open in desktop
Empty file.
2 changes: 1 addition & 1 deletion tests/fixtures/env/teams/demo/workloadValues/wd2.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
a: b
{}
Empty file added tests/fixtures/env/teams/demo/workloadValues/wd3.managed.managed.yaml
View file
Open in desktop
Empty file.
2 changes: 1 addition & 1 deletion tests/fixtures/env/teams/demo/workloadValues/wd3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
a: b
{}
4 changes: 4 additions & 0 deletions values-changes.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -392,3 +392,7 @@ changes:
- 'platformBackups.persistentVolumes'
- 'teamConfig.{team}.backups'
- 'apps.httpbin'
- 'apps.gitea.images'
- 'apps.gitea.resources.postgresql'
- 'apps.gitea.resources.memcached'
- 'apps.gitea.resources.memcachedMetrics'
76 changes: 49 additions & 27 deletions values-schema.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1686,28 +1686,22 @@ properties:
description: This password was generated and cannot be changed without manual intervention.
x-secret: '{{ randAlphaNum 20 }}'
readOnly: true
image:
database:
additionalProperties: false
properties:
gitea:
$ref: '#/definitions/imageSimple'
postgresql:
$ref: '#/definitions/imageSimple'
memcached:
$ref: '#/definitions/imageSimple'
maxConnections:
type: number
description: Maximum amount of total connections of the application to the database server
maxIdleConnections:
type: number
description: Maximum amount of idle connections of the application to the database server
resources:
additionalProperties: false
properties:
gitea:
$ref: '#/definitions/resources'
init:
$ref: '#/definitions/resources'
postgresql:
$ref: '#/definitions/resources'
memcached:
$ref: '#/definitions/resources'
memcachedMetrics:
$ref: '#/definitions/resources'
networkPolicies:
$ref: '#/definitions/appNetworkPolicyConfig'
grafana:
Expand Down Expand Up @@ -1779,8 +1773,15 @@ properties:
required:
- secret
- credentials
image:
$ref: '#/definitions/imageSimple'
database:
additionalProperties: false
properties:
maxConnections:
type: number
description: Maximum amount of total connections of the application to the database server
maxIdleConnections:
type: number
description: Maximum amount of idle connections of the application to the database server
resources:
additionalProperties: false
properties:
Expand Down Expand Up @@ -1903,6 +1904,18 @@ properties:
$ref: '#/definitions/imageSimple'
postgresql:
$ref: '#/definitions/imageSimple'
database:
additionalProperties: false
properties:
maxConnections:
type: number
description: Maximum amount of total connections of the application to the database server
minConnections:
type: number
description: Minimum amount of connections of the application to the database server
initialConnections:
type: number
description: Initial amount of connections of the application to the database server on startup
resources:
additionalProperties: false
properties:
Expand Down Expand Up @@ -3028,10 +3041,6 @@ properties:
enabled:
type: boolean
default: false
resources:
$ref: '#/definitions/resources'
backupSidecarResources:
$ref: '#/definitions/resources'
retentionPolicy:
$ref: '#/definitions/backupRetentionPolicy'
schedule:
Expand All @@ -3046,10 +3055,6 @@ properties:
enabled:
type: boolean
default: false
resources:
$ref: '#/definitions/resources'
backupSidecarResources:
$ref: '#/definitions/resources'
retentionPolicy:
$ref: '#/definitions/backupRetentionPolicy'
schedule:
Expand All @@ -3064,10 +3069,6 @@ properties:
enabled:
type: boolean
default: false
resources:
$ref: '#/definitions/resources'
backupSidecarResources:
$ref: '#/definitions/resources'
retentionPolicy:
$ref: '#/definitions/backupRetentionPolicy'
schedule:
Expand Down Expand Up @@ -3166,6 +3167,13 @@ properties:
type: string
resources:
$ref: '#/definitions/resources'
backupSidecarResources:
$ref: '#/definitions/resources'
postgresqlParameters:
type: object
additionalProperties:
type: string
default: {}
replicas:
type: integer
default: 2
Expand All @@ -3188,6 +3196,13 @@ properties:
type: string
resources:
$ref: '#/definitions/resources'
backupSidecarResources:
$ref: '#/definitions/resources'
postgresqlParameters:
type: object
additionalProperties:
type: string
default: {}
replicas:
type: integer
default: 2
Expand All @@ -3210,6 +3225,13 @@ properties:
type: string
resources:
$ref: '#/definitions/resources'
backupSidecarResources:
$ref: '#/definitions/resources'
postgresqlParameters:
type: object
additionalProperties:
type: string
default: {}
replicas:
type: integer
default: 2
Expand Down
8 changes: 3 additions & 5 deletions values/gitea/gitea-otomi-db.gotmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,7 @@ imageName: ghcr.io/{{ $cnpg.pgDefaultPlatformImage }}
{{- end }}

postgresql:
parameters:
max_connections: "32"
shared_buffers: 64MB
work_mem: 2MB
parameters: {{ toYaml $gdb.postgresqlParameters | nindent 4 }}

{{- if and $b.enabled (eq $obj.type "minioLocal" "linode") }}
backup:
Expand Down Expand Up @@ -66,4 +63,5 @@ clusterSpec:
externalClusters: {{ toYaml $gdb.externalClusters | nindent 4 }}
{{- end }}

resources: {{- toYaml $gdb.resources | nindent 2 }}
resources: {{ toYaml $gdb.resources | nindent 2 }}
backupSidecarResources: {{ toYaml $gdb.backupSidecarResources | nindent 2 }}
4 changes: 2 additions & 2 deletions values/gitea/gitea.gotmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,9 +128,9 @@ gitea:
name: gitea-db-secret
key: password
- name: GITEA__DATABASE__MAX_OPEN_CONNS
value: "32"
value: {{ $g.database.maxConnections | quote }}
- name: GITEA__DATABASE__MAX_IDLE_CONNS
value: "6"
value: {{ $g.database.maxIdleConnections | quote }}

- name: GITEA__CACHE__ADAPTER
value: "redis"
Expand Down
8 changes: 3 additions & 5 deletions values/harbor/harbor-otomi-db.gotmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,7 @@ imageName: ghcr.io/{{ $cnpg.pgDefaultPlatformImage }}
{{- end }}

postgresql:
parameters:
max_connections: "32"
shared_buffers: 64MB
work_mem: 2MB
parameters: {{ toYaml $hdb.postgresqlParameters | nindent 4 }}

{{- if and $b.enabled (eq $obj.type "minioLocal" "linode") }}
backup:
Expand Down Expand Up @@ -63,4 +60,5 @@ clusterSpec:
externalClusters: {{ toYaml . | nindent 4 }}
{{- end }}

resources: {{- toYaml $hdb.resources | nindent 2 }}
resources: {{ toYaml $hdb.resources | nindent 2 }}
backupSidecarResources: {{ toYaml $hdb.backupSidecarResources | nindent 2 }}
4 changes: 2 additions & 2 deletions values/harbor/harbor.gotmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,8 @@ core:
xsrfKey: {{ $h | get "core.xsrfKey" nil }}

database:
maxOpenConns: 32
maxIdleConns: 6
maxOpenConns: {{ $h.database.maxConnections }}
maxIdleConns: {{ $h.database.maxIdleConnections }}
type: external
external:
# erase default password value from the Harbor Helm chart
Expand Down
9 changes: 3 additions & 6 deletions values/keycloak/keycloak-otomi-db.gotmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,8 @@ imageName: {{ $v.otomi.linodeLkeImageRepository }}/ghcr/{{ $cnpg.pgDefaultPlatfo
imageName: ghcr.io/{{ $cnpg.pgDefaultPlatformImage }}
{{- end }}


postgresql:
parameters:
max_connections: "32"
shared_buffers: 64MB
work_mem: 2MB
parameters: {{ toYaml $kdb.postgresqlParameters | nindent 4 }}

{{- if and $b.enabled (eq $obj.type "minioLocal" "linode") }}
backup:
Expand Down Expand Up @@ -66,4 +62,5 @@ clusterSpec:
externalClusters: {{ toYaml . | nindent 4 }}
{{- end }}

resources: {{- toYaml $kdb.resources | nindent 2 }}
resources: {{ toYaml $kdb.resources | nindent 2 }}
backupSidecarResources: {{ toYaml $kdb.backupSidecarResources | nindent 2 }}
6 changes: 3 additions & 3 deletions values/keycloak/keycloak.gotmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,11 +85,11 @@ extraEnv: |
- name: KC_HOSTNAME
value: keycloak.{{ $v.cluster.domainSuffix }}
- name: KC_DB_POOL_INITIAL_SIZE
value: "5"
value: {{ $k.database.initialConnections | quote }}
- name: KC_DB_POOL_MIN_SIZE
value: "5"
value: {{ $k.database.minConnections | quote }}
- name: KC_DB_POOL_MAX_SIZE
value: "32"
value: {{ $k.database.maxConnections | quote }}
- name: KC_BOOTSTRAP_ADMIN_USERNAME
valueFrom:
secretKeyRef:
Expand Down