Skip to content

helm: make MySQL DB_URL extensible/overridable, wire Grafana datasource envs, and add smoke-test CI#350

Merged
klesh merged 7 commits into
apache:mainfrom
kahirokunn:support-custom-params
Sep 3, 2025
Merged

helm: make MySQL DB_URL extensible/overridable, wire Grafana datasource envs, and add smoke-test CI#350
klesh merged 7 commits into
apache:mainfrom
kahirokunn:support-custom-params

Conversation

@kahirokunn
Copy link
Copy Markdown
Contributor

@kahirokunn kahirokunn commented Sep 2, 2025

Fixes #340 #349

What changed (approach)

  1. Extensible & overridable MySQL DB URL

    • New value: mysql.extraParams (e.g., &tls=skip-verify), appended to the generated DSN.
    • New value: option.assembleDbUrl (default true). When set to false, the chart skips auto-assembly so users can supply DB_URL via lake.envs or an external Secret/ConfigMap.
    • Plumbs DB_CUSTOM_PARAMS via ConfigMap to keep env composition explicit.
  2. Grafana datasource picks up MySQL envs

    • Adds option.connectionConfigmapName and wires it into grafana.extraEnvFrom, ensuring the MYSQL_* envs are visible to Grafana for the DevLake datasource initialization.
    • Keeps the default name stable, but allows overriding where installations differ.
  3. New GitHub Actions smoke-test (repro & prevention)

    • Spins up a temporary kind cluster, installs the chart with Helm, waits for readiness, and performs basic health checks (Grafana up; MySQL datasource connectable).
    • Runs on every push and PR, acting as a regression guard for the DB/Grafana integration path.

Why this solves the issues

Backward compatibility

  • option.assembleDbUrl: true by default → current installs keep their behavior.
  • mysql.extraParams defaults to empty (no change).

New/updated values

mysql:
  extraParams: ""   # e.g., "&tls=skip-verify&autocommit=true"

option:
  assembleDbUrl: true
  connectionConfigmapName: "devlake-mysql-auth-config"

Local verification

I reproduced the scenarios locally and confirmed the chart works end-to-end (install, readiness, and Grafana datasource connectivity) with the changes in this PR.

Result: Works completely in my local environment.

Screenshot (attached):

CleanShot 2025-09-03 at 00 33 18@2x

Notes for reviewers

  • CI is intentionally lightweight (kind + Helm) to keep signal high and cost low.

- Introduce `DB_CUSTOM_PARAMS` in `configmap.yaml`.
- Enhance `DB_URL` in `deployments.yaml` to include custom params.
- Define `mysql.extraParams` in `values.yaml` for configuration.

Signed-off-by: kahirokunn <okinakahiro@gmail.com>
Signed-off-by: kahirokunn <okinakahiro@gmail.com>
@kahirokunn kahirokunn force-pushed the support-custom-params branch 2 times, most recently from 51b3336 to 936fe3b Compare September 2, 2025 15:08
@kahirokunn kahirokunn marked this pull request as draft September 2, 2025 15:14
@kahirokunn kahirokunn force-pushed the support-custom-params branch 2 times, most recently from 137df06 to 3563d50 Compare September 2, 2025 15:33
@kahirokunn kahirokunn marked this pull request as ready for review September 2, 2025 15:33
@kahirokunn kahirokunn changed the title Support custom params and Add Smoke test CI helm: make MySQL DB_URL extensible/overridable, wire Grafana datasource envs, and add smoke-test CI Sep 2, 2025
- Introduced `connectionConfigmapName` to optionally override the default ConfigMap name for better configurability.

Signed-off-by: kahirokunn <okinakahiro@gmail.com>
@kahirokunn kahirokunn force-pushed the support-custom-params branch 8 times, most recently from bfd2d43 to c7bf213 Compare September 3, 2025 00:32
Signed-off-by: kahirokunn <okinakahiro@gmail.com>
Signed-off-by: kahirokunn <okinakahiro@gmail.com>
Signed-off-by: kahirokunn <okinakahiro@gmail.com>
…ng MySQL probes to TCP

Signed-off-by: kahirokunn <okinakahiro@gmail.com>
@kahirokunn kahirokunn force-pushed the support-custom-params branch from b0a9def to 8417819 Compare September 3, 2025 02:13
Copy link
Copy Markdown
Contributor

@klesh klesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@klesh klesh merged commit 9d32ec3 into apache:main Sep 3, 2025
3 checks passed
@klesh
Copy link
Copy Markdown
Contributor

klesh commented Sep 3, 2025

Release is failing https://github.com/apache/incubator-devlake-helm-chart/actions/runs/17422005314
Can you help take a look when you find time, thanks.

@kahirokunn
Copy link
Copy Markdown
Contributor Author

Hi @klesh ,

I looked into the failing release run https://github.com/apache/incubator-devlake-helm-chart/actions/runs/17422005314. It’s being blocked by our Actions allowlist policy— helm/kind-action@v1.12.0 isn’t currently permitted.

Would you be open to adding helm/kind-action to the allowlist? The helm organization is widely used and trusted in the Kubernetes community, and this action is a straightforward way to provision a local kind cluster for our chart tests.
If adding it isn’t feasible, I can rework the workflow to set up kind/Helm/kubectl via shell steps (or approved/verified actions). That approach will work, though it does introduce a bit more maintenance for future version bumps.

Please let me know which direction you’d prefer—happy to adjust accordingly.
Thanks a lot for your help!

@kahirokunn kahirokunn deleted the support-custom-params branch September 3, 2025 04:01
@klesh
Copy link
Copy Markdown
Contributor

klesh commented Sep 3, 2025

@kahirokunn Thanks for the quick response.
I personally am open the the idea, unfortunately, Apache Policy doesn't allow 3rd party actions.
Can we workaround that?

@kahirokunn
Copy link
Copy Markdown
Contributor Author

Okay, I have created a new PR.
I hope you will check it out.
#351

@klesh
Copy link
Copy Markdown
Contributor

klesh commented Sep 3, 2025

Thank you very much for the quick fix. 👍

@kahirokunn
Copy link
Copy Markdown
Contributor Author

Thank you so much for your kind words! I truly appreciate your support.
I’m always grateful for the amazing open-source community, and I’m glad if I can contribute—even in a small way—to making it better together.

fabioluciano pushed a commit to fabioluciano/incubator-devlake-helm-chart that referenced this pull request May 6, 2026
…urce envs, and add smoke-test CI (apache#350)

* Add custom MySQL DSN query params feature

- Introduce `DB_CUSTOM_PARAMS` in `configmap.yaml`.
- Enhance `DB_URL` in `deployments.yaml` to include custom params.
- Define `mysql.extraParams` in `values.yaml` for configuration.

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* Enable optional DB URL auto-assembly for MySQL

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* fix: Pass MySQL env vars to Grafana for datasource setup

- Introduced `connectionConfigmapName` to optionally override the default ConfigMap name for better configurability.

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* Add Helm smoke test workflow

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* Link Secret and ConfigMap names in values.yaml

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* Add customizable probes for MySQL containers

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* fix(mysql): Resolve environment-dependent startup failures by switching MySQL probes to TCP

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

---------

Signed-off-by: kahirokunn <okinakahiro@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1.0.2 release broke grafana devlake datasource in the default installation

2 participants