PMM-12619: Make PMM environment variables more consistent #2857

idoqo · 2024-02-28T20:24:33Z

Link to the Feature Build: SUBMODULES-0

If this PR adds or removes or alters one or more API endpoints, please review and add or update the relevant API documents as well:

API Docs updated

If this PR is related to some other PRs in this or other repositories, please provide links to those PRs:

Links to related pull requests (optional).

codecov · 2024-02-28T20:26:16Z

Codecov Report

All modified and coverable lines are covered by tests ✅

❗ No coverage uploaded for pull request base (v3@c6044df). Click here to learn what that means.

❗ Current head 8c2ad98 differs from pull request most recent head 9d049da. Consider uploading reports for the commit 9d049da to get more accurate results

Additional details and impacted files

@@          Coverage Diff          @@
##             v3    #2857   +/-   ##
=====================================
  Coverage      ?   43.52%           
=====================================
  Files         ?      364           
  Lines         ?    42558           
  Branches      ?        0           
=====================================
  Hits          ?    18525           
  Misses        ?    22427           
  Partials      ?     1606

Flag	Coverage Δ
admin	`11.54% <ø> (?)`
vmproxy	`68.57% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

managed/services/server/server.go

managed/utils/envvars/parser.go

managed/services/checks/checks.go

managed/CONTRIBUTING.md

.github/workflows/api-tests.yml

agent/docker-compose.yml

api-tests/docker-compose.yml

build/docker/server/README.md

docs/process/v2_to_v3_environment_variables.md

managed/utils/envvars/parser.go

agent/docker-compose.yml

build/docker/server/README.md

docs/process/v2_to_v3_environment_variables.md

BupycHuk

please check if we have any other env variables used that we missed in this PR

managed/CONTRIBUTING.md

BupycHuk · 2024-03-20T14:48:46Z

docker-compose.yml

+      - PMM_CLICKHOUSE_DATABASE=pmm
+      - PMM_CLICKHOUSE_BLOCK_SIZE=10000
+      - PMM_CLICKHOUSE_POOL_SIZE=2
+      - PMM_TEST_PMM_DISABLE_BUILTIN_CLICKHOUSE=1


@idoqo this one isn't addressed yet

docker-compose.yml

managed/services/supervisord/supervisord.go

managed/utils/envvars/parser.go

.github/workflows/api-tests.yml

ademidoff · 2024-04-02T11:33:39Z

managed/utils/envvars/parser.go

+//   - PMM_METRICS_RESOLUTION, PMM_METRICS_RESOLUTION_MR, PMM_METRICS_RESOLUTION_HR, PMM_METRICS_RESOLUTION_LR are durations of metrics resolution;
+//   - PMM_DATA_RETENTION is the duration of how long keep time-series data in ClickHouse;
+//   - PMM_ENABLE_AZURE_DISCOVER enables Azure Discover;
+//   - PMM_ENABLE_RBAC enables Access control;


@idoqo @BupycHuk
Guys, do we want this to be PMM_ENABLE_ACCESS_CONTROL going forward? That seems to be in line with the other migration, where we rename the gRPC service "Roles" to AccessControl.

BupycHuk · 2024-04-01T21:31:14Z

managed/services/supervisord/supervisord.go

+    PMM_POSTGRES_SSL_KEY_PATH="{{ .PostgresSSLKeyPath }}",
+    PMM_POSTGRES_SSL_CERT_PATH="{{ .PostgresSSLCertPath }}",
+    PMM_CLICKHOUSE_DATASOURCE_ADDR="{{ .ClickhouseDataSourceAddr }}",
+    PERCONA_DEV_PMM_CLICKHOUSE_HOST="{{ .ClickhouseHost }}",


please check if we have it anywhere else

I meant that it supposed to be PMM_CLICKHOUSE_HOST

BupycHuk · 2024-04-02T11:36:31Z

managed/utils/envvars/parser.go

+//   - PMM_METRICS_RESOLUTION, PMM_METRICS_RESOLUTION_MR, PMM_METRICS_RESOLUTION_HR, PMM_METRICS_RESOLUTION_LR are durations of metrics resolution;
+//   - PMM_DATA_RETENTION is the duration of how long keep time-series data in ClickHouse;
+//   - PMM_ENABLE_AZURE_DISCOVER enables Azure Discover;
+//   - PMM_ENABLE_RBAC enables Access control;


docs/process/v2_to_v3_environment_variables.md

managed/CONTRIBUTING.md

managed/utils/envvars/parser.go

BupycHuk

CI is failing and we still have PERCONA_DEV_PMM_CLICKHOUSE_HOST which supposed to be PMM_CLICKHOUSE_HOST

managed/utils/envvars/parser.go

BupycHuk · 2024-04-15T08:42:14Z

managed/utils/envvars/parser_test.go

-	t.Run("Unknown env variables", func(t *testing.T) {
-		t.Parallel()
-
-		envs := []string{"UNKNOWN_VAR=VAL", "ANOTHER_UNKNOWN_VAR=VAL"}
-		expectedEnvVars := &models.ChangeSettingsParams{}
-		expectedWarns := []string{
-			`unknown environment variable "UNKNOWN_VAR=VAL"`,
-			`unknown environment variable "ANOTHER_UNKNOWN_VAR=VAL"`,
-		}
-
-		gotEnvVars, gotErrs, gotWarns := ParseEnvVars(envs)
-		assert.Equal(t, gotEnvVars, expectedEnvVars)
-		assert.Nil(t, gotErrs)
-		assert.Equal(t, expectedWarns, gotWarns)
-	})
-


can we return this test?

managed/utils/envvars/parser_test.go

BupycHuk · 2024-04-23T08:59:08Z

CI is still failing with message

managed/services/server/server.go:491: line is 171 characters (lll)
  		return status.Error(codes.FailedPrecondition, "High resolution for metrics is set via PMM_METRICS_RESOLUTION_HR (or PMM_METRICS_RESOLUTION) environment variable.")

* refactored environment variables * update migration doc * replace test line * drop no lint rule * fix parser tests * re-order table values * remove deprecated env variables * fix tests * use singluar advisor * introduce PMM_DEV_ prefix for dev-related variables * clean up environment variables * drop printf * drop unused variable * update tests * add notes for removed variable * update oauth variables * update changed variable list * update variable check validation * refactor tests * surpress linter error

…2982) * PMM-12619: Make PMM environment variables more consistent (#2857) * refactored environment variables * update migration doc * replace test line * drop no lint rule * fix parser tests * re-order table values * remove deprecated env variables * fix tests * use singluar advisor * introduce PMM_DEV_ prefix for dev-related variables * clean up environment variables * drop printf * drop unused variable * update tests * add notes for removed variable * update oauth variables * update changed variable list * update variable check validation * refactor tests * surpress linter error * fix docker readme * restore error checks * include backup management variable * Update for grammar

refactored environment variables

e009ada

percona-robot reviewed Feb 28, 2024

View reviewed changes

managed/services/server/server.go Outdated Show resolved Hide resolved

managed/utils/envvars/parser.go Outdated Show resolved Hide resolved

managed/services/checks/checks.go Outdated Show resolved Hide resolved

update migration doc

526f2a6

idoqo commented Feb 28, 2024

View reviewed changes

managed/CONTRIBUTING.md Outdated Show resolved Hide resolved

idoqo added 3 commits February 28, 2024 21:54

replace test line

881108d

drop no lint rule

f40cc41

Merge branch 'v3' into PMM-12619-env-vars-rename

68a08ad

idoqo marked this pull request as ready for review March 3, 2024 11:17

idoqo requested review from a team as code owners March 3, 2024 11:17

idoqo requested review from ademidoff, BupycHuk and JiriCtvrtka and removed request for a team March 3, 2024 11:17

fix parser tests

21f48ea

BupycHuk reviewed Mar 4, 2024

View reviewed changes

idoqo added 3 commits March 5, 2024 19:09

re-order table values

3311ed6

remove deprecated env variables

00d71ba

fix tests

2812cb8

ademidoff reviewed Mar 8, 2024

View reviewed changes

agent/docker-compose.yml Outdated Show resolved Hide resolved

build/docker/server/README.md Show resolved Hide resolved

docs/process/v2_to_v3_environment_variables.md Show resolved Hide resolved

BupycHuk self-requested a review March 8, 2024 15:38

idoqo added 2 commits March 11, 2024 22:41

Merge branch 'v3' into PMM-12619-env-vars-rename

8a66c6d

use singluar advisor

47c31c6

idoqo force-pushed the PMM-12619-env-vars-rename branch from 5bf5bf7 to 47c31c6 Compare March 11, 2024 21:47

idoqo requested a review from ademidoff March 12, 2024 02:00

idoqo added 2 commits March 12, 2024 11:44

introduce PMM_DEV_ prefix for dev-related variables

dd23674

Merge branch 'v3' into PMM-12619-env-vars-rename

2819f8a

BupycHuk reviewed Mar 20, 2024

View reviewed changes

BupycHuk reviewed Mar 25, 2024

View reviewed changes

.github/workflows/api-tests.yml Outdated Show resolved Hide resolved

idoqo added 4 commits March 26, 2024 11:34

drop unused variable

0ae4b04

Merge branch 'v3' into PMM-12619-env-vars-rename

78bae3e

update tests

88b44ba

add notes for removed variable

e920f5f

idoqo requested a review from BupycHuk March 27, 2024 11:32

idoqo added 2 commits April 2, 2024 10:02

Merge branch 'v3' into PMM-12619-env-vars-rename

9c4c072

update oauth variables

3e64259

ademidoff approved these changes Apr 2, 2024

View reviewed changes

BupycHuk reviewed Apr 2, 2024

View reviewed changes

idoqo added 3 commits April 2, 2024 13:29

update changed variable list

d52729d

Merge branch 'v3' into PMM-12619-env-vars-rename

8c2ad98

update variable check validation

fd81967

BupycHuk requested changes Apr 15, 2024

View reviewed changes

idoqo added 3 commits April 23, 2024 09:12

refactor tests

4f0939b

Merge branch 'v3' into PMM-12619-env-vars-rename

1557b56

Merge branch 'v3' into PMM-12619-env-vars-rename

01d86a2

idoqo requested a review from BupycHuk April 23, 2024 08:19

BupycHuk approved these changes Apr 23, 2024

View reviewed changes

surpress linter error

9d049da

idoqo merged commit 3aed8f3 into v3 Apr 24, 2024
22 checks passed

idoqo deleted the PMM-12619-env-vars-rename branch April 24, 2024 10:26

ademidoff mentioned this pull request Apr 30, 2024

PMM-7 Update ENV variables in pmm3-aws-staging-start Percona-Lab/jenkins-pipelines#2621

Open

idoqo mentioned this pull request Apr 30, 2024

PMM-13090: Sync environment variables with v3-api-breaking-changes #2982

Merged

idoqo mentioned this pull request Jun 5, 2024

PMM-11231 PMM Updates UI #2904

Open

This was referenced Jun 5, 2024

PMM-7 Update docker compose file percona/grafana#755

Open

PMM-7 Update docker compose file percona/grafana-dashboards#1605

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PMM-12619: Make PMM environment variables more consistent #2857

PMM-12619: Make PMM environment variables more consistent #2857

idoqo commented Feb 28, 2024

codecov bot commented Feb 28, 2024 •

edited

Loading

BupycHuk left a comment

BupycHuk Mar 20, 2024

ademidoff Apr 2, 2024

BupycHuk Apr 2, 2024

BupycHuk Apr 1, 2024

BupycHuk Apr 2, 2024

BupycHuk Apr 3, 2024

BupycHuk Apr 2, 2024

BupycHuk left a comment

BupycHuk Apr 15, 2024

BupycHuk commented Apr 23, 2024

PMM-12619: Make PMM environment variables more consistent #2857

PMM-12619: Make PMM environment variables more consistent #2857

Conversation

idoqo commented Feb 28, 2024

codecov bot commented Feb 28, 2024 • edited Loading

Codecov Report

BupycHuk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BupycHuk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BupycHuk commented Apr 23, 2024

codecov bot commented Feb 28, 2024 •

edited

Loading