Functionality to walk a `config.Value` tree #1081

pietern · 2023-12-20T09:56:15Z

Changes

This change adds:

A config.Walk function to walk a configuration tree
A config.Path type to represent a value's path inside a tree
Functions to create a config.Path from a string, or convert one to a string

Tests

Additional unit tests with full coverage.

This change adds: * A `config.Walk` function to walk a configuration tree * A `config.Path` type to represent a value's path inside a tree * Functions to create a `config.Path` from a string, or convert one to a string

pietern · 2023-12-20T09:57:14Z

This change is in support of another WIP change to start using config.Value for our bundle configuration.

I broke this out into a separate PR to keep the main one reviewable.

libs/config/path_string_test.go

andrewnester · 2023-12-21T15:18:59Z

libs/config/path_string_test.go

+		},
+		{
+			// But after an index there must be a dot.
+			input: "foo[1]bar",


Spaces are not allowed in the path, is it correct?

This is not covered in the parsing code. They will become part of the keys where applicable (i.e. in the middle of a key, or after an index).

Do you think it's worth defining rules for these?

I think we can follow up on this later including addressing dots in the key names (if we have a case for this)

libs/config/walk.go

andrewnester · 2023-12-22T10:27:40Z

libs/config/path_string.go

+//   - foo.bar[1]
+//   - foo.bar[1][2]
+//   - .
+func NewPathFromString(input string) (Path, error) {


I just realised that since this will be eventually used for working with real config, we might run into issues where the key itself can contain . like for example

"spark_conf": { "spark.databricks.cluster.profile": "singleNode", "spark.master": "local[*, 4]" },

What if we format the path in the following way instead

"foo"[1]."bar"

Then if the key has spaces or dots the path will be less confusing, like

"foo.foo"[1]."bar baz"

Great suggestion -- we can do this later if/when we need to.

andrewnester

Chatted offline: we can followup on dots and spaces in keys later if needed

CLI: * Tweak command groups in CLI help ([#1092](#1092)). * Fixed storage-credentials list command in text output ([#1094](#1094)). * Allow account client auth with environment variables when no `.databrickscfg` file present ([#1097](#1097)). * Always log with text format by default ([#1096](#1096)). Bundles: * Added exec.NewCommandExecutor to execute commands with correct interpreter ([#1075](#1075)). * Use resource key as name in permissions code ([#1087](#1087)). * Add support for reprompts if user input does not match template schema ([#946](#946)). * Improve experience for multiple builtin templates ([#1052](#1052)). * Improve error when bundle root is not writable ([#1093](#1093)). * Change recommended production deployment path from `/Shared` to `/Users` ([#1091](#1091)). * Allow referencing bundle resources by name ([#872](#872)). * Make libs/exec fallback to `sh` if `bash` cannot be found ([#1114](#1114)). Internal: * Functionality to walk a `config.Value` tree ([#1081](#1081)). * Rename libs/config -> libs/dyn ([#1086](#1086)). * Added codecov integration on code push ([#1095](#1095)). * Consolidate functions to convert `dyn.Value` to native types ([#1100](#1100)). * Define constant for the invalid `dyn.Value` ([#1101](#1101)). API Changes: * Changed `databricks serving-endpoints update-config` command with new required argument order. * Changed `databricks account ip-access-lists create` command with new required argument order. * Changed `databricks account ip-access-lists replace` command with new required argument order. * Changed `databricks account ip-access-lists update` command with new required argument order. * Changed `databricks ip-access-lists create` command with new required argument order. * Changed `databricks ip-access-lists replace` command with new required argument order. * Changed `databricks ip-access-lists update` command with new required argument order. OpenAPI commit a7a9dc025bb80303e676bf3708942c6aa06689f1 (2024-01-04) Dependency updates: * Bump github.com/hashicorp/terraform-exec from 0.19.0 to 0.20.0 ([#1088](#1088)). * Bump go.uber.org/mock from 0.3.0 to 0.4.0 ([#1089](#1089)). * Bump github.com/hashicorp/terraform-json from 0.18.0 to 0.20.0 ([#1090](#1090)). * Bump golang.org/x/term from 0.15.0 to 0.16.0 ([#1103](#1103)). * Bump github.com/cloudflare/circl from 1.3.3 to 1.3.7 ([#1105](#1105)). * Bump golang.org/x/sync from 0.5.0 to 0.6.0 ([#1104](#1104)). * Upgrade golang.org/x/crypto to v0.17.0 in internal module ([#1110](#1110)). * Upgrade Go SDK to v0.29.0 ([#1111](#1111)).

Functionality to walk a config.Value tree

875454d

This change adds: * A `config.Walk` function to walk a configuration tree * A `config.Path` type to represent a value's path inside a tree * Functions to create a `config.Path` from a string, or convert one to a string

pietern requested review from andrewnester and shreyas-goenka December 20, 2023 09:56

Comments

89cae7c

andrewnester reviewed Dec 21, 2023

View reviewed changes

Address comments

40e1425

pietern requested a review from andrewnester December 22, 2023 09:59

andrewnester reviewed Dec 22, 2023

View reviewed changes

andrewnester approved these changes Dec 22, 2023

View reviewed changes

pietern added this pull request to the merge queue Dec 22, 2023

Merged via the queue into main with commit a1297d7 Dec 22, 2023
4 checks passed

pietern deleted the value-path branch December 22, 2023 10:43

pietern mentioned this pull request Jan 11, 2024

Release v0.212.0 #1116

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Functionality to walk a `config.Value` tree #1081

Functionality to walk a `config.Value` tree #1081

pietern commented Dec 20, 2023

pietern commented Dec 20, 2023

andrewnester Dec 21, 2023

pietern Dec 22, 2023

andrewnester Dec 22, 2023

andrewnester Dec 22, 2023

pietern Dec 22, 2023

andrewnester left a comment •

edited

Loading

Functionality to walk a config.Value tree #1081

Functionality to walk a config.Value tree #1081

Conversation

pietern commented Dec 20, 2023

Changes

Tests

pietern commented Dec 20, 2023

andrewnester Dec 21, 2023

Choose a reason for hiding this comment

pietern Dec 22, 2023

Choose a reason for hiding this comment

andrewnester Dec 22, 2023

Choose a reason for hiding this comment

andrewnester Dec 22, 2023

Choose a reason for hiding this comment

pietern Dec 22, 2023

Choose a reason for hiding this comment

andrewnester left a comment • edited Loading

Choose a reason for hiding this comment

Functionality to walk a `config.Value` tree #1081

Functionality to walk a `config.Value` tree #1081

andrewnester left a comment •

edited

Loading