Library to convert config.Value to Go struct #904
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
andrewnester
approved these changes
Oct 23, 2023
pietern
added a commit
that referenced
this pull request
Oct 24, 2023
This is similar to #904 but instead of converting the dynamic configuration to Go structs, this normalizes a `config.Value` according to the type of a Go struct and returns the new, normalized `config.Value`. This will be used to ensure that two config.Value trees are type-compatible before we can merge them (i.e. instances from different files). Warnings and errors during normalization are accumulated and returned as a `diag.Diagnostics` structure. We can use this to surface warnings about unknown fields, or errors about invalid types, in aggregate instead of one-by-one.
github-merge-queue bot
pushed a commit
that referenced
this pull request
Oct 25, 2023
## Changes This is similar to #904 but instead of converting the dynamic configuration to Go structs, this normalizes a `config.Value` according to the type of a Go struct and returns the new, normalized `config.Value`. This will be used to ensure that two `config.Value` trees are type-compatible before we can merge them (i.e. instances from different files). Warnings and errors during normalization are accumulated and returned as a `diag.Diagnostics` structure. We can use this to surface warnings about unknown fields, or errors about invalid types, in aggregate instead of one-by-one. This approach is inspired by the pattern to accumulate diagnostics in Terraform provider code. ## Tests New unit tests.
shreyas-goenka
added a commit
that referenced
this pull request
Nov 2, 2023
CLI: * Fix URL for bundle template documentation ([#903](#903)). * Library to convert config.Value to Go struct ([#904](#904)). * Loading an empty file yields a nil ([#906](#906)). * Fix pattern validation for input properties ([#912](#912)). * Simplified code generation logic for handling path and request body parameters and JSON input ([#905](#905)). * Add support for multiline descriptions when using template enums ([#916](#916)). * Move bundle configuration filename code ([#917](#917)). * Add configuration normalization code ([#915](#915)). * Add welcome message to bundle templates ([#907](#907)). * Consolidate bundle configuration loader function ([#918](#918)). * Upload terraform state even if apply fails ([#923](#923)). * Use UserName instead of Id to check if identity used is a service principal ([#924](#924)). * `make snapshot` to build file in `.databricks/databricks` ([#927](#927)). * Persist deployment metadata in WSFS ([#845](#845)). * Run make fmt from fmt job ([#929](#929)). * Add override to support YAML inputs for apps ([#921](#921)). * Add GitHub issue templates ([#925](#925)). * Remove resolution of repo names against the Databricks Github account ([#940](#940)). * Fix metadata computation for empty bundle ([#939](#939)). Bundles: * **FILL THIS IN MANUALLY BY MOVING RELEVANT ITEMS FROM ABOVE LIST** Internal: * **FILL THIS IN MANUALLY BY MOVING RELEVANT ITEMS FROM ABOVE LIST** API Changes: * Added `databricks apps` command group. * Added `databricks account network-policy` command group. OpenAPI commit 5903bb39137fd76ac384b2044e425f9c56840e00 (2023-10-23) Dependency updates: * Bump google.golang.org/grpc from 1.58.2 to 1.58.3 ([#920](#920)). * Bump the Go SDK in the CLI ([#919](#919)). * Bump Terraform provider to v1.29.0 ([#926](#926)). * Bump github.com/google/uuid from 1.3.1 to 1.4.0 ([#932](#932)).
Merged
github-merge-queue bot
pushed a commit
that referenced
this pull request
Nov 2, 2023
CLI: * Added GitHub issue templates for CLI and DABs issues ([#925](#925)). * Added override to support YAML inputs for apps ([#921](#921)). * Simplified code generation logic for handling path and request body parameters and JSON input ([#905](#905)). Bundles: * Fixed URL for bundle template documentation in init command help docs ([#903](#903)). * Fixed pattern validation for input parameters in a bundle template ([#912](#912)). * Fixed multiline description rendering for enum input parameters in bundle templates ([#916](#916)). * Changed production mode check for whether identity used is a service principal to use UserName ([#924](#924)). * Changed bundle deploy to upload partial terraform state even if deployment fails ([#923](#923)). * Added support for welcome messages to bundle templates ([#907](#907)). * Added support for uploading bundle deployment metadata to WSFS ([#845](#845)). Internal: * Loading an empty yaml file yields a nil ([#906](#906)). * Library to convert config.Value to Go struct ([#904](#904)). * Remove default resolution of repo names against the Databricks Github account([#940](#940)). * Run make fmt from fmt job ([#929](#929)). * `make snapshot` to build file in `.databricks/databricks` ([#927](#927)). * Add configuration normalization code ([#915](#915)). API Changes: * Added `databricks apps` command group. * Added `databricks account network-policy` command group. Dependency updates: * Bump Terraform provider from v1.28.0 to v1.29.0 ([#926](#926)). * Bump the Go SDK in the CLI from v0.23 to v0.24 ([#919](#919)). * Bump google.golang.org/grpc from 1.58.2 to 1.58.3 ([#920](#920)). * Bump github.com/google/uuid from 1.3.1 to 1.4.0 ([#932](#932)). OpenAPI commit 5903bb39137fd76ac384b2044e425f9c56840e00 (2023-10-23)
github-merge-queue bot
pushed a commit
that referenced
this pull request
Nov 15, 2023
## Changes This PR is the counterpart to #904. With this change, we are able to convert a `config.Value` into a Go struct, make modifications to the Go struct, and reflect those changes in a new `config.Value`. This functionality allows us to incrementally introduce this configuration representation to existing bundle mutators. Bundle mutators expect a `*bundle.Bundle` argument and mutate its configuration directly. These mutations are not reflected in the corresponding `config.Value` (once introduced), which means we cannot use the `config.Value` as source of truth until we update _all_ mutators. To address this, we can run `convert.ToTyped` and `convert.FromTyped` at the mutator boundary (from `bundle.Apply`) and capture changes made to the Go struct. Then we can incrementally make mutators aware of the `config.Value` configuration and have them mutate that structure directly. ## Tests New unit tests pass. Manual spot checks against the bundle configuration type.
github-merge-queue bot
pushed a commit
that referenced
this pull request
Jan 5, 2024
## Changes The nil value is a real valid value that we need to represent. To accommodate this we introduced `dyn.KindInvalid` as the zero-value for `dyn.Kind` (see #904), but did not yet update the comments on `dyn.NilValue` or add tests for `kind.go`. This also moves `KindNil` to be last in the definition order (least likely to care about it). ## Tests Tests pass.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
Now that we have a new YAML loader (see #828), we need code to turn this into our Go structs.
Tests
New unit tests pass.
Confirmed that we can replace our existing loader/converter with this one and that existing unit tests for bundle loading still pass.