Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make bundle loaders return diagnostics #1319

Merged
merged 18 commits into from
Mar 28, 2024
Merged

Make bundle loaders return diagnostics #1319

merged 18 commits into from
Mar 28, 2024

Conversation

pietern
Copy link
Contributor

@pietern pietern commented Mar 27, 2024

Changes

The function signature of Cobra's PreRunE function has an error return value. We'd like to start returning diag.Diagnostics after loading a bundle, so this is incompatible. This change modifies all usage of PreRunE to load a bundle to inline function calls in the command's RunE function.

Tests

  • Unit tests pass.
  • Integration tests pass.

The bundle path was previously stored on the `config.Root` type under the
assumption that the first configuration file being loaded would set it. This is
slightly counterintuitive and we know what the path is upon construction of the
bundle. The new location for this property reflects this.
PR #604 added functionality to load a bundle without a `databricks.yml` if both
the `DATABRICKS_BUNDLE_ROOT` and `DATABRICKS_BUNDLE_INCLUDES` environment
variables were set. We never ended up using this in downstream tools so this
can be removed.
This is incompatible with the function signature of Cobra's `PreRunE`
functions, so this change updates any use of PreRunE to be inlined in the main
run function.
Base automatically changed from bundle-load-from-mutator to main March 27, 2024 10:54
@pietern pietern marked this pull request as ready for review March 27, 2024 12:10
@pietern
Copy link
Contributor Author

pietern commented Mar 27, 2024

Integration tests pass.

@pietern pietern enabled auto-merge March 27, 2024 12:55
Copy link
Contributor

@shreyas-goenka shreyas-goenka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM. You should add a unit test for the profile priority order change and also call it out in the description though.

@pietern pietern added this pull request to the merge queue Mar 28, 2024
Merged via the queue into main with commit b21e3c8 Mar 28, 2024
4 checks passed
@pietern pietern deleted the bundle-load-root branch March 28, 2024 10:39
github-merge-queue bot pushed a commit that referenced this pull request Mar 28, 2024
## Changes

We no longer need to store load diagnostics on the `config.Root` type
itself and instead can return them from the `config.Load` call directly.
It is up to the caller of this function to append them to previous
diagnostics, if any.

Background: previous commits moved configuration loading of the entry
point into a mutator, so now all diagnostics naturally flow from
applying mutators.

This PR depends on #1319.

## Tests

Unit and manual validation of the debug statements in the validate
command.
github-merge-queue bot pushed a commit that referenced this pull request Apr 3, 2024
## Changes

Errors in normalization mean hard failure as of #1319.

We currently allow malformed configurations and ignore the malformed
fields and should continue to do so.

## Tests

* Tests pass.
* No calls to `diag.Errorf` from `libs/dyn`
andrewnester added a commit that referenced this pull request Apr 3, 2024
CLI:
 * Added `auth describe` command ([#1244](#1244)).
 * Fixed message for successful auth describe run ([#1336](#1336)).

Bundles:
 * Make bundle validation print text output by default ([#1335](#1335)).
 * Use UserName field to identify if service principal is used ([#1310](#1310)).
 * Allow unknown properties in the config file for template initialization ([#1315](#1315)).
 * Remove support for DATABRICKS_BUNDLE_INCLUDES ([#1317](#1317)).
 * Make `bundle.deployment` optional in the bundle schema ([#1321](#1321)).
 * Add allow list for resources when bundle `run_as` is set ([#1233](#1233)).
 * Fix the generated DABs JSON schema ([#1322](#1322)).
 * Make bundle loaders return diagnostics ([#1319](#1319)).
 * Add `bundle debug terraform` command ([#1294](#1294)).
 * Allow specifying CLI version constraints required to run the bundle ([#1320](#1320)).

Internal:
 * Retain location information of variable reference ([#1333](#1333)).
 * Define `dyn.Mapping` to represent maps ([#1301](#1301)).
 * Return `diag.Diagnostics` from mutators ([#1305](#1305)).
 * Fix flaky test in `libs/process` ([#1314](#1314)).
 * Move path field to bundle type ([#1316](#1316)).
 * Load bundle configuration from mutator ([#1318](#1318)).
 * Return diagnostics from `config.Load` ([#1324](#1324)).
 * Return warning for nil primitive types during normalization ([#1329](#1329)).
 * Include `dyn.Path` in normalization warnings and errors ([#1332](#1332)).
 * Make normalization return warnings instead of errors ([#1334](#1334)).

API Changes:
 * Added `databricks lakeview migrate` command.
 * Added `databricks lakeview unpublish` command.
 * Changed `databricks ip-access-lists get` command . New request type is .

OpenAPI commit e316cc3d78d087522a74650e26586088da9ac8cb (2024-04-03)
Dependency updates:
 * Bump github.com/databricks/databricks-sdk-go from 0.36.0 to 0.37.0 ([#1326](#1326)).
@andrewnester andrewnester mentioned this pull request Apr 3, 2024
github-merge-queue bot pushed a commit that referenced this pull request Apr 3, 2024
Breaking Change:
* Add allow list for resources when bundle `run_as` is set
([#1233](#1233)).
* Make bundle validation print text output by default
([#1335](#1335)).

CLI:
* Added `auth describe` command
([#1244](#1244)).
* Fixed message for successful auth describe run
([#1336](#1336)).

Bundles:
* Use UserName field to identify if service principal is used
([#1310](#1310)).
* Allow unknown properties in the config file for template
initialization ([#1315](#1315)).
* Remove support for DATABRICKS_BUNDLE_INCLUDES
([#1317](#1317)).
* Make `bundle.deployment` optional in the bundle schema
([#1321](#1321)).
* Fix the generated DABs JSON schema
([#1322](#1322)).
* Make bundle loaders return diagnostics
([#1319](#1319)).
* Add `bundle debug terraform` command
([#1294](#1294)).
* Allow specifying CLI version constraints required to run the bundle
([#1320](#1320)).

Internal:
* Retain location information of variable reference
([#1333](#1333)).
* Define `dyn.Mapping` to represent maps
([#1301](#1301)).
* Return `diag.Diagnostics` from mutators
([#1305](#1305)).
* Fix flaky test in `libs/process`
([#1314](#1314)).
* Move path field to bundle type
([#1316](#1316)).
* Load bundle configuration from mutator
([#1318](#1318)).
* Return diagnostics from `config.Load`
([#1324](#1324)).
* Return warning for nil primitive types during normalization
([#1329](#1329)).
* Include `dyn.Path` in normalization warnings and errors
([#1332](#1332)).
* Make normalization return warnings instead of errors
([#1334](#1334)).
API Changes:
 * Added `databricks lakeview migrate` command.
 * Added `databricks lakeview unpublish` command.
* Changed `databricks ip-access-lists get` command . New request type is
.

OpenAPI commit e316cc3d78d087522a74650e26586088da9ac8cb (2024-04-03)
Dependency updates:
* Bump github.com/databricks/databricks-sdk-go from 0.36.0 to 0.37.0
([#1326](#1326)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants