v0.218.1

This is a bugfix release.

CLI:

• Pass DATABRICKS_CONFIG_FILE for auth profiles (#1394).

Bundles:

• Show a better error message for using wheel tasks with older DBR versions (#1373).
• Allow variable references in non-string fields in the JSON schema (#1398).
• Fix variable overrides in targets for non-string variables (#1397).
• Fix bundle schema for variables (#1396).
• Fix bundle documentation URL (#1399).

Internal:

• Removed autogenerated docs for the CLI commands (#1392).
• Remove JSON.parse call from homebrew-tap action (#1393).
• Ensure that Python dependencies are installed during upgrade (#1390).

Snapshot

Snapshot build of the main branch. Updated automatically on each commit.

v0.218.0

This release marks the general availability of Databricks Asset Bundles.

CLI:

• Publish Docker images (#1353).
• Add support for multi-arch Docker images (#1362).
• Do not prefill https:// in prompt for Databricks Host (#1364).
• Add better documentation for the auth login command (#1366).
• Add URLs for authentication documentation to the auth command help (#1365).

Bundles:

• Fix compute override for foreach tasks (#1357).
• Transform artifact files source patterns in build not upload stage (#1359).
• Convert between integer and float in normalization (#1371).
• Disable locking for development mode (#1302).
• Resolve variable references inside variable lookup fields (#1368).
• Added validate mutator to surface additional bundle warnings (#1352).
• Upgrade terraform-provider-databricks to 1.40.0 (#1376).
• Print host in bundle validate when passed via profile or environment variables (#1378).
• Cleanup remote file path on bundle destroy (#1374).
• Add docs URL for run_as in error message (#1381).
• Enable job queueing by default (#1385).
• Added support for job environments (#1379).
• Processing and completion of positional args to bundle run (#1120).
• Add legacy option for run_as (#1384).

API Changes:

• Changed databricks lakehouse-monitors cancel-refresh command with new required argument order.
• Changed databricks lakehouse-monitors create command with new required argument order.
• Changed databricks lakehouse-monitors delete command with new required argument order.
• Changed databricks lakehouse-monitors get command with new required argument order.
• Changed databricks lakehouse-monitors get-refresh command with new required argument order.
• Changed databricks lakehouse-monitors list-refreshes command with new required argument order.
• Changed databricks lakehouse-monitors run-refresh command with new required argument order.
• Changed databricks lakehouse-monitors update command with new required argument order.
• Changed databricks account workspace-assignment update command to return response.

OpenAPI commit 94684175b8bd65f8701f89729351f8069e8309c9 (2024-04-11)

Dependency updates:

• Bump github.com/databricks/databricks-sdk-go from 0.37.0 to 0.38.0 (#1361).
• Bump golang.org/x/net from 0.22.0 to 0.23.0 (#1380).

v0.217.1

CLI:

• Don't attempt auth in auth profiles --skip-validate (#1282).
• Fixed typo in error template for auth describe (#1341).

Bundles:

• Correctly transform libraries in for_each_task block (#1340).
• Do not emit warning on YAML anchor blocks (#1354).
• Fixed pre-init script order (#1348).
• Execute preinit after entry point to make sure scripts are loaded (#1351).

Dependency updates:

• Bump internal terraform provider version to 1.39 (#1339).
• Bump golang.org/x/term from 0.18.0 to 0.19.0 (#1343).
• Bump github.com/hashicorp/hc-install from 0.6.3 to 0.6.4 (#1344).
• Bump golang.org/x/mod from 0.16.0 to 0.17.0 (#1345).
• Bump golang.org/x/oauth2 from 0.18.0 to 0.19.0 (#1347).
• Bump golang.org/x/sync from 0.6.0 to 0.7.0 (#1346).

Changes

Tests

v0.217.0

Breaking Change:

• Add allow list for resources when bundle run_as is set (#1233).
• Make bundle validation print text output by default (#1335).

CLI:

• Added auth describe command (#1244).
• Fixed message for successful auth describe run (#1336).

Bundles:

• Use UserName field to identify if service principal is used (#1310).
• Allow unknown properties in the config file for template initialization (#1315).
• Remove support for DATABRICKS_BUNDLE_INCLUDES (#1317).
• Make bundle.deployment optional in the bundle schema (#1321).
• Fix the generated DABs JSON schema (#1322).
• Make bundle loaders return diagnostics (#1319).
• Add bundle debug terraform command (#1294).
• Allow specifying CLI version constraints required to run the bundle (#1320).

Internal:

• Retain location information of variable reference (#1333).
• Define dyn.Mapping to represent maps (#1301).
• Return diag.Diagnostics from mutators (#1305).
• Fix flaky test in libs/process (#1314).
• Move path field to bundle type (#1316).
• Load bundle configuration from mutator (#1318).
• Return diagnostics from config.Load (#1324).
• Return warning for nil primitive types during normalization (#1329).
• Include dyn.Path in normalization warnings and errors (#1332).
• Make normalization return warnings instead of errors (#1334).

v0.216.0

CLI:

• Propagate correct User-Agent for CLI during OAuth flow (#1264).
• Add usage string when command fails with incorrect arguments (#1276).

Bundles:

• Include dyn.Path as argument to the visit callback function (#1260).
• Inline logic to set a value in dyn.SetByPath (#1261).
• Add assertions for the dyn.Path argument to the visit callback (#1265).
• Add dyn.MapByPattern to map a function to values with matching paths (#1266).
• Filter current user from resource permissions (#1262).
• Retain location annotation when expanding globs for pipeline libraries (#1274).
• Added deployment state for bundles (#1267).
• Do CheckRunningResource only after terraform.Write (#1292).
• Rewrite relative paths using dyn.Location of the underlying value (#1273).
• Push deployment state right after files upload (#1293).
• Make Append function to dyn.Path return independent slice (#1295).
• Move bundle tests into bundle/tests (#1299).
• Upgrade Terraform provider to 1.38.0 (#1308).

Internal:

• Add integration test for mlops-stacks initialization (#1155).
• Update actions/setup-python to v5 (#1290).
• Update codecov/codecov-action to v4 (#1291).

API Changes:

• Changed databricks catalogs list command.
• Changed databricks online-tables create command.
• Changed databricks lakeview publish command.
• Added databricks lakeview create command.
• Added databricks lakeview get command.
• Added databricks lakeview get-published command.
• Added databricks lakeview trash command.
• Added databricks lakeview update command.
• Moved settings related commands to databricks settings and databricks account settings.

OpenAPI commit 93763b0d7ae908520c229c786fff28b8fd623261 (2024-03-20)

Dependency updates:

• Bump golang.org/x/oauth2 from 0.17.0 to 0.18.0 (#1270).
• Bump golang.org/x/mod from 0.15.0 to 0.16.0 (#1271).
• Update Go SDK to v0.35.0 (#1300).
• Update Go SDK to v0.36.0 (#1304).

v0.215.0

CLI:

• The SDK update fixes fs cp calls timing out when copying large files.

Bundles:

• Fix summary command when internal Terraform config doesn't exist (#1242).
• Configure cobra.NoArgs for bundle commands where applicable (#1250).
• Fixed building Python artifacts on Windows with WSL (#1249).
• Add --validate-only flag to run validate-only pipeline update (#1251).
• Only transform wheel libraries when using trampoline (#1248).
• Return application_id for service principal lookups (#1245).
• Support relative paths in artifact files source section and always upload all artifact files (#1247).
• Fix DBConnect support in VS Code (#1253).

Internal:

• Added test to verify scripts.Execute mutator works correctly (#1237).

API Changes:

• Added databricks permission-migration command group.
• Updated nesting of the databricks settings and databricks account settings commands
• Changed databricks vector-search-endpoints delete-endpoint command with new required argument order.
• Changed databricks vector-search-indexes create-index command with new required argument order.
• Changed databricks vector-search-indexes delete-data-vector-index command with new required argument order.
• Changed databricks vector-search-indexes upsert-data-vector-index command with new required argument order.

OpenAPI commit d855b30f25a06fe84f25214efa20e7f1fffcdf9e (2024-03-04)

Dependency updates:

• Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 (#1252).
• Update Go SDK to v0.34.0 (#1256).

v0.214.1

CLI:

• Improved error message when no .databrickscfg (#1223).
• Use Go SDK Iterators when listing resources with the CLI (#1202).

Bundles:

• Only set ComputeID value when --compute-id flag provided (#1229).
• Add correct tag value for models in dev mode (#1230).
• Upgrade Terraform provider to 1.37.0 (#1235).

Internal:

• Fix CLI nightlies on our UC workspaces (#1225).
• Handle alias types for map keys in toTyped conversion (#1232).

v0.214.0

CLI:

• Add support for UC Volumes to the databricks fs commands (#1209).

Bundles:

• Use dynamic configuration model in bundles (#1098).
• Allow use of variables references in primitive non-string fields (#1219).
• Add an experimental default-sql template (#1051).
• Add an experimental dbt-sql template (#1059).

Internal:

• Add fork-user to winget release workflow (#1214).
• Use any as type for data sources and resources in tf/schema (#1216).
• Avoid infinite recursion when normalizing a recursive type (#1213).
• Fix issue where interpolating a new ref would rewrite unrelated fields (#1217).
• Use dyn.Value as input to generating Terraform JSON (#1218).

API Changes:

• Changed databricks lakehouse-monitors update command with new required argument order.
• Added databricks online-tables command group.

OpenAPI commit cdd76a98a4fca7008572b3a94427566dd286c63b (2024-02-19)
Dependency updates:

• Bump Terraform provider to v1.36.2 (#1215).
• Bump github.com/databricks/databricks-sdk-go from 0.32.0 to 0.33.0 (#1222).

v0.213.0

CLI:

• Ignore environment variables for auth profiles (#1189).
• Update LICENSE file to match Databricks license language (#1013).

Bundles:

• Added bundle deployment bind and unbind command (#1131).
• Use allowlist for Git-related fields to include in metadata (#1187).
• Added --restart flag for bundle run command (#1191).
• Generate correct YAML if custom_tags or spark_conf is used for pipeline or job cluster configuration (#1210).

Internal:

• Move folders package into libs (#1184).
• Log time it takes for profile to load (#1186).
• Use mockery to generate mocks compatible with testify/mock (#1190).
• Retain partially valid structs in convert.Normalize (#1203).
• Skip for_each_task when generating the bundle schema (#1204).
• Regenerate the CLI using the same OpenAPI spec as the SDK (#1205).
• Avoid race-conditions while executing sub-commands (#1201).

API Changes:

• Added databricks tables exists command.
• Added databricks lakehouse-monitors command group.
• Removed databricks files get-status command.
• Added databricks files create-directory command.
• Added databricks files delete-directory command.
• Added databricks files get-directory-metadata command.
• Added databricks files get-metadata command.
• Added databricks files list-directory-contents command.
• Removed databricks pipelines reset command.
• Changed databricks account settings delete-personal-compute-setting command with new required argument order.
• Removed databricks account settings read-personal-compute-setting command.
• Changed databricks account settings update-personal-compute-setting command with new required argument order.
• Added databricks account settings get-personal-compute-setting command.
• Removed databricks settings delete-default-workspace-namespace command.
• Removed databricks settings read-default-workspace-namespace command.
• Removed databricks settings update-default-workspace-namespace command.
• Added databricks settings delete-default-namespace-setting command.
• Added databricks settings delete-restrict-workspace-admins-setting command.
• Added databricks settings get-default-namespace-setting command.
• Added databricks settings get-restrict-workspace-admins-setting command.
• Added databricks settings update-default-namespace-setting command.
• Added databricks settings update-restrict-workspace-admins-setting command.
• Changed databricks token-management create-obo-token command with new required argument order.
• Changed databricks token-management get command to return .
• Changed databricks dashboards create command . New request type is .
• Added databricks dashboards update command.

OpenAPI commit c40670f5a2055c92cf0a6aac92a5bccebfb80866 (2024-02-14)
Dependency updates:

• Bump github.com/hashicorp/hc-install from 0.6.2 to 0.6.3 (#1200).
• Bump golang.org/x/term from 0.16.0 to 0.17.0 (#1197).
• Bump golang.org/x/oauth2 from 0.16.0 to 0.17.0 (#1198).
• Bump github.com/databricks/databricks-sdk-go from 0.30.1 to 0.32.0 (#1199).