Skip to content

Latest commit

 

History

History
2010 lines (1555 loc) · 230 KB

CHANGELOG.md

File metadata and controls

2010 lines (1555 loc) · 230 KB
Raw

Version changelog

1.41.0

New Features and Improvements

  • Added resources for configuring Serverless network connectivity (#3402).
  • Data source databricks_current_metastore should not error out for non-UC workspaces (#3497).

Exporter

  • Adjust retry error on deadline exceeded (#3491).

Internal Changes

  • Added account workspace setting type (#3415).
  • Added test config (#3487).
  • Refactor CustomizeSchema for ResourceProvider to skip unfeasible customizations (#3481).
  • Bump golang.org/x/net from 0.22.0 to 0.23.0 (#3493).

1.40.0

New Features and Improvements

Documentation Changes

  • Use correct provider alias in AWS E2 provisioning guide (#3425).
  • Document pricing_tier of databricks_mws_workspaces resource (#3433).
  • Updated documentation for Delta Sharing resources; refactoring relevant resources & data sources (#3466).
  • Added description for new fields in databricks_lakehouse_monitor
  • Update about using Databricks-managed service principals in all clouds (#3482)

Exporter

  • Decrease a need for get-status calls for directories when listing workspace objects (#3470).
  • Don't emit permissions and grants if object is going to be ignored (#3417).
  • Don't generate references to ignored resources (#3418).
  • Generate resource data from the object status obtained during the listing operation (#3460).
  • Improve emitting of directories (#3462).
  • Skip listing of jobs deployed with DABs (#3445).

Internal Changes

  • Migrated databricks_cluster resource to Go SDK (#3376).
  • Used ClusterSpec in databricks_cluster resource (#3459).
  • Bump Go SDK to v0.38.0 (#3458).
  • Bump golang.org/x/mod from 0.16.0 to 0.17.0 (#3435).
  • Clean up integration tests and expose helper functions for identifying cloud (#3438).
  • Fix integration tests for databricks_entitlements (#3467).
  • Switch to use structs from Go SDK for databricks_recipient resource (#3400).
  • Make libraries in resource_job slice instead of set (#3398).
  • Fixed TestUcAccJobRunAsMutations
  • Ensure tests are only run in the selected environment (#3476)
  • Use fixture cluster for databricks_sql_permissions integration test (#3483)
  • Improve integration tests for databricks_model_serving (#3479)

1.39.0

New Features and Improvements

Documentation Changes

  • Clarify disable as user deletion (#3360).
  • Add MODEL to documentation about data_object_type in databricks_share resource (#3380).
  • Fix broken links in metastore-related docs (#3356).
  • Update documentation about databricks-managed service principals (#3350).
  • Fix code example for databricks_storage_credentials data source (#3327).
  • Fix storage credential doc guide (#3340).
  • Improve description of depends_on block (#3390).
  • Improve job library documentation (#3399).
  • [DOCS] change master to main & GH image raw links (#3358).
  • [DOC] Removing NOTEBOOK_FILE from databricks_share (#3353).
  • [MLOPS-717] Rename Amazon Bedrock (#3403).

Exporter

  • Don't generate resources with empty names (#3345).
  • Remove too aggressive omitting for volume_type attribute of databricks_volume resource (#3328).
  • Add an option for exporting secret values into terraform.tfvars (#3372).
  • Add support for generation of dependencies between resources (#3377).
  • Change listing behavior for metastore and storage credentials (#3367).
  • Modify grants for some UC objects to allow handling of other owners (#3357).

Internal Changes

  • Add integration test for ForEachTask (#3283).
  • Bump github.com/hashicorp/hcl/v2 from 2.20.0 to 2.20.1 (#3405).
  • Bump github.com/zclconf/go-cty from 1.14.3 to 1.14.4 (#3393).
  • Bump go-sdk version to 0.36.0 (#3391).
  • Bump go-sdk version to 0.37.0 (#3410).
  • Bump golang.org/x/mod from 0.15.0 to 0.16.0 (#3329).
  • Bump google.golang.org/protobuf from 1.32.0 to 1.33.0 (#3368).
  • Disable Lakehouse Monitoring integration tests on GCP (#3355).
  • Increase default timeout for databricks_vector_search_endpoint (#3332).
  • Make StructToSchema work for recursive data types (#3302).
  • Port databricks_secret_scope, databricks_secret and databricks_secret_acl to Go SDK (#3373).
  • SCIM patch to remove resources must not contain a value field (#3374).
  • Update Aliases function for ResourceProvider to handle recursive structures more gracefully (#3338).
  • Update Go SDK to latest + Fix backwards incompatible change (#3333).
  • Update databricks_model_serving resource to match latest API spec/SDK (#3093).
  • Update master to main in diff schema (#3331).
  • improve databricks_aws_crossaccount_policy data source (#3343).
  • lock metastore assignment test (#3386).
  • Added support for updating columns in databricks_sql_table resource (#3359).

1.38.0

New Features and Improvements

  • Use workspace-level default for enable_serverless_compute if not specified in databricks_sql_global_config (#3279). Previously, this resource would disable serverless unless you set the deprecated enable_serverless_compute attribute to true. From this release, this resource will not modify enable_serverless_compute, leaving it as its default value (enabled) if unspecified.
  • Fixed webhook_notifications diff suppression for databricks_job (#3309).
  • Perform case-insensitive error message checking when using force in databricks_user (#3295).
  • Use stable IDs for settings resources (#3314).

Documentation Changes

  • Improved documentation for INTERNAL and INTERNAL_AND_EXTERNAL delta sharing scope in terraform (#3287).
  • Added user guide for new behaviours with UC by default (#3318).
  • Expanded the note about private link in the troubleshooting guide (#3319).
  • Document storage_location attribute of the databricks_registered_model (#3249).
  • Various documentation improvements (#3288).
  • Changed accounts_id to required (#3244).

Exporter

  • Improved support for Unity Catalog (#3281).
  • Added generation of import blocks for easier importing existing resources in Terraform 1.5+ (#3293).
  • Added support for databricks_file resource (#3301).

Internal Changes

  • Set Computed field to true for common.Resource account_id attribute (#3310).
  • Removed unnecessary alias from cluster resource (#3298).
  • Bump Go SDK (#3296).
  • Bump github.com/hashicorp/hcl/v2 from 2.19.1 to 2.20.0 (#3317).
  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.32.0 to 2.33.0 (#3300).
  • Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 (#3323).
  • Bump github.com/zclconf/go-cty from 1.14.2 to 1.14.3 (#3322).

1.37.1

New Features and Improvements

  • Removed CustomizeDiff and Client Side Validation for databricks_grants (#3290).
  • Added terraform support for restrict ws admins setting (#3243).

Internal Changes

1.37.0

New Features and Improvements

Documentation Changes

  • Fix images and add note on setting multiple authorizations for workspace setup (#3259).
  • Remove data_object_type=TABLE only restriction in databricks_share (#3108).
  • Remove legacy guides (#3282).
  • Update for_each_task docs. (#3271).

Exporter

  • Support for some Unity Catalog resources (#3242).
  • Rework handling of listings and interactive prompting (#3241).
  • UC exporter databricks storage credential feature (#3219).

Internal Changes

  • Add CustomDiffFunc for health in sql_endpoint resources (#3227).
  • Bump github.com/databricks/databricks-sdk-go 0.33.0 (#3275).
  • Suppress diff on whitespace change for resources that often use HERE-docs (#3251).

1.36.3

New Features and Improvements

Documentation Changes

  • Added information on id and other exposed attributes where appropriate (#3237).
  • Fixed docs in metastore databricks_grants example (#3239).

Exporter

  • Detect & handle deleted workspace objects (notebooks/files/directories) when running in incremental mode (#3225).

Internal Changes

  • Make IterFields take in aliases (#3207).

1.36.2

New Features and Improvements

Exporter

  • Omitted git_provider only for well-known Git URLs (#3216).

Internal Changes

  • Bumped github.com/zclconf/go-cty from 1.14.1 to 1.14.2 (#3144).
  • Bumped golang.org/x/mod from 0.14.0 to 0.15.0 (#3229).

1.36.1

New Features and Improvements

  • Fixed create storage credentials with owner for account (#3184).

Documentation Changes

Internal Changes

  • Fixed test: TestUcAccResourceSqlTable_Managed (#3226).

1.36.0

New Features and Improvements

  • Added databricks_volumes as data source (#3150).
  • Fixed updating owners for UC resources (#3189).
  • Validated metastore id for databricks_grant and databricks_grants resources (#3159).
  • Fixed databricks_connection regression when creating without owner (#3186).
  • Allow using empty strings as job parameters (#3158).
  • Changed type of value field of JobsHealthRule to int64 (#3215).

Documentation Changes

  • Various documentation updates (#3198).
  • Fixed typo in docs (#3166).

Exporter

  • Timestamps are now added to log entries (#3146).
  • Add retries for Search, ReadContext and Import operations when importing the resource (#3202).
  • Performance improvements for big workspaces (#3167).
  • Fix generation of cluster policy resources (#3185).
  • Skip emitting of clusters that come from more cluster sources (#3161).

Internal Changes

  • Migrated cluster schema to use the go-sdk struct (#3076).
  • Updated actions/setup-go to v5 (#3154).
  • Changed default branch from master to main (#3174).
  • Added .codegen.json configuration (#3180).
  • Used common.Resource consistently throughout the provider (#3193).
  • Fixed unit test (#3201).
  • Added test code for job task order (#3183).
  • Added unit test for customizable_schema.go (#3192).
  • Extended customizable schema with AtLeastOneOf, ExactlyOneOf, RequiredWith (#3182).
  • Fixed notebook parameters in acceptance test (#3205).
  • Introduced Generic Settings Resource (#2997).
  • Suppress diff should apply to new fields added in the same chained call to CustomizableSchema (#3200).

1.35.0

New Features and Improvements:

Documentation Changes:

  • Reference OpenTofu support in TF Readme (#3141).

Exporter:

  • Better handling of ignored objects (#3055).
  • Don't omit source for notebook tasks from workspace when there is a Git configuration (#3120).
  • Improvements in jobs, DLT pipelines and policies (#3140).
  • Small refactoring (#3115).

Internal Changes:

  • Add Self-Assume Capability to the Identity-Bound policy for IAM roles provisioned by TF (#3124).
  • Bump TF provider to go 1.21 (#3129).
  • Bump github.com/cloudflare/circl from 1.3.6 to 1.3.7 (#3088).
  • Migrate databricks_mws_private_access_settings to Go SDK (#3135).
  • Refactor catalog/resource_grants to use Go SDK (#3090).
  • Remove Authenticate call from TF Configure (#3100).
  • Remove debug binary (#3107).
  • Support Go SDK Mocking Library in Terraform Unit Tests (#3121).
  • Update go SDK to 0.30.0 (#3145).
  • Use double quotes with filters in SCIM (#3136).

1.34.0

New Features and Improvements:

Documentation Changes:

  • Expand troubleshooting guide with documentation about private link problems (#3064).
  • Add a note about workspace or account-level provider usage (#3074).
  • Use tasks in the databricks_job examples (#3097).

Exporter:

Internal Changes:

  • Added an item to check if the ticket opener wants to do a bug fix (#3020).
  • Updated Go SDK to v0.29.0 (#3098).
  • Migrated SQL Warehouse to Go SDK (#3044).
  • Fixed TestClustersDataSourceErrorsOut test that run too long (#3073).
  • Fixed TestAccServicePrinicpalHomeDeleteNotDeleted (#3075).
  • Fixed diff schema on PRs (#3071).
  • Used diffSuppressor instead of makeEmptyBlockSuppressFunc (#3099).
  • Force creation of home folder if not created (#3052).

1.33.0

New Features and Improvements:

  • Added support for ownership changes for unity catalog resources (#3029).

Exporter:

  • Ignore workspace assets of deleted users and service principals (#2980).

1.32.0

New Features and Improvements:

  • Improved debugging of errors (#744)
  • Removed requirement to provide accountId in mws_credentials resource (#3028).
  • Validated metastore_id field in worspace UC resources except metastore resource (#3035), (#3039).
  • Wait for resource to be updated during grants resource modification (#3026).
  • Deprecated top level task attributes (#2993).

Documentation Changes:

Exporter:

  • Removed dead code that reflects old Jobs API 2.0 structures (#2981).

Internal Changes:

  • Bumped Go SDK to 0.28.1 (#3043).
  • Bumped github.com/hashicorp/terraform-plugin-sdk/v2 from 2.30.0 to 2.31.0 (#3033).
  • Added acceptance tests for databricks_grants (#3034).
  • Migrated library create to use go-sdk (#2998).

1.31.1

New Features and Improvements:

Bugfixes:

  • Bugfix for azure authentication (#732)

Exporter:

  • Exporter: fix suppression of empty webhook_notifications blocks in tasks (#2954).

Internal Changes:

1.31.0

New Features and Improvements:

Bugfixes:

  • Do not persist state from failed updates in databricks_workspace_conf (#2914).
  • Fix force_destroy behaviour for databricks_schema (#2907).
  • Fix databricks_workspace_conf nightly race (#2952).
  • Force sending the content field when creating empty databricks_workspace_file resources (#2931).
  • Generate correct url field for databricks_group resource on account level (#2911).
  • Make default_namespace_setting resource name singular (#2925).
  • Fix: resolved issue with run_as_role not being applied on databricks_sql_query resource (#2940).
  • Remove diff suppression for a job's run_as block (#2943).

Exporter:

Documentation changes:

  • Add documentation for condition_task in databricks_job resource (#2955).
  • Clarify that any_file also works for reading (#2937).
  • Documented use of libraries in the databricks_cluster_policy resource (#2934).
  • Small databricks_catalog_workspace_binding docs change (#2923).
  • [DOC] Refresh Unity Catalog guides with the latest updates (#2917).
  • [JOBS-13079] Add description to job settings (#2932).
  • doc fixes (#2987).
  • removing outdated note from doc (#2969).

Internal Changes:

  • Add integration test coverage for delta sharing parameters (#2991).
  • Bump Go SDK to v0.26.1 (#2968).
  • Fixed unit tests for the upcoming Go SDK release (#2956).
  • Migrate delete to use go-sdk for library resource (#2960).
  • Migrate jobs delete to go-sdk (#2948).
  • Moved contains function to common package (#2990).

1.30.0

New Features and Improvements:

Bugfixes:

Exporter:

  • Fixed emitting of group membership (#2868).
  • Don't omit display_name in databricks_user if it's not equal to user_name (#2851).
  • Follow recommendations of identity team on decreasing load on SCIM API (#2773).
  • Add case-insensitive match type for dependencies (#2854).
  • Disable generation of empty or default blocks for databricks_job and databricks_cluster (#2857).

Documentation Changes:

  • Update databricks_job documentation with queue block (#2872).
  • Use correct label in the issue template for documentation (#2850).
  • Modified github issue template to identify regressions faster (#2847).
  • Fixed documented default value for data_security_mode (#2866).
  • Add a note about setting entitlements with databricks_entitlements for account-level identities (#2910).
  • Add the missing region field to databricks_metastore resource block (#2859).

Internal Changes:

  • Skiped GCP for UC integration tests using system schemas or statement execution API (#2877).
  • Added integration test for SQL warehouse permissions in databricks_permissions (#2845).
  • Bumped github.com/databricks/databricks-sdk-go from 0.24.0 to 0.25.0 (#2909).
  • Enabled and fix tests (#2901).
  • Fix Acceptance Tests (#2916).

1.29.0

New Features and Improvements:

Bugfixes:

Exporter:

  • Exporter: Correct treatment of disallowed characters in notebooks/directory names (#2831).
  • Exporter: Added support for exporting databricks_access_control_rule_set on account level (#2699).
  • Exporter: Adjusted generation & normalization of job names (2836).
  • Exporter: Clarified behaviour of services not marked with listing (#2785).
  • Exporter: Fixed generation of names for databricks_library resources (#2832).
  • Clarified experimental status of the exporter (#2816).
  • Exporter: Ensured that name of databricks_repo is unique (#2842).

Documentation changes:

  • Added account_id parameter to provider blocks in the guides (#2817).
  • Changed CREATE_TABLE to CREATE_EXTERNAL_TABLE (#2812).
  • Corrected max number of pinned clusters in the databricks_cluster docs (#2825).
  • Put empty_result_state under options in the databricks_sql_alert resource (#2834).
  • Corrected grammar & rewording for better readability (part 1) (#2781).

Internal changes:

  • Added additional integration tests for condition_task tasks in databricks_job (#2830).
  • Bump github.com/databricks/databricks-sdk-go from 0.23.0 to 0.24.0 (#2835).
  • Bump github.com/hashicorp/hcl/v2 from 2.18.1 to 2.19.1 (#2819).
  • Enabled Cluster Integration test (#2815).
  • Changed Jobs List operations to use token-based pagination (#2810).

1.28.1

  • Fixed read method for databricks_storage_credential resource (#2804).

1.28.0

Exporter:

Documentation:

Other Changes:

  • Bumped databricks-sdk-go dependency to 0.21.0 (#2738).
  • Bumped github.com/databricks/databricks-sdk-go from 0.21.0 to 0.22.0 (#2761).
  • Bumped github.com/databricks/databricks-sdk-go from 0.22.0 to 0.23.0 (#2794).
  • Bumped github.com/hashicorp/hcl/v2 from 2.18.0 to 2.18.1 (#2776).
  • Bumped github.com/zclconf/go-cty from 1.14.0 to 1.14.1 (#2777).
  • Used terraform-field-dev as code owner instead of field-dev-ecosystem (#2718).
  • GitHub Actions workflow to compute provider schema diff (#2740).

1.27.0

Exporter:

  • Exporter: parallel listing of workspace objects (#2691).
  • Exporter: removed modifiedAt != 0 check in the incremental mode (#2736).

Other Changes:

  • Bumped github.com/databricks/databricks-sdk-go from 0.19.2 to 0.20.0 (#2714).

1.26.0

Documentation:

Exporter:

1.25.1

  • Fixed the issue with cannot reset nil reader by bumping Go SDK to 0.19.2 (#2684).
  • Changed validation for max_concurrent_runs in databricks_job to allow 0 value (#2682).

1.25.0

  • Added IS_OWNER permission for SQL Warehouse (#2600).
  • Added managed_identity_id to databricks_storage_credential to support user-assigned managed identities (#2536).
  • Added options field to UC catalog resource to support foreign catalog creation (#2616).
  • Detected run_as drift in job resource (#2626).
  • Removed callback field from acceptance test framework (#2649).
  • Fixed force_new attributes for databricks_schema and databricks_volume (#2635).
  • Fixed do not attempt to delete default schema for foriegn catalogs (#2622).

Documentation:

  • Updated documentation for marketplace admin role (#2638).
  • Updated documentation for share and share recipient (#2641).
  • Updated documentation for group, service principal and user(#2644).

Other Changes:

  • Bumped github.com/databricks/databricks-sdk-go from 0.17.0 to 0.19.1 (#2660).
  • Bumped github.com/hashicorp/hcl/v2 from 2.17.0 to 2.18.0 (#2636).
  • Added doc strings for ResourceFixtures (#2633).

1.24.1

  • Fixed verification of workspace reachability by using scim/me which is always available (#2618).

1.24.0

Documentation:

  • Added troubleshooting guide for grants/permissions config drifts (#2576).
  • Updated doc for databricks_model_serving to include new fields (#2579).
  • Added instructions for group rule set management (#2561).
  • Added missing documentation for CMK support on GCP (#2604).

Exporter:

Other Changes:

1.23.0

Exporter:

Documentation:

  • Documented missing environment variables for authentication (#2541).
  • Documented run_if field and suppress diff when field is not present (#2435).
  • Updated diagram with Databricks resources (#2526).
  • Updated metastore.md (#2547).

Other Changes:

1.22.0

Exporter:

Documentation updates:

  • Added documentation notes about legacy cluster type & data access (#2437).
  • Added one more item to the troubleshooting guide (#2477).
  • Added clarification that databricks_schema and databricks_sql_table should be imported by their full name, not just by name (#2491).
  • Added more common issues for troubleshooting (#2486).
  • Added additional documentation (#2516).
  • Linked model serving docs to top level README (#2474).

Other notes:

  • Added code owners for terraform-provider-databricks (#2498).
  • Added support for new Delve binary name format (#2497).
  • Configured merge queue for the provider (#2533).
  • Removed unused dlvLoadConfig configuration from settings.json (#2499).

Updated dependency versions:

  • Bump github.com/databricks/databricks-sdk-go from 0.13.0 to 0.14.1 (#2523).
  • Bump golang.org/x/mod from 0.11.0 to 0.12.0 (#2462).

1.21.0

  • Added condition_task to the databricks_job resource (private preview) (#2459).
  • Added AccountData, AccountClient and define generic databricks data utilites for defining workspace and account-level data sources (#2429).
  • Added documentation link to existing Databricks Terraform modules (#2439).
  • Added experimental compute field to databricks_job resource (#2401).
  • Added import example to doc for databricks_group_member resource (#2453).
  • Added support for subscriptions in dashboards & alert SQL tasks in databricks_job (#2447).
  • Fixed model serving integration test (#2460, #2461).
  • Fixed databricks_job resource file arrival trigger parameter name (#2438).
  • Fixed catalog_workspace_binding_test (#2463, #2451).

1.20.0

1.19.0

1.18.0

Updated dependency versions:

  • Bump github.com/hashicorp/hcl/v2 from 2.16.2 to 2.17.0 (#2359).
  • Bump github.com/stretchr/testify from 1.8.3 to 1.8.4 (#2354).
  • Bump github.com/zclconf/go-cty from 1.13.1 to 1.13.2 (#2329).

1.17.0

Updated dependency versions:

  • Bump github.com/stretchr/testify from 1.8.2 to 1.8.3 (#2317).
  • Bump github.com/databricks/databricks-sdk-go from v0.8.1 to v0.9.0 (#2327).

1.16.1

1.16.0

1.15.0

1.14.3

1.14.2

Updated dependency versions:

  • Bump github.com/databricks/databricks-sdk-go from 0.6.0 to 0.7.0 (#2195).
  • Bump golang.org/x/mod from 0.9.0 to 0.10.0 (#2192).

1.14.1

1.14.0

Updated dependency versions:

  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.25.0 to 2.26.1 (#2140).
  • Bump github.com/zclconf/go-cty from 1.13.0 to 1.13.1 (#2124).

1.13.0

Updated dependency versions:

  • Bump github.com/databricks/databricks-sdk-go from 0.4.0 to 0.5.0 (#2096, #2112).
  • Bump github.com/hashicorp/hcl/v2 from 2.16.1 to 2.16.2 (#2097).

1.12.0

Updated dependency versions:

  • Bump github.com/databricks/databricks-sdk-go from v0.3.3 to v0.4.0 (#2086).
  • Bump golang.org/x/mod from 0.8.0 to 0.9.0 (#2076).

1.11.1

1.11.0

Updated dependency versions:

  • Bump github.com/stretchr/testify from 1.8.1 to 1.8.2 (#2049).

1.10.1

Updated dependency versions:

  • Bump github.com/databricks/databricks-sdk-go from v0.3.2 to v0.3.3 (#2044).
  • Bump github.com/zclconf/go-cty from 1.12.1 to 1.13.0 (#2043).

1.10.0

Updated dependency versions:

  • Removed unused github.com/Azure/go-autorest/autorest and github.com/mitchellh/go-homedir (#2026).
  • Bump Go from 1.18 to 1.19 (#2029).
  • Bump CodeQL to v2 (#2027).
  • Bump github.com/golang-jwt/jwt/v4 from 4.4.3 to 4.5.0 (#2030).
  • Bump github.com/hashicorp/hcl/v2 from 2.15.0 to 2.16.1 (#1971, #1997).
  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.24.1 to 2.25.0 (#2031).
  • Bump golang.org/x/mod from 0.7.0 to 0.8.0 (#1996).
  • Bump golang.org/x/net from 0.6.0 to 0.7.0 (#2028).
  • Bump google.golang.org/api from 0.108.0 to 0.110.0 (#1984, #1995).

1.9.2

Documentation:

1.9.1

Updated dependency versions:

  • Bump github.com/Azure/go-autorest/autorest/adal from 0.9.21 to 0.9.22 (#1953).
  • Bump github.com/Azure/go-autorest/autorest/azure/auth (#1952).
  • Bump google.golang.org/api from 0.106.0 to 0.107.0 (#1929).
  • Bump google.golang.org/api from 0.107.0 to 0.108.0 (#1954).

1.9.0

Updated dependency versions:

  • Bump github.com/hashicorp/go-retryablehttp from 0.7.1 to 0.7.2 (#1913).
  • Bump google.golang.org/api from 0.105.0 to 0.106.0 (#1914).

1.8.0

Experimental Resource Exporter

  • Added command-line flag to export all users and service principals (#1893).
  • Added emit for secret scopes from Spark Conf and Env Vars (#1897).
  • Fixed issue with exporting of resources with simple name (#1891).
  • Generate references that are matching only to a prefix (#1890).
  • Updated list of supported services and associated resources (#1894).
  • Use databricks_group_role instead of deprecated databricks_group_instance_profile (#1905).
  • Use dbfs_path as reference for init scripts in clusters (#1892).
  • Logging improvements (#1895).

Updated dependency versions:

  • Bump google.golang.org/api from 0.104.0 to 0.105.0 (#1863).

1.7.0

Updated dependency versions:

  • Bump github.com/golang-jwt/jwt/v4 from 4.4.2 to 4.4.3 (#1817).
  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.24.0 to 2.24.1 (#1761).
  • Bump golang.org/x/mod from 0.6.0 to 0.7.0 (#1760).
  • Bump google.golang.org/api from 0.102.0 to 0.104.0 (#1758, #1839).

1.6.5

1.6.4

  • Reverted PRO as default warehouse_type in databricks_sql_endpoint (#1741).
  • Moved testing of clusters, instance pools, and SQL Warehouses to internally configured test environment variables (#1739).

Updated dependency versions:

  • Bump google.golang.org/api from 0.101.0 to 0.102.0 (#1736).

1.6.3

1.6.2

Updated dependency versions:

  • Bump github.com/stretchr/testify from 1.8.0 to 1.8.1 (#1689).
  • Bump github.com/zclconf/go-cty from 1.11.1 to 1.12.0 (#1714).
  • Bump golang.org/x/mod from 0.5.1 to 0.6.0 (#1690).
  • Bump google.golang.org/api from 0.99.0 to 0.101.0 (#1713, #1691).

1.6.1

Updated dependency versions:

  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.23.0 to 2.24.0 (#1671).
  • Bump github.com/zclconf/go-cty from 1.11.0 to 1.11.1 (#1672).
  • Bump google.golang.org/api from 0.98.0 to 0.99.0 (#1673).

1.6.0

1.5.0

Updated dependency versions:

  • Bump google.golang.org/api from 0.97.0 to 0.98.0 (#1652).

1.4.0

Documentation improvements:

Updated dependency versions:

  • Bump github.com/hashicorp/hcl/v2 from 2.14.0 to 2.14.1 (#1634).
  • Bump google.golang.org/api from 0.96.0 to 0.97.0 (#1633).

1.3.1

Documentation improvements:

Updated dependency versions:

  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.21.0 to 2.23.0 (#1597, #1611).
  • Bump google.golang.org/api from 0.94.0 to 0.96.0 (#1598, #1612).

1.3.0

Documentation improvements:

  • Added sql_task configuration block in databricks_job resource (#1589).
  • Added supported languages in Unity Shared clusters (#1587).
  • Removed "Public Preview" marker from Unity Catalog resources docs (#1570).
  • Fixed instance pool docs (#1581).

Updated dependency versions:

  • Bump github.com/hashicorp/hcl/v2 from 2.13.0 to 2.14.0 (#1582).
  • Bump google.golang.org/api from 0.93.0 to 0.94.0 (#1569).

1.2.1

  • Use resize API to scale databricks_cluster while it's running (#1541).
  • Updated to latest Unity Catalog privileges model (#1556).
  • Added cluster policy support for databricks_pipeline (#1554).
  • Fixed databricks_node_type by skipping nodes that aren't available in subscription/region (#1534).
  • Fixed sending of active flag in the databricks_user (#1536).
  • Fixed Azure CLI tests on Go 1.19 (#1538).
  • Various doc updates (#1553, #1552, #1544, #1543).

Updated dependency versions:

  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.20.0 to 2.21.0 (#1540).
  • Bump github.com/zclconf/go-cty from 1.10.0 to 1.11.0 (#1558).
  • Bump google.golang.org/api from 0.90.0 to 0.93.0 (#1525, #1545).
  • Bump gopkg.in/ini.v1 from 1.66.6 to 1.67.0 (#1526).

1.2.0

Updated dependency versions:

  • Bump github.com/Azure/go-autorest/autorest from 0.11.27 to 0.11.28 (#1513).
  • Bump github.com/Azure/go-autorest/autorest/adal from 0.9.20 to 0.9.21 (#1512).
  • Bump github.com/Azure/go-autorest/autorest/azure/cli from 0.4.5 to 0.4.6 (#1515).
  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.19.0 to 2.20.0 (#1516).
  • Bump google.golang.org/api from 0.87.0 to 0.90.0 (#1499, #1514).

1.1.0

  • Added databricks_sql_warehouses data resource to list all warehouses in a workspace with a name filter (#1460).
  • Added databricks_sql_warehouse data resource to list SQL warehouse attributes based on single id (#1460).
  • Added databricks_cluster data resource to list cluster attributes based on single id (#1460).
  • Added Azure Managed Identity documentation examples (#1471).
  • Added more attributes to databricks_cluster (#1459).
  • Added more attributes to databricks_instance_pool (#1463).
  • Added feature request GitHub issue template (#1482).
  • Added tf:optional tag to simplify the code (#1395).
  • Fixed databricks_pipeline incorrect generation of cluster blocks (#1416).
  • Fixed databricks_table update for column block (#1468).
  • Fixed reads for any_file and anonymous_function in databricks_sql_permissions (#1477).
  • Tuned integration tests for databricks_mws_* (#1483).
  • Removed integration tests for databricks_azure_adls_gen1_mount (#1461).

Updated dependency versions:

  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.18.0 to 2.19.0 (#1475).
  • Bump google.golang.org/api from 0.86.0 to 0.87.0 (#1476).

1.0.2

  • Added metastore to databricks_grants (#1447).
  • Added update support for databricks_mws_log_delivery (#1439).
  • Fixed name generation for exported databricks_notebook (#1435).
  • Fixed databricks_job recreation on changed docker_image URL (#1437).
  • Fixed replace databricks_mws_networks on vpc_endpoints change (#1453).
  • Fixed diff suppress in databricks_external_location on skip_validation (#1421).
  • Switched to SQL warehouses API for databricks_sql_endpoint (#1414).
  • Improve docs for databricks_permissions and databricks_obo_token (#1462).
  • Improve docs for abfs block in databricks_mount (#1446).
  • Improve dev docs for tf:"suppress_diff", tf:"force_new" and tf:"sensitive" tags (#1465).
  • Moved couple of acceptance tests out of preview (#1433).
  • Migrated to Go 1.18: interface{} -> any (#1466).

Updated dependency versions:

  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.17.0 to 2.18.0 (#1450).

1.0.1

  • Added better handling of missing objects during import (#1417).
  • Fix problem in the databricks_node_type data source (#1400).
  • Fix update for databricks_storage_credential resource (#1403).
  • Improve databricks_job resource (#1428).
  • Upgraded UC API from 2.0 to 2.1 (#1418).
  • Sanitized host provider field prior to auth flow (#1385).
  • Use delta_sharing_scope instead of delta_sharing_enabled (#1398).
  • Added instructions to create missing .terraform.lock.hcl (#1397).
  • Clarified databricks_permissions doc for service principals (#1426).
  • Clarified databricks_cluster autotermination_minutes default value (#1419).
  • Fully moved codebase from databrickslabs to databricks namespace (#1429).
  • Various integration testing improvements (#1425, #1427, #1420).

Updated dependency versions:

  • Bump Go from 1.16.x to 1.18.x (#1413).
  • Bump github.com/golang-jwt/jwt/v4 from 4.4.1 to 4.4.2 (#1407).
  • Bump github.com/hashicorp/hcl/v2 from 2.12.0 to 2.13.0 (#1406).
  • Bump github.com/stretchr/testify from 1.7.3 to 1.8.0 (#1387, #1408, #1422).
  • Bump google.golang.org/api from 0.84.0 to 0.86.0 (#1386, #1423).

1.0.0

To make Databricks Terraform Provider generally available, we've moved it from https://github.com/databrickslabs to https://github.com/databricks. We've worked closely with the Terraform Registry team at Hashicorp to ensure a smooth migration. Existing terraform deployments continue to work as expected without any action from your side. We ask you to replace databrickslabs/databricks with databricks/databricks in all your .tf files.

You should have .terraform.lock.hcl file in your state directory that is checked into source control. terraform init will give you the following warning.

Warning: Additional provider information from registry 

The remote registry returned warnings for registry.terraform.io/databrickslabs/databricks:
- For users on Terraform 0.13 or greater, this provider has moved to databricks/databricks. Please update your source in required_providers.

After you replace databrickslabs/databricks with databricks/databricks in the required_providers block, the warning will disappear. Do a global "search and replace" in *.tf files. Alternatively you can run python3 -c "$(curl -Ls https://dbricks.co/updtfns)" from the command-line, that would do all the boring work for you.

If you didn't check-in .terraform.lock.hcl to the source code version control, you may you may see Failed to install provider error. Please follow the simple steps described in the troubleshooting guide.

0.6.2

  • Add a warning in databricks_permissions token usage docs (#1380).

Updated dependency versions:

  • Bump google.golang.org/api from 0.83.0 to 0.84.0
  • Bump github.com/stretchr/testify from 1.7.2 to 1.7.3

0.6.1

  • Added databricks_service_principal and databricks_service_principals data resources (#1370).
  • Updated databricks_pipeline resource to match latest APIs (#1368).
  • Made gcp_managed_network_config optional in databricks_mws_workspaces (#1365).
  • Enforced consistent naming for resource files (#1366, #1369).
  • Updated resources diagram (#1373).

Updated dependency versions:

  • Bump google.golang.org/api from 0.82.0 to 0.83.0

0.6.0

  • Added delta_sharing_* support to databricks_metastore (#1334).
  • Added databricks_git_credentials pat discovery from common environment variables (#1353).
  • Added databricks_permissions for databricks_pipeline (#1361).
  • Added network_id to network block in databricks_mws_workspaces for GCP (#1360).
  • Added azure_managed_identity block to databricks_storage_credential and databricks_metastore_data_access resources (#1354).
  • Update docs regarding importing of databricks_sql_* resources (#1349).
  • Apply ownership for UC objects during creation (#1338).
  • Re-create purged cluster for databricks_mount for Google Storage (#1333).
  • Various documentation fixes (#1350).

Updated dependency versions:

  • Bump google.golang.org/api from 0.80.0 to 0.81.0
  • Bump gopkg.in/ini.v1 from 1.66.4 to 1.66.6
  • Bump google.golang.org/api from 0.81.0 to 0.82.0
  • Bump github.com/stretchr/testify from 1.7.1 to 1.7.2
  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.16.0 to 2.17.0

0.5.9

  • Added warning section for debug mode (#1325).
  • Added ability to specify tags for databricks_job (#1337).
  • Upgraded AWS provider for AWS guides. Added examples for account-level identities (#1332).
  • Updated docs to use application_id as privilege for databricks_service_principal (#1336).
  • Added databricks_service_principal_role resource (#1340).
  • Fixed itegration testing image (#1342, #1343).
  • Added skip_validation for databricks_external_location (#1330).
  • Added alert_on_last_attempt to databricks_job (#1341).
  • Skip make test on doc-only changes (#1339).
  • Improve common package test coverage (#1344).
  • Re-create purged cluster for databricks_mount for AWS S3 (#1345).

Updated dependency versions:

  • Bump google.golang.org/api from 0.79.0 to 0.80.0
  • Bump github.com/Azure/go-autorest/autorest/adal from 0.9.19 to 0.9.20

0.5.8

  • Update aws_iam_policy_document in databricks_mws_customer_managed_keys docs to restrict KMS policy to caller AWS account (#1309).
  • Added gcs destination to init_scripts in databricks_cluster (#1308).
  • Clarify optionality of databricks_mws_workspaces.deployment_name in docs and examples (#1315).
  • Update databricks_mws_log_delivery docs (#1320).
  • Fix updating databricks_service_principal on Azure (#1322).
  • Added tf:suppress_diff on artifact_location for databricks_mlflow_experiment (#1323).

Updated dependency versions:

  • Bump github.com/Azure/go-autorest/autorest/adal from 0.9.18 to 0.9.19
  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.14.0 to 2.16.0
  • Bump google.golang.org/api from 0.77.0 to 0.79.0

0.5.7

  • Added external_id and force attributes to databricks_service_principal resource (#1293).
  • Added documentation for databicks_git_credential resource (#1295).
  • Added documentation for git_source in databricks_job (#1297).
  • Fix job_cluster.num_workers in databricks_job (#1284).
  • Various documentation improvements (#1292), (#1296), (#1298).

Updated dependency versions:

  • Bump google.golang.org/api from 0.75.0 to 0.77.0
  • Removed github.com/pkg/errors dependency

0.5.6

  • Added databricks_views data resource, making databricks_tables return only managed or external tables in Unity Catalog (#1274).
  • Added default timeout of 20m to databricks_mount (#1280).
  • Made common.DataResource deterministic (#1279).
  • Fixed exporting text-only widgets (#1278).
  • Updated devcontainer to support ARM (#1256).
  • Various documentation fixes (#1285, #1282, #1281, #1276).

Updated dependency versions:

  • Bump github.com/hashicorp/hcl/v2 from 2.11.1 to 2.12.0
  • Bump github.com/Azure/go-autorest/autorest from 0.11.26 to 0.11.27

0.5.5

  • Added configuration generators for databricks_sql_* resources in experimental Resource Exporter (#1199).
  • Added google_credentials provider agument that has the same semantics as credentials argument in official google provider (#1214).
  • Fixed databricks_grants on UC external location empty list error (#1202).
  • Fixed errors in databricks_permissions resource for auto-purged databricks_cluster (#1252).
  • Various documentation fixes (#1231, #1239, #1254, #1240).

Updated dependency versions:

  • Bump google.golang.org/api from 0.71.0 to 0.75.0
  • Bump github.com/golang-jwt/jwt/v4 from 4.3.0 to 4.4.1
  • Bump github.com/stretchr/testify from 1.7.0 to 1.7.1
  • Bump github.com/hashicorp/go-retryablehttp from 0.7.0 to 0.7.1
  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.11.0 to 2.14.0
  • Bump github.com/Azure/go-autorest/autorest from 0.11.24 to 0.11.26

0.5.4

  • Completely removed custom client-side validation in databricks_service_principal (#1193).
  • Added export functionality for Databricks SQL objects - endpoints, queries, dashboards, widgets, visualizations (#1199).

Updated dependency versions:

  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.10.1 to 2.11.0

0.5.3

  • Failures in exporter resource listing no longer halt the entire command run (#1166).
  • Removed client-side validation in databricks_service_principal for application_id, that may not always be available in the planning stage (#1165).
  • Use correct HTTP verb for modifying databricks_permissions on databricks_sql_endpoint entities. Authorized user, assumingly part of admins group, is no longer sending CAN_MANAGE permission in the HTTP PUT request (#1163).
  • Added diff suppression for min_num_clusters field in databricks_sql_endpoint (#1172).
  • Added special case for handling Cannot access cluster that was terminated or unpinned more than 30 days ago error in databricks_cluster as an indication of resource removed on the platform side (#1177).
  • Fixed updating of databricks_table resources (#1175).
  • Fixed configuration drift in databricks_grant by reading existing permissions and removing them in subsequent update calls (#1164).

Updated dependency versions:

  • Bump google.golang.org/api from 0.70.0 to 0.71.0

0.5.2

  • Added databricks_catalogs, databricks_schemas, and databricks_tables data resources (#1155).
  • Fixed databricks_metastore_assignment configuration drift by properly deleting metastore assignment and detecting manual changes from Account console. This also means that de-assigned metastore from a workspace would mark it as remotely removed. Manual assignment of different metastore would also trigger resource updates (#1146).
  • Fixed databricks_table creation in managed mode (#1151).
  • Fixed databricks_sql_endpoint timeout propagation (#1142).
  • Multiple documentation fixes.

Updated dependency versions:

  • Bump google.golang.org/api from 0.69.0 to 0.70.0

0.5.1

  • Added an extended documentation from provisioning AWS PrivateLink workspace (#1084).
  • Added databricks_jobs data resource to get a map of all job names and their ids (#1138).

Updated dependency versions:

  • Bump google.golang.org/api from 0.68.0 to 0.69.0

0.5.0

  • Added workspace_url attribute to the databricks_current_user data source (#1107).
  • Fixed issue at databricks_mount where new cluster was created for S3 mount even when cluster_id was specified (#1064).
  • Allow to disable auto-termination for Databricks SQL endpoints (#900).
  • Added new gcp_attributes to databricks_cluster and databricks_instance_pool (#1126).
  • Added exporter functionality for databricks_ip_access_list and databricks_workspace_conf (#1125).
  • Added graviton selector for databricks_node_type and databricks_spark_version data sources (#1127).
  • Added interactive mode to resource exporter (#1010).
  • Added preview support for git_source in databricks_job (#1090).
  • Multiple other fixes and documentation improvements.

Updated dependency versions:

  • Bump github.com/golang-jwt/jwt/v4 from 4.2.0 to 4.3.0
  • Bump google.golang.org/api from 0.67.0 to 0.68.0
  • Bump gopkg.in/ini.v1 from 1.66.3 to 1.66.4

0.4.9

  • Prevent creation of databricks_group with users and admins reserved names (#1089).
  • Added support for shared clusters in multi-task databricks_job (#1082).
  • Added diff suppression for external_id in databricks_group (#1099).
  • Added diff suppression for external_id in databricks_user (#1097).
  • Added users, service_principals, and child_groups exported properties to databricks_group data resource (#1085).
  • Added various documentation improvements.

Deprecations

  • databricks_group.members is deprecated in favor of users, service_principals, and child_groups exported properties. Please do slight modifications of your configuration.

Updated dependency versions:

  • Bump google.golang.org/api from 0.66.0 to 0.67.0

0.4.8

  • Added support for tf:suppress_diff on primitive types (#984).
  • Fixed issue arises when destroying databricks_sql_global_config with instance profile set (#1076).
  • Added setting of SQL configuration parameters in databricks_sql_global_config (#1080).
  • Added support for release channels in databricks_sql_endpoint configuration (#1078).
  • Added documentation for databricks_mlflow_webhook resource (#1086).

Updated dependency versions:

  • Bump google.golang.org/api from 0.65.0 to 0.66.0

0.4.7

  • Added optional force argument to databricks_group resource to ignore cannot create group: Group with name X already exists. errors and implicitly import the specific group into Terraform state, enforcing entitlements defined in the instance of resource (#1066).
  • Added support to configure permissions for all MLflow models (#1044).
  • Fixed databricks_service_principal display_name update (#1065).
  • Added documentation for Unity Catalog resources.

Updated dependency versions:

  • Bump gopkg.in/ini.v1 from 1.66.2 to 1.66.3

0.4.6

  • Clarified error messages around azure_workspace_resource_id provider configuration (#1049).
  • Added optional force argument to databricks_user resource to ignore cannot create user: User with username X already exists errors and implicitly import the specific user into Terraform state, enforcing entitlements defined in the instance of resource (#1048).
  • Added databricks_user_role resource, that can assign roles on Databricks Account or databricks_instance_profile for data access (#1047).

Deprecations

  • databricks_user_instance_profile is deprecated in favor of databricks_user_role. Please do slight modifications of your configuration.

Updated dependency versions:

  • Bump github.com/Azure/go-autorest/autorest/azure/auth from 0.5.10 to 0.5.11
  • Bump github.com/Azure/go-autorest/autorest/azure/cli from 0.4.3 to 0.4.5
  • Bump github.com/Azure/go-autorest/autorest from 0.11.23 to 0.11.24

0.4.5

  • Cross-linked resource documentation (#1027).
  • Added Azure example for sql_global_config (#1028).

Updated dependency versions:

  • Bump google.golang.org/api from 0.63.0 to 0.65.0

0.4.4

  • Added support for running provider in a debug mode from Visual Studio Code through Debug Provider run configuration in order to troubleshoot more complicated issues.
  • Allowed managing of libraries on databricks_cluster outside of Terraform state for resources without any library configuration blocks. This should simplify PaaS-like CI/CD workflows (#1024).
  • Added experimental resources.

Behavior changes

0.4.3

  • Added support for databricks_permissions for databricks_mlflow_experiment and databricks_mlflow_model (#1013).
  • Added Using XXX auth explanation to HTTP 403 errors, which should help troubleshooting misconfigured authentication or provider aliasing. Example error message now looks like: cannot create group: /2.0/preview/scim/v2/Groups is only accessible by admins. Using databricks-cli auth: host=https://XXX.cloud.databricks.com/, token=***REDACTED***, profile=demo. All sensitive configuration parameters (token, password, and azure_client_secret) are redacted and replaced with ***REDACTED*** (#821).
  • Improved documentation with regards to public subnets in AWS quick start (#1005).
  • Added databricks_mount code genration for exporter tooling (#1006).
  • Increase dependency check frequency (#1007).
  • Added experimental resources.

0.4.2

  • Added optional auth_type provider conf to enforce specific auth type to be used in very rare cases, where a single Terraform state manages Databricks workspaces on more than one cloud and More than one authorization method configured error is a false positive. Valid values are pat, basic, azure-client-secret, azure-msi, azure-cli, and databricks-cli (#1000).
  • Added DBC format support for databricks_notebook (#989).
  • Fixed creating new databricks_mws_workspaces with token {} block (#994).
  • Added automated documentation formatting with make fmt-docs, so that all HCL examples look consistent (#999).
  • Increased codebase unit test coverage to 91% to improve stability (#996, #992, #991, #990).

Updated dependency versions:

  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.10.0 to 2.10.1

0.4.1

  • Added databricks_library resource to install library on databricks_cluster (#904).
  • Added databricks_clusters data resource to list all clusters in the workspace, which might be used to install databricks_library on all clusters (#955).
  • Fixed refresh of library blocks on a stopped databricks_cluster (#952).
  • Whenever a library fails to get installed on a running databricks_cluster, we now automatically remove this library, so that the clean state of managed libraries is properly maintained. Without this fix users had to manually go to Clusters UI and remove library from a cluster, where it failed to install. Libraries add up to CREATE and UPDATE timeouts of databricks_cluster resource. (#599).
  • Added token block to databricks_mws_workspaces to avoid unnecessary provider aliasing (#957).
  • Fixed disabling databricks_global_init_script (#958).
  • Fixed configuration drift issues with aws_attributes, azure_attributes, gcp_attributes, and email_notifications configuration blocks in databricks_cluster, databricks_job, and databricks_instance_pool resources (#981).
  • Improved Databricks CLI auth by eagerly resolving host, username, password, and token from the specified profile. Added explicit logging of auth parameters in debug logs (#965).
  • TLS timeouts, which may occur during Azure MSI auth, are no longer failing API requests and retried within a normal policy (#966).
  • debug_headers provider conf is also logging the Host header to help troubleshooting auth issues (#964).
  • Added new experimental resources and increased test coverage.

Updated dependency versions:

  • Bump github.com/golang-jwt/jwt/v4 from 4.1.0 to 4.2.0
  • Bump google.golang.org/api from 0.60.0 to 0.63.0
  • Bump github.com/Azure/go-autorest/autorest from 0.11.22 to 0.11.23
  • Bump github.com/Azure/go-autorest/autorest/azure/auth from 0.5.9 to 0.5.10
  • Bump gopkg.in/ini.v1 from 1.66.0 to 1.66.2
  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.9.0 to 2.10.0

0.4.0

  • Added databricks_mlflow_model and databricks_mlflow_experiment resources (#931)
  • Added support for repo_path to databricks_permissions resource (#875).
  • Added external_id to databricks_user and databricks_group (#927).
  • Fixed databricks_repo creation corner cases on MS Windows OS (#911).
  • Fixed configuration drift for databricks_cluster.aws_attributes.zone_id with auto, which resulted in unwanted cluster restarts (#937).
  • Added new experimental resources, increased test coverage, and automated integration testing infrastructure.
  • Multiple documentation improvements and new guides.

Behavior changes

  • Renamed allow_sql_analytics_access to databricks_sql_access in databricks_user, databricks_group, and databricks_service_principal resources.
  • Removed deprecated azure_use_pat_for_spn, azure_use_pat_for_cli, azure_pat_token_duration_seconds provider attributes.
  • Removed deprecated azure_workspace_name, azure_resource_group, azure_subscription_id in favor of just using azure_workspace_resource_id.
  • Renamed DATABRICKS_AZURE_WORKSPACE_RESOURCE_ID environment variable to DATABRICKS_AZURE_RESOURCE_ID.
  • DATABRICKS_AZURE_CLIENT_SECRET environment variable is no longer having any effect in favor of just using ARM_CLIENT_SECRET.
  • DATABRICKS_AZURE_CLIENT_ID environment variable is no longer having any effect in favor of just using ARM_CLIENT_ID.
  • DATABRICKS_AZURE_TENANT_ID environment variable is no longer having any effect in favor of just using ARM_TENANT_ID.

Updated dependency versions:

  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.7.1 to 2.9.0
  • Bump github.com/Azure/go-autorest/autorest/adal from 0.9.16 to 0.9.17
  • Bump github.com/golang-jwt/jwt/v4 from 4.0.0 to 4.1.0
  • Bump github.com/zclconf/go-cty from 1.9.1 to 1.10.0
  • Bump github.com/Azure/go-autorest/autorest from 0.11.21 to 0.11.22

0.3.11

  • Added databricks_sql_global_config resource to provide global configuration for SQL Endpoints (#855)
  • Added databricks_mount resource to mount arbitrary cloud storage (#497)
  • Improved implementation of databricks_repo by creating the parent folder structure (#895)
  • Fixed databricks_job error related to randomized job IDs (#901)
  • Replace databricks_group on name change (#890)
  • Names of scopes in databricks_secret_scope can have / characters in them (#892)

Deprecations

  • databricks_aws_s3_mount, databricks_azure_adls_gen1_mount, databricks_azure_adls_gen2_mount, and databricks_azure_blob_mount are deprecated in favor of databricks_mount.

Updated dependency versions:

  • Bump google.golang.org/api from 0.59.0 to 0.60.0

0.3.10

  • Added private_access_level and allowed_vpc_endpoint_ids to databricks_mws_private_access_settings resource, which is also now updatable (#867).
  • Fixed missing diff skip for skip_validation in databricks_instance_profile (#860).
  • Added support for pipeline_task (871) and python_wheel_task (#872) to databricks_job.
  • Improved enterprise HTTPS proxy support for creating workspaces in PrivateLink environments (#882).
  • Added hostname attribute to odbc_params in databricks_sql_endpoint (#868).
  • Improved documentation (#858, #870).

Updated dependency versions:

  • Bumped google.golang.org/api from 0.58.0 to 0.59.0

0.3.9

  • Added initial support for multiple task orchestration in databricks_job #853
  • Fixed provider crash for new terraform states related to bug #813 in Terraform SDK v2.8.0 (#854)
  • Re-added skip_validation to databricks_instance_profile resource #762
  • Removed direct dependency on aws-sdk-go.

Updated dependency versions:

  • Reverted github.com/hashicorp/terraform-plugin-sdk/v2 from 2.8.0 to 2.7.0

0.3.8

  • Added databricks_repo resource to manage Databricks Repos (#771)
  • Added support for Azure MSI authentication (#743)
  • Added support to create databricks_user on the account level (#818)
  • Already deleted databricks_token don't fail the apply (#808)
  • Default terraform-mount clusters created for mounting for databricks_aws_s3_mount, databricks_azure_adls_gen1_mount, databricks_azure_adls_gen2_mount, and databricks_azure_blob_mount have now spark.scheduler.mode as FIFO (#828)
  • Fixed crash when using non-Azure authentication to mount Azure resources (#831)
  • Fixed replacement of instance_pool_id in databricks_cluster, when driver_instance_pool_id was not explicitly specified (#824)
  • Ingorning diff customization for permissions resource, so that new workspace deployments won't fail without explicit dependency on a workspace resource
  • Multiple documentation improvements

Deprecations

  • azure_workspace_name, azure_resource_group, azure_subscription_id, and azure_workspace_resource_id are deprecated and would be removed in v0.4.0. Please rewrite provider configuration with host = data.azurerm_databricks_workspace.example.workspace_url to achieve the same effect. Please check azurerm_databricks_workspace resource documentation for details.
  • azure_use_pat_for_spn, azure_use_pat_for_cli, and azure_pat_token_duration_seconds are deprecated to fully switch to AAD token authentication in the near future.
  • DATABRICKS_AZURE_CLIENT_SECRET environment variable is deprecated in favor of just using ARM_CLIENT_SECRET.
  • DATABRICKS_AZURE_CLIENT_ID environment variable is deprecated in favor of just using ARM_CLIENT_ID.
  • DATABRICKS_AZURE_TENANT_ID environment variable is deprecated in favor of just using ARM_TENANT_ID.

Updated dependency versions:

  • Bump github.com/aws/aws-sdk-go from 1.40.19 to 1.40.54
  • Bump github.com/Azure/go-autorest/autorest from 0.11.19 to 0.11.21
  • Bump github.com/Azure/go-autorest/autorest/azure/cli from 0.4.2 to 0.4.3
  • Bump github.com/Azure/go-autorest/autorest/adal from 0.9.14 to 0.9.16
  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.7.0 to 2.8.0
  • Bump github.com/zclconf/go-cty from 1.9.0 to 1.9.1
  • Bump golang.org/x/mod from 0.4.2 to 0.5.1
  • Bump google.golang.org/api from 0.52.0 to 0.58.0
  • Bump gopkg.in/ini.v1 from 1.62.0 to 1.63.2

0.3.7

  • Added databricks_obo_token resource to create On-Behalf-Of tokens for a Service Principal in Databricks workspaces on AWS. It is very useful, when you want to provision resources within a workspace through narrowly-scoped service principal, that has no access to other workspaces within the same Databricks Account (#736)
  • Added support for IAM credential passthrough with is_meta_instance_profile property for databricks_instance_profile (#745)
  • Fixed incorrect workspace update bug and added more validation error messaging (#649)
  • Clarify network modification procedure on active workspaces (#732)
  • Updated AWS IAM policy templates version to 2012-10-17 (databricks_aws_bucket_policy, databricks_aws_assume_role_policy, and databricks_aws_crossaccount_policy) (#688)
  • Various bug fixes in Databricks SQL resources

Updated dependency versions:

  • Bump github.com/aws/aws-sdk-go to v1.40.12
  • Bump github.com/hashicorp/hcl/v2 to v2.10.1
  • Bump github.com/zclconf/go-cty to v1.9.0
  • Bump golang.org/x/time to v0.0.0-20210723032227-1f47c861a9ac
  • Bump golang.org/x/tools to v0.1.5

0.3.6

  • Added support for hybrid pools (#689)
  • Added support for always_running jobs, which are restarted on resource updates (#715)
  • Azure CLI auth is now forcing JSON output (#717)
  • databricks_permissions are getting validation on terraform plan stage (#706)
  • Added databricks_directory resource (#690)
  • Added run_as_role field to databricks_sql_query (#684)
  • Added user_id attribute for databricks_user data resource, so that it's possible to dynamically create resources based on members of the group (#714)
  • Added more selectors to databricks_node_type data source (#723)
  • Azure auth with SPN now uses AAD token by default instead of PAT. Previous behavior (using PAT) could be restored by setting azure_use_pat_for_spn to true (#721)
  • deployment_name for databricks_mws_workspaces is now optional, how it should have been. This enables creation of Databricks workspaces without an account prefix.
  • To replicate default behavior of Databricks SQL UI, enable_photon is now true by default for databricks_sql_endpoint.
  • Various documentation and bugfixes

Updated dependency versions:

  • Bump github.com/aws/aws-sdk-go from 1.38.51 to 1.38.71
  • Bump github.com/Azure/go-autorest/autorest/azure/auth from 0.5.7 to 0.5.8
  • Bump github.com/Azure/go-autorest/autorest from 0.11.18 to 0.11.19
  • Bump github.com/Azure/go-autorest/autorest/adal from 0.9.13 to 0.9.14
  • Bump github.com/zclconf/go-cty from 1.8.3 to 1.8.4
  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.6.1 to 2.7.0

0.3.5

  • Fixed setting of permissions for SQLA endpoints (#661)
  • Added support for preloading of Docker images into instance pools (#663)
  • Added the databricks_user data source (#648)
  • Fixed support for spot_instance_policy in SQLA Endpoints (#665)
  • Added documentation for databricks_pipeline resource (#673)
  • Fixed mapping for databricks_service_principal on AWS (#656)
  • Made preview environment tests to run on a release basis

Updated dependency versions:

  • Bump github.com/zclconf/go-cty from 1.8.2 to 1.8.3
  • Bump github.com/aws/aws-sdk-go from 1.38.30 to 1.38.51

0.3.4

  • Fixed state refresh bugs in databricks_sql_permissions (#620, #619)
  • Fixed workspace_ids_filter mapping for databricks_mws_log_delivery (#635)
  • Multiple documentation improvements (#597, eb60d10, edcd4b1, 404bdab, #615, f14b825, e615c3a, #612)
  • Mounting clusters are recreated now, even when they are deleted (#637)
  • Fixed handling of empty blocks for clusters/jobs/instance pools (22cdf2f)
  • Mark instance pool attributes as ForceNew when it's requited (#629)
  • Switched to use https://staticcheck.io/ for static code analysis (#602)

Behavior changes

  • The customer_managed_key_id field in databricks_mws_workspaces resource is deprecated and should be replaced with managed_services_customer_managed_key_id (and optionally storage_customer_managed_key_id). databricks_mws_customer_managed_keys now requires the parameter use_cases (#642). If you've used the resource before, please add use_cases = ["MANAGED_SERVICES"] to keep the behaviour.

Updated dependency versions:

  • Bump github.com/aws/aws-sdk-go to v1.38.30
  • Bump github.com/hashicorp/go-retryablehttp to v0.7.0
  • Bump github.com/hashicorp/hcl/v2 to v2.10.0
  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 to v2.6.1
  • Bump github.com/zclconf/go-cty to v1.8.2

0.3.3

  • Added resources for SQL Analytics queries and dashboards: databricks_sql_query, databricks_sql_visualization, databricks_sql_dashboard, databricks_sql_widget (#553)
  • Added databricks_sql_permissions resource (#545)
  • Fixed documentation bugs (#603)
  • Improved resource exporter (#593)
  • Added missing properties to databricks_mws_private_access_settings (#590)
  • Include SQLA data source ID in databricks_sql_endpoint state (#601)
  • Apply debug_truncate_bytes also for response dumps (#589)
  • More verbose logging of databricks_cluster termination reason (#588)
  • Move non-auth provider config documentation into separate section (#587)

0.3.2

  • Fixed minor issues to add support for GCP (#558)
  • Fixed databricks_permissions for SQL Analytics Entities (#535)
  • Fixed incorrect HTTP 404 handling on create (#564, #576)
  • Fixed incorrect escaping of notebook names (#566)
  • Fixed entitlements for databricks_group (#549)
  • Fixed rate limiting to perform more than 1 request per second (#577)
  • Added support for spot instances on Azure (#571)
  • Added job schedules support for pause_status as a optional field. (#575)
  • Fixed minor documentation issues.

Updated dependency versions:

  • Bump github.com/aws/aws-sdk-go from 1.37.20 to 1.38.10
  • Bump github.com/hashicorp/hcl/v2 from 2.9.0 to 2.9.1
  • Bump github.com/zclconf/go-cty from 1.8.0 to 1.8.1
  • Bump github.com/google/go-querystring from 1.0.0 to 1.1.0

0.3.1

  • Added databricks_global_init_script resource to configure global init scripts (#487).
  • Added databricks_sql_endpoint resource (#498)
  • Added experimental resource exporter to generate configuration for entire workspace.
  • Improved user-facing documentaiton (#508, #516, #511, #504, [#492](Update docs in various places))
  • Simplified authentication issues debugging (#490)
  • Made cleaner error message for no config profile (#491)
  • Allow tokens without comment or expiration (#495)
  • Ensured consistent slashes in notebook paths for different OSes (#500)
  • Fix error message panic in command result parsing (#502)
  • Updated databricks_group data resource to allow non-alphanumeric characters in group name filter (#507)

Behavior changes

  • Assigning any permission to admins would result in an error, so that behavior is consistent (#486).

Updated dependency versions:

  • github.com/zclconf/go-cty from 1.2.1 to 1.7.1
  • github.com/Azure/go-autorest/autorest/azure/auth from 0.5.6 to 0.5.7
  • github.com/hashicorp/hcl/v2 from 2.3.0 to 2.8.2
  • github.com/aws/aws-sdk-go from 1.37.1 to 1.37.11
  • github.com/Azure/go-autorest/autorest from 0.11.17 to 0.11.18

0.3.0

  • Added configurable provisioning timeout for databricks_mws_workspaces, so that local DNS cache issues would be more tolerated.
  • Added [databricks_current_user] to simplify applying the same Terraform configuration by different users in the shared workspace for testing purposes.
  • Added client-side rate limiting to release the pressure on backend APIs and prevent client blocking (#465)
  • Member usernames, group names and instance profile names in databricks_group data source are now sorted and providing consistent behavior between runs (#449)
  • Fixed redundant multiple mounting clusters (#445)
  • Added optional parameter azure_environment to provider config which defaults to public (#437).
  • Added databricks_service_principal resource.
  • skip_validation from databricks_instance_profile was removed and is always set to true.
  • Added propagation of terraform version to User-Agent header, along with type of resource used.
  • databricks_notebook & databricks_dbfs_file got new source field to specify location of a local file.
  • databricks_notebook can have language field optional, as long as source is set to a file with .py, .scala, .sql, or .r extension.
  • databricks_me data source was added to represent user_name, home & id of the caller user (or service principal).
  • Added validation for secret scope name in databricks_secret, databricks_secret_scope and databricks_secret_acl. Non-compliant names may cause errors.
  • Added databricks_spark_version data source.
  • Fixed support for single node clusters support by allowing num_workers to be 0.
  • Fixed bug in destruction of IP access lists (#426).
  • All resource imports are now making call to corresponding Databricks API by default (#471).

Behavior changes

  • Removed deprecated library_jar, library_egg, library_whl, library_pypi, library_cran, and library_maven from databricks_cluster and databricks_job in favor of more API-transparent library configuration block.
  • Removed deprecated notebook_path and notebook_base_parameters from databricks_job in favor of notebook_task configuration block.
  • Removed deprecated jar_uri, jar_main_class_name, and jar_parameters from databricks_job in favor of spark_jar_task configuration block.
  • Removed deprecated python_file and python_parameters from databricks_job in favor of spark_python_task configuration block.
  • Removed deprecated spark_submit_parameters from databricks_job in favor of spark_submit_task configuration block.
  • Removed deprecated databricks_scim_user resource in favor of databricks_user.
  • Removed deprecated databricks_scim_group resource in favor of databricks_group.
  • Removed deprecated databricks_default_user_roles data source in favor of databricks_group data source.
  • Removed deprecated basic_auth and azure_auth provider configuration blocks in favor of documented authentication methods.
  • format, overwrite, and mkdirs were removed from databricks_notebook. To follow expected behavior of Terraform, notebooks are always overwritten.
  • skip_validation from databricks_instance_profile was removed and is always set to true for subsequent requests.
  • databricks_mws_workspace got verify_workspace_runnning removed and now validates all every deployment. In case deployment failed, it removes workspace that failed and returns error message with explanation.
  • default_tags were removed from databricks_instance_pool. disk_spec got new attribute disk_type, that contains azure_disk_volume_type and ebs_volume_type. This change is made to closer reflect API structure.
  • databricks_notebook & databricks_dbfs_file got content attribute renamed to content_base64 and now share the same logic to work with local files.

0.2.9

  • Fixed documentation issues.
  • Added missing resource importers and test to cover it.
  • Migrated build from TravisCI to GitHub Actions.
  • Fixed custom config_file issue configuration handling (#420).

Deprecations

  • databricks_notebook has got overwrite, mkdirs and format parameters, that always have to be set to certain values in order to follow expected behavior of terraform. These fields would be removed in 0.3 and always set to proper values.
  • databricks_notebook & databricks_dbfs_file field content is deprecated and would be renamed to content_base64 to further increase clarity.
  • databricks_dbfs_file has got content, content_b64_md5, overwrite, mkdirs, validate_remote_file fields deprecated and they will be removed in the next version, where critical code path will be shared with databricks_notebook.
  • network_error_messages and verify_workspace_runnning from databricks_mws_workspaces is deprecated and going to be removed in 0.3.
  • error_messages from databricks_mws_networks are deprecated and would be removed in 0.3.
  • ebs_volume_type and azure_disk_volume_type from databricks_instance_pool is going to be moved to disk_type sub-block in 0.3, which means you'll slightly have to modify configuration while migrating to 0.3. Computed field default_tags is going to be removed from resource. This is done to further increase maintainability of provider in the future.

Updated dependency versions:

  • github.com/aws/aws-sdk-go 35.36
  • github.com/hashicorp/go-retryablehttp 0.6.8
  • github.com/Azure/go-autorest/autorest 0.11.12

Behavior changes

  • min_idle_instances for databricks_instance_pool is now optional.
  • skip_validation for databricks_instance_profile is going to be removed in 0.3.

0.2.8

  • Added databricks_workspace_conf resource.
  • Added databricks_mws_log_delivery resource for billable usage & audit logs consumption.
  • Added databricks_node_type data source for simpler selection of node types across AWS & Azure.
  • Added Azure Key Vault support for databricks_secret_scope for Azure CLI authenticated users.
  • Added is_pinned support for databricks_cluster resource.
  • Fixed restarting cluster on changes in cluster configuration aren't related to the cluster configuration (issue #379)
  • Fixed issue #383 by cleaning up clusters that fail to start.
  • Fixed issue #382 by ignoring any incoming changes to deployment name of databricks_mws_workspaces, as well as propagating the right error messages.
  • Internal: API for retrieval of the cluster events.
  • Increased code coverage to 71%.

Updated dependency versions:

  • github.com/Azure/go-autorest/autorest v0.11.9
  • github.com/Azure/go-autorest/autorest/adal v0.9.5
  • github.com/Azure/go-autorest/autorest/azure/auth v0.5.3
  • github.com/Azure/go-autorest/autorest/azure/cli v0.4.2
  • gopkg.in/ini.v1 1.62.0

Deprecations

  • network_error_messages from databricks_mws_workspaces is deprecated and going to be removed in 0.3

0.2.7

  • Small fixes

0.2.6

  • Added support for customer managed keys for Accounts API.
  • Added databricks_user resource.
  • Added databricks_user_instance_profile resource.
  • Added databricks_group data source.

Updated dependency versions:

  • github.com/Azure/go-autorest/autorest v0.11.6
  • github.com/Azure/go-autorest/autorest/adal v0.9.4
  • github.com/Azure/go-autorest/autorest/azure/auth v0.5.2
  • github.com/Azure/go-autorest/autorest/azure/cli v0.4.1
  • gopkg.in/ini.v1 v1.61.0

Deprecations

  • databricks_scim_user is no longer receiving fixes and will be removed in 0.3, please rewrite using the databricks_user resource, which has more consistent semantics with databricks_group and works better with identity provider SCIM sync.
  • databricks_scim_group is no longer receiving fixes and will be removed in 0.3. Please rewrite using the databricks_group resource.
  • databricks_default_user_roles is no longer receiving fixes and will be removed in 0.3, please rewrite using databricks_user & databricks_group resources.

Behavior changes

  • State changes to legacy spark.databricks.delta.preview.enabled config option are now ignored by databricks_job & databricks_cluster
  • Libraries, which are installed on all clusters and are not part of cluster resource definition, won't be waited for INSTALLED status
  • Fixed "Secret scope ACL is MANAGE for all users by default" (issue 322). If you were relying on setting MANAGE permission to all users by default, you need to add initial_manage_principal = "users" to your resource "databricks_secret_scope" declaration.

0.2.5

0.2.4

  • Added Azure CLI authentication to bridge the gap of local development workflows and let more people use the provider.
  • All authentication is completely lazy-initialized, which makes it provider overall more stable.
  • Significantly increased unit test coverage, which runs before every merge of a pull request.
  • Introduced constantly running integration test suite environments: azsp, azcli & awsmt
  • Numerous stability improvements for clusters, mounts, libraries, notebooks, files, authentication and TLS connectivity.
  • Added ability to mount storage without explicitly defining a cluster, though it will still launch auto-terminating terraform-mount cluster to perform the mount.
  • databricks_cluster & databricks_job now share significant portion of configuration wiring code, therefore increasing the stability of codebase.
  • Added support for Terraform 0.13 local builds for those who develop or cannot wait for next release.
  • Added AWS IAM Policy data helpers to simplify new deployments.
  • Migrated all documentation to Terraform Registry format, therefore having a single always-accurate place for end-user guides.
  • Internally, codebase has been split into multiple packages, which should make further contributions simpler.

Updated dependency versions:

  • github.com/Azure/go-autorest/autorest v0.11.4
  • github.com/Azure/go-autorest/autorest/adal v0.9.2
  • github.com/Azure/go-autorest/autorest/azure/auth v0.5.1
  • github.com/aws/aws-sdk-go v1.34.13
  • gopkg.in/ini.v1 v1.60.2

Deprecations

  • library_* is no longer receiving fixes and will be removed in 0.3, please rewrite cluster & job resources to use library configuration block.
  • basic_auth provider block is no longer receiving fixesand will be removed in 0.3, please use username and password options
  • azure_auth provider block is no longer receiving fixesand will be removed in 0.3, please use azure_* options

Behavior changes

  • Previously, mounts code paths were different functions. This release unifies them to be a single testable codebase with different configuration options & re-use of the critical code paths. For maintainability reasons, there's no longer check performed on container & storage account names, but rather on high-level mount source uri.