Skip to content

direct: refactor deploy/plan interaction#3350

Merged
denik merged 3 commits intomainfrom
denik/refactor-deploy-plan
Aug 5, 2025
Merged

direct: refactor deploy/plan interaction#3350
denik merged 3 commits intomainfrom
denik/refactor-deploy-plan

Conversation

@denik
Copy link
Copy Markdown
Contributor

@denik denik commented Aug 4, 2025

Changes

"bundle deploy" no longer calculates and classifies the diff, it relies on action types calculated during "bundle plan". Note, this will need to be further developed once we handle ${resources} references - we might want to recalculate the diff in some cases.

We no longer have UpdateUpdatesID setting on the resource. Instead, resources can implement optional method DoUpdateWithID that can return an updated id as part of the update. Currently, only volumes uses that.

The original DoUpdate method no longer allows to return ID. By convention, if possible, it also checks that ID is not changed by the backend.

There is new ActionType: "update_with_id". This is what triggers DoUpdateWithID() call. However, when stringified, this is presented as "update" so that it's compatible with terraform output in "bundle plan".

Why

The main motivation is to have information at plan-time on whether we're going to update ID or not. If not, we can resolve ${resources...id} right away and have more parallelism at deploy time.

Additionally, this simplifies usual case DoUpdate() that resources need to implement.

Tests

Existing tests. The new action type is tested by #3349

@eng-dev-ecosystem-bot
Copy link
Copy Markdown
Collaborator

eng-dev-ecosystem-bot commented Aug 4, 2025

Run: 16749777131

Env ✅‌pass 🔄‌flaky 🙈‌skip
✅‌ aws linux 308 459
✅‌ aws windows 309 458
🔄‌ aws-ucws linux 406 8 359
🔄‌ aws-ucws windows 406 9 358
✅‌ azure linux 308 458
🔄‌ azure windows 300 9 457
🔄‌ azure-ucws linux 408 8 356
🔄‌ azure-ucws windows 416 1 355
✅‌ gcp linux 307 460
✅‌ gcp windows 308 459
17 failing tests:
Test Name aws-ucws linux aws-ucws windows azure windows azure-ucws linux azure-ucws windows
TestAccept/bundle/deploy/dashboard/detect-change 🔄‌flaky 🔄‌flaky ✅‌pass 🔄‌flaky ✅‌pass
TestAccept/bundle/deploy/dashboard/nested-folders 🔄‌flaky 🔄‌flaky ✅‌pass 🔄‌flaky ✅‌pass
TestAccept/bundle/deploy/dashboard/simple 🔄‌flaky 🔄‌flaky 🔄‌flaky 🔄‌flaky ✅‌pass
TestAccept/bundle/deploy/dashboard/simple_outside_bundle_root 🔄‌flaky 🔄‌flaky ✅‌pass 🔄‌flaky ✅‌pass
TestAccept/bundle/deploy/dashboard/simple_syncroot 🔄‌flaky 🔄‌flaky ✅‌pass 🔄‌flaky ✅‌pass
TestAccept/bundle/deployment/bind/cluster ✅‌pass ✅‌pass 🔄‌flaky ✅‌pass ✅‌pass
TestAccept/bundle/deployment/bind/dashboard 🔄‌flaky 🔄‌flaky ✅‌pass 🔄‌flaky ✅‌pass
TestAccept/bundle/deployment/bind/dashboard/recreation 🔄‌flaky 🔄‌flaky ✅‌pass 🔄‌flaky ✅‌pass
TestAccept/bundle/deployment/bind/job/job-abort-bind ✅‌pass ✅‌pass 🔄‌flaky ✅‌pass ✅‌pass
TestAccept/bundle/destroy/jobs-and-pipeline ✅‌pass ✅‌pass 🔄‌flaky ✅‌pass ✅‌pass
TestAccept/bundle/destroy/jobs-and-pipeline/DATABRICKS_CLI_DEPLOYMENT=terraform ✅‌pass ✅‌pass 🔄‌flaky ✅‌pass ✅‌pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=no/NBOOK=no/PY=yes ✅‌pass ✅‌pass 🔄‌flaky ✅‌pass ✅‌pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=yes/NBOOK=yes/PY=no ✅‌pass ✅‌pass 🔄‌flaky ✅‌pass ✅‌pass
TestAccept/bundle/templates/default-python/integration_classic ✅‌pass ✅‌pass 🔄‌flaky ✅‌pass ✅‌pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=direct-exp/UV_PYTHON=3.11 ✅‌pass ✅‌pass 🔄‌flaky ✅‌pass ✅‌pass
TestDashboardAssumptions_WorkspaceImport 🔄‌flaky 🔄‌flaky ✅‌pass 🔄‌flaky 🔄‌flaky
TestFetchRepositoryInfoAPI_FromRepo ✅‌pass 🔄‌flaky ✅‌pass ✅‌pass ✅‌pass

@denik denik temporarily deployed to test-trigger-is August 4, 2025 09:57 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is August 4, 2025 12:05 — with GitHub Actions Inactive
// TODO: Name, SchemaName changes should result in re-create
for _, change := range changes {
if change.Path.String() == ".name" {
return deployplan.ActionTypeUpdateWithID
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an example jobs or pipelines allow changing the name but they don't implement DoUpdateWithId, how does it work with them?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline - this change is for volumes only.

Base automatically changed from denik/volume-change-name to main August 5, 2025 09:55
@denik denik temporarily deployed to test-trigger-is August 5, 2025 12:13 — with GitHub Actions Inactive
denik added 3 commits August 5, 2025 14:16
"bundle deploy" no longer calculates and classifies the diff, it relies
on action types calculated during "bundle plan". (Note, this will need to
be further developed once we handle ${resources} references - we might want to
recalculate the diff in some cases.).

We no longer have UpdateUpdatesID setting on the resource. Instead,
resources can implement optional method DoUpdateWithID that can return an
updated id as part of the update. Currently, only volume uses that.

The original DoUpdate method no longer allows to return ID. By
convention, if possible, it also checks that ID is not changed by the
backend.

There is new ActionType: "update_with_id". This is what triggers
DoUpdateWithID() call. However, when stringified, this is presented as
"update" so that it's compatible with terraform output in "bundle plan".

The main motivation is to have information at plan-time on whether we're
going to update ID or not. If not, we can resolve ${resources.*.*.id}
right away and have more parallelism at deploy time.
@denik denik force-pushed the denik/refactor-deploy-plan branch from eb4d356 to b5d1309 Compare August 5, 2025 12:16
@denik denik temporarily deployed to test-trigger-is August 5, 2025 12:16 — with GitHub Actions Inactive
@denik denik added this pull request to the merge queue Aug 5, 2025
Merged via the queue into main with commit 0437317 Aug 5, 2025
13 checks passed
@denik denik deleted the denik/refactor-deploy-plan branch August 5, 2025 13:20
alyssa-db pushed a commit that referenced this pull request Aug 7, 2025
## Changes
"bundle deploy" no longer calculates and classifies the diff, it relies
on action types calculated during "bundle plan". Note, this will need to
be further developed once we handle ${resources} references - we might
want to recalculate the diff in some cases.

We no longer have UpdateUpdatesID setting on the resource. Instead,
resources can implement optional method DoUpdateWithID that can return
an updated id as part of the update. Currently, only volumes uses that.

The original DoUpdate method no longer allows to return ID. By
convention, if possible, it also checks that ID is not changed by the
backend.

There is new ActionType: "update_with_id". This is what triggers
DoUpdateWithID() call. However, when stringified, this is presented as
"update" so that it's compatible with terraform output in "bundle plan".

## Why
The main motivation is to have information at plan-time on whether we're
going to update ID or not. If not, we can resolve ${resources.*.*.id}
right away and have more parallelism at deploy time.

Additionally, this simplifies usual case DoUpdate() that resources need
to implement.

## Tests
Existing tests. The new action type is tested by
#3349
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants