direct: Support partial plan/deploy with --select#5413
Merged
Conversation
andrewnester
reviewed
Jun 2, 2026
andrewnester
reviewed
Jun 2, 2026
Collaborator
|
Commit: 820dc0e |
shreyas-goenka
approved these changes
Jun 2, 2026
| // FilterToSelected reduces the plan to the nodes in selected (format "type.name", | ||
| // e.g. "jobs.my_job") plus their transitive dependencies as recorded in each | ||
| // entry's DependsOn field. Nodes not reachable from the selected set are removed. | ||
| func (p *Plan) FilterToSelected(selected []string) { |
Contributor
There was a problem hiding this comment.
neat - in the future we can also limit state fetching to only these resources.
Filter deployment to a subset of resources using --select (e.g. --select my_job or --select jobs.my_job). Unqualified names that match exactly one resource type are resolved automatically; ambiguous names require a qualified form and produce an actionable error. Co-authored-by: Isaac
- Unit tests: multiple selectors (both kept), dependency not auto-included - Acceptance: --select A --select B keeps both resources - Acceptance: selecting a resource that depends on another fails at plan time with an engine-specific error; output split per engine Co-authored-by: Isaac
When a selected resource references another via an unresolved
\${resources.<type>.<name>.*} variable, that dependency is
automatically added to the selection. The expansion is transitive:
A → B → C selects all three when only A is given.
Co-authored-by: Isaac
Remove the dyn.Value BFS from the SelectResources mutator. Dependency auto-inclusion now uses the DependsOn entries that CalculatePlan already computes: - Direct engine: Plan.FilterToSelected BFS-expands the selection using DependsOn, then RunPlan returns the filtered plan. - Terraform engine: config is filtered to the exact selection before writing the terraform JSON (no dep tracking). - Validate/summary: simple config filter applied post-Initialize. New deployplan.Plan.FilterToSelected method and unit tests. Co-authored-by: Isaac
StringArrayVar rendered an ugly "stringArray" type in --help and did not split comma-separated values. Switch to StringSliceVar, matching the existing --var flag: cleaner "strings" help label and support for --select a,b in addition to repeated --select. Co-authored-by: Isaac
--select relies on the direct engine's plan dependency graph to expand transitive dependencies. On the terraform engine that graph isn't available, so rather than silently planning/deploying every resource, ProcessBundleRet now errors early with a hint to switch to the direct engine. Pin the select/select-dep acceptance tests to the direct engine and add select-terraform-error covering the rejection on terraform. Co-authored-by: Isaac
Drop the --select flag from validate and summary; the feature is only meaningful when a deployment plan is computed. This removes the now-dead FilterSelectedResources mutator and Resources.FilterResources helper, and simplifies ProcessBundleRet to just the terraform-engine rejection check (filtering happens in RunPlan via plan.FilterToSelected). Co-authored-by: Isaac
Run the select/select-dep acceptance tests on both engines (instead of pinning to direct) and capture output in per-engine out.plan.<engine>.txt files so the terraform rejection is visible. Commands are prefixed with errcode so the script records each result without aborting midway. This makes the dedicated select-terraform-error test redundant, so it is removed. Move the engine check before Initialize so the engine mismatch is reported before per-resource validation, and point the error at the direct-engine docs instead of repeating the env/config knobs. Co-authored-by: Isaac
Combine plan and deploy in the select/deploy test using readplanarg, and add EnvMatrix.READPLAN so the subset is deployed both inline and from a serialized plan (--plan). The serialized plan (out.plan.direct.json) contains only foo and its dependency bar, and the deploy creates exactly those two regardless of path. Also restore the minimal diff in cmd/bundle/plan.go: keep the existing InitFunc guard instead of making it unconditional. Co-authored-by: Isaac
Merge select/plan and select/deploy into one acceptance/bundle/select test against a single config. It covers selector resolution (not-found, qualified, unqualified-unique, repeated, comma-separated, transitive dependency inclusion) and an end-to-end deploy of the selected subset, including the serialized-plan (READPLAN) variant. Co-authored-by: Isaac
… units The mutator only resolves and normalizes b.Select (validating existence and expanding to type.name); it does not filter. Rename it to ResolveSelect to reflect that — filtering happens later via plan.FilterToSelected. Move the resolution-error cases (not-found, qualified-not-found) into a dedicated acceptance/bundle/select_errors test, pinned to direct (on terraform every --select hits the engine rejection first, covered by bundle/select). Trim the unit test to the cases acceptance can't cover: selector normalization and the ambiguous-selector error (unreachable in a loadable bundle because UniqueResourceKeys forbids duplicate keys across types). Co-authored-by: Isaac
Always set InitFunc (matching deploy.go) instead of guarding it behind a flag check — the guard was a micro-optimization and running the no-op mutator is harmless. Inline it in the ProcessOptions literal for clarity. Co-authored-by: Isaac
…test
The effective engine is only known for certain after the state is pulled
(stateDesc.Engine), so move the --select direct-engine check there instead
of resolving it separately before Initialize. ResolveSelect goes back to
pure selector resolution.
Add acceptance/bundle/select_errors/ambiguous documenting that an ambiguous
selector ("thing" as both a job and a pipeline) is rejected at load time by
UniqueResourceKeys, before selector resolution. Split select_errors into
missing/ and ambiguous/; both run on terraform and direct (resolution
errors are engine-independent now that the engine check runs later).
Co-authored-by: Isaac
…biguous}
Move the happy-path test to select/ok, and fold select_errors/{missing,
ambiguous} into select/. Common settings (Local/Cloud/Ignore) live in
select/test.toml; the deploy-specific RecordRequests and READPLAN matrix
stay in select/ok/test.toml.
Co-authored-by: Isaac
Co-authored-by: Isaac
The committed script used `trace musterr ... bundle deploy` while output.txt was generated from `errcode trace ... bundle plan`, so CI regenerated and diverged. Restore the intended errcode/plan script that matches the output. Co-authored-by: Isaac
Rename select/ok to select/basic and extend it: - Enable on cloud (Local+Cloud) with a $UNIQUE_NAME-templated bundle and a destroy cleanup trap, pinned to the direct engine (the only engine where --select works). The serialized plan is written to an ignored plan.json so cloud runs don't diverge on remote state. - After the partial deploy (--select jobs.foo deploys foo+bar), run a full plan/deploy with no selection: the plan shows only baz to add with foo and bar unchanged, deploy creates baz, and a final plan reports no changes. Add select/rejected (terraform, local) covering the --select rejection that select/basic no longer exercises now that it is direct-only. Co-authored-by: Isaac
denik
force-pushed
the
denik/select-deployment
branch
from
b06638a to
d718aa4
Compare
Apply the serialized-plan path to the full deploy after the partial deploy too: write the plan JSON to a temp (ignored) file and deploy via readplanarg. The JSON plans stay out of the recorded output because they embed remote state that differs between local and cloud; the textual `bundle plan` output is what's recorded and asserted. Co-authored-by: Isaac
Set the select_used bool metric in the ResolveSelect mutator (runs only when --select is non-empty). Add a telemetry acceptance test asserting the metric is emitted on deploy. Co-authored-by: Isaac
Instead of a dedicated telemetry test, capture the select_used metric in the existing select/basic test via the print_telemetry_bool_values helper after the partial deploy. Also check `bundle summary` there, confirming foo and bar are deployed while baz is not. Co-authored-by: Isaac
Collaborator
|
Commit: 63b717b |
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Jun 4, 2026
## Release v1.2.0 ### CLI * `experimental open` now opens every DABs resource type that has a workspace URL, picking up `catalogs`, `schemas`, `volumes`, `database_instances`, `database_catalogs`, `synced_database_tables`, `postgres_catalogs`, `postgres_synced_tables`, `quality_monitors`, `vector_search_endpoints`, and `vector_search_indexes` ([#5346](#5346)). ### Bundles * Retry transient HTTP 5xx and 408 errors in direct deployment engine ([#5349](#5349), [#5364](#5364)). * Preserve `.designer.ipynb` suffix when translating notebook task paths so Lakeflow Designer files referenced from a `notebook_task` resolve correctly in the workspace ([#5370](#5370)). * Fix script output dropping last line without trailing newline ([#4995](#4995)). * engine/direct: Add `--select` flag to `bundle plan` and `bundle deploy` to plan/deploy a subset of resources (e.g. `--select my_job` or `--select jobs.my_job`); resources referenced by the selection are included transitively ([#5413](#5413)). * Support `purge_on_delete: true` on `postgres_projects` so bundles can hard-delete a Lakebase project on destroy (skipping the soft-delete retention window) ([#5414](#5414)). * Support terraform references in direct engine ([#5392](#5392)) * Support lifecycle.started for SQL warehouses ([#5348](#5348)) ### Dependency updates * Bump Go toolchain to 1.26.4 ([#5420](#5420)). * Bump `github.com/databricks/databricks-sdk-go` from v0.136.0 to v0.141.0 ([#5361](#5361)) * Bump Terraform provider from v1.115.0 to v1.117.0 ([#5421](#5421))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
New --select option for plan & deploy that allows selecting one or more resources to deploy. The dependencies are included as well. File uploads are done in full as with usual deploy. Direct engine only.
Why
If you're working on a single resource from the target and target contains many jobs, sometimes you don't want to pollute your workspace with (/ wait for) other resources.
Tests
New acceptance and unit tests.