acc: always run acceptance tests locally by radakam · Pull Request #6196 · databricks/cli

radakam · 2026-08-07T08:33:37Z

Changes

Drop the Local test.toml knob. Every acceptance test already runs against the testserver, so local runs include all tests by default; Cloud / CloudSlow still opt into real-workspace runs. DATABRICKS_TEST_SKIPLOCAL keeps skipping acceptance tests on cloud PR/integration runs (with withchanged re-enabling touched tests).

Most of the diff is mechanical Local = true removals from test.toml / out.test.toml.

After this merges, any test.toml that still sets Local will fail config loading with an undecoded-key error. Drop the Local line (tests always run locally now).

Why

There are now no remaining Local = false tests. Keeping an opt-out that nobody uses invites new cloud-only gaps and forces every test.toml / out.test.toml to repeat Local = true.

Every acceptance test already runs against the testserver, so the Local test.toml knob only existed to opt out. Drop it: local runs include all tests by default, and DATABRICKS_TEST_SKIPLOCAL still skips them on cloud PR/integration runs.

These only restated the root default after Local was removed, so delete them.

eng-dev-ecosystem-bot · 2026-08-07T09:21:39Z

Integration test report

Commit: 9a6b40a

Run: 31163668999

	Env	💚RECOVERED	🙈SKIP	✅pass	🙈skip	Time
💚	aws linux	4	4	303	1111	6:52
💚	aws windows	4	4	305	1109	5:29
💚	azure linux	4	4	302	1111	6:25
💚	azure windows	4	4	304	1109	8:07
💚	gcp linux	1	5	303	1111	6:05
💚	gcp windows	1	5	305	1109	5:44

8 interesting tests: 4 RECOVERED, 4 SKIP

	Test Name	aws linux	aws windows	azure linux	azure windows	gcp linux	gcp windows
💚	TestAccept	💚R	💚R	💚R	💚R	💚R	💚R
🙈	TestAccept/bundle/invariant/no_drift	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/ssh/connection	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
💚	TestFetchRepositoryInfoAPI_FromRepo	💚R	💚R	💚R	💚R	🙈S	🙈S
💚	TestFetchRepositoryInfoAPI_FromRepo/root	💚R	💚R	💚R	💚R
💚	TestFetchRepositoryInfoAPI_FromRepo/subdir	💚R	💚R	💚R	💚R

Top 8 slowest tests (at least 2 minutes):

duration	env	testname
5:18	azure windows	TestAccept
3:26	gcp windows	TestAccept
3:01	aws windows	TestAccept
2:34	azure windows	TestAccept/bundle/apps/job_permissions/DATABRICKS_BUNDLE_ENGINE=terraform
2:16	aws windows	TestAccept/bundle/apps/job_permissions/DATABRICKS_BUNDLE_ENGINE=terraform
2:12	aws linux	TestAccept/bundle/apps/job_permissions/DATABRICKS_BUNDLE_ENGINE=terraform
2:05	gcp windows	TestAccept/bundle/apps/job_permissions/DATABRICKS_BUNDLE_ENGINE=terraform
2:02	aws windows	TestAccept/bundle/apps/compute_size/DATABRICKS_BUNDLE_ENGINE=terraform

denik · 2026-08-07T10:01:31Z

Most of the diff is mechanical Local = true removals from test.toml / out.test.toml. The meaningful bits:

acceptance/internal/config.go — remove Local from TestConfig
acceptance/acceptance_test.go — local runs no longer gate on config.Local; skip-local cloud modes apply to all acceptance tests
acceptance/internal/materialized_config.go — stop writing Local into out.test.toml
acceptance/skiplocal_test.go — comments for the new semantics
acceptance/internal/config_test.go — fixtures no longer use Local

This does not need to be in PR description, it's already seen in the diff.

denik · 2026-08-07T10:04:36Z

 const (
 	SkipLocalEnvVar = "DATABRICKS_TEST_SKIPLOCAL"

-	// SkipLocalAll skips every test with Local = true.
+	// SkipLocalAll skips every acceptance test.


We should probably delete this?

~/work/cli % git grep -B 5 DATABRICKS_TEST_SKIPLOCAL=true Taskfile.yml- test-update-aws: Taskfile.yml- desc: Update acceptance test output (integration, requires deco access) Taskfile.yml- sources: *ACC_SOURCES_UPDATE Taskfile.yml- generates: *ACC_GENERATES_UPDATE Taskfile.yml- cmds: Taskfile.yml: - "deco env run -i -n aws-prod-ucws -- env DATABRICKS_TEST_SKIPLOCAL=true go test ./acceptance -run ^TestAccept$ -update -timeout=1h -v"

and we should not support DATABRICKS_TEST_SKIPLOCAL=true anymore, right?

This could be a follow up PR, given that it would be nice to merge this one quickly to minimize conflicts.

github-actions · 2026-08-07T10:06:38Z

Approval status: pending

`/acceptance/apps/` - needs approval

10 files changed
Suggested: @MarioCadenas
Also eligible: @fjakobs, @Shridhad, @atilafassina, @keugenek, @igrekun, @pkosiec, @pffigueiredo, @ditadi, @calvarjorge

`/acceptance/auth/` - needs approval

6 files changed
Suggested: @Divyansh-db
Also eligible: @simonfaltum, @renaudhartert-db, @chrisst, @mihaimitrea-db, @tejaskochar-db, @tanmay-db, @hectorcast-db, @parthban-db, @rauchy

`/acceptance/bundle/` - needs approval

1050 files changed
Suggested: @denik
Also eligible: @andrewnester, @pietern, @shreyas-goenka, @janniklasrose, @anton-107, @lennartkats-db

`/acceptance/cmd/sandbox/` - needs approval

22 files changed
Suggested: @pietern
Also eligible: @anwell-db, @akshaysingla-db, @shuochen0311, @samhuan-db

`/acceptance/experimental/air/` - needs approval

14 files changed
Suggested: @riddhibhagwat-db
Also eligible: @vinchenzo-db, @apeforest, @bfontain, @lu-wang-dl, @panchalhp-db, @maggiewang-db, @ben-hansen-db, @pardis-beikzadeh-db

`/acceptance/localenv/` - needs approval

21 files changed
Suggested: @anton-107
Also eligible: @rugpanov, @rclarey, @misha-db

`/acceptance/pipelines/` - needs approval

37 files changed
Suggested: @kanterov
Also eligible: @lennartkats-db, @jefferycheng1

General files (require maintainer)

184 files changed
Based on git history:

@denik -- recent work in acceptance/, acceptance/bundle/invariant/, acceptance/internal/

_{Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db, @janniklasrose, @lennartkats-db) can approve all areas.

See OWNERS for ownership rules.}

The Local knob was removed in #6196; regenerate the two out.test.toml files this branch adds so they match the current snapshot format.

#5587 was branched before #6196 removed Local, so it landed still setting it and now fails config loading with an undecoded-key error.

Local is no longer a test.toml key; keeping it fails config loading.

#6196 removed the Local config field and rewrote the other test.toml files to drop it, but #5587 added this test.toml with Local = true around the same time, so it was not swept. The acceptance harness rejects undecoded keys, failing the test on config parse before any assertion runs. Remove the key. Co-authored-by: Isaac

#6196 removed the Local config field and regenerated every out.test.toml snapshot, but this test was added on this branch in parallel, so its committed snapshot still carried Local = true. The post-test "no files changed" check regenerates it and fails on the diff. Regenerate to match. Co-authored-by: Isaac

radakam added 2 commits August 7, 2026 08:31

acc: always run acceptance tests locally

a6dcc50

Every acceptance test already runs against the testserver, so the Local test.toml knob only existed to opt out. Drop it: local runs include all tests by default, and DATABRICKS_TEST_SKIPLOCAL still skips them on cloud PR/integration runs.

acc: tighten comments after dropping Local

69ea231

radakam marked this pull request as ready for review August 7, 2026 08:42

acc: drop no-op Cloud=false leaf test.toml files

9a6b40a

These only restated the root default after Local was removed, so delete them.

denik reviewed Aug 7, 2026

View reviewed changes

denik approved these changes Aug 7, 2026

View reviewed changes

radakam added this pull request to the merge queue Aug 7, 2026

radakam mentioned this pull request Aug 7, 2026

acc: drop DATABRICKS_TEST_SKIPLOCAL=true #6200

Open

Merged via the queue into main with commit aebca14 Aug 7, 2026
36 checks passed

radakam deleted the acc-local-always-default branch August 7, 2026 11:08

radakam added a commit that referenced this pull request Aug 7, 2026

Drop Local from the telemetry test snapshots

7f95348

The Local knob was removed in #6196; regenerate the two out.test.toml files this branch adds so they match the current snapshot format.

radakam added a commit that referenced this pull request Aug 7, 2026

Drop the Local knob left behind by a concurrent merge

1cb9377

#5587 was branched before #6196 removed Local, so it landed still setting it and now fails config loading with an undecoded-key error.

radakam added a commit that referenced this pull request Aug 7, 2026

acc: drop Local from job_runs wait tests after #6196

bcfaf78

Local is no longer a test.toml key; keeping it fails config loading.

Navigation Menu

Uh oh!

acc: always run acceptance tests locally - #6196

acc: always run acceptance tests locally#6196
radakam merged 3 commits into
mainfrom
acc-local-always-default

radakam commented Aug 7, 2026 •

edited

Loading

Uh oh!

eng-dev-ecosystem-bot commented Aug 7, 2026

Uh oh!

denik commented Aug 7, 2026

Uh oh!

denik Aug 7, 2026

Uh oh!

denik Aug 7, 2026

Uh oh!

github-actions Bot commented Aug 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

radakam commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Why

Uh oh!

eng-dev-ecosystem-bot commented Aug 7, 2026

Integration test report

Uh oh!

denik commented Aug 7, 2026

Uh oh!

denik Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

denik Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 7, 2026

Approval status: pending

/acceptance/apps/ - needs approval

/acceptance/auth/ - needs approval

/acceptance/bundle/ - needs approval

/acceptance/cmd/sandbox/ - needs approval

/acceptance/experimental/air/ - needs approval

/acceptance/localenv/ - needs approval

/acceptance/pipelines/ - needs approval

General files (require maintainer)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

radakam commented Aug 7, 2026 •

edited

Loading

`/acceptance/apps/` - needs approval

`/acceptance/auth/` - needs approval

`/acceptance/bundle/` - needs approval

`/acceptance/cmd/sandbox/` - needs approval

`/acceptance/experimental/air/` - needs approval

`/acceptance/localenv/` - needs approval

`/acceptance/pipelines/` - needs approval