Migrate bundle state filer read off workspace-files to /workspace/export - #6219
Merged
Merged
Conversation
Collaborator
Integration test reportCommit: 4612ac8
8 interesting tests: 4 SKIP, 3 flaky, 1 RECOVERED
Top 12 slowest tests (at least 2 minutes):
|
Sankalp-Mittal
changed the title
Sankalp-Mittal
force-pushed
the
sankalp-mittal/workspace-import-migration
branch
from
August 12, 2026 13:25
2a99228 to
af88c8f
Compare
Sankalp-Mittal
force-pushed
the
sankalp-mittal/workspace-export-migration
branch
from
August 12, 2026 13:46
291eb0a to
b11f1be
Compare
Sankalp-Mittal
changed the base branch from
sankalp-mittal/workspace-import-migration
to
main
August 12, 2026 13:46
Sankalp-Mittal
marked this pull request as ready for review
August 12, 2026 13:52
shreyas-goenka
approved these changes
Aug 13, 2026
| // Read via the raw apiClient.Do (not the SDK's Workspace.Download) so | ||
| // auth.WorkspaceIDHeaders can drop the CLI-only "none" workspace-id sentinel | ||
| // that Download would send literally. See PR #6149 for the write-path equivalent. | ||
| urlPath := "/api/2.0/workspace/export?path=" + url.QueryEscape(absPath) + "&direct_download=true" |
Contributor
There was a problem hiding this comment.
Why can't we use the SDK again here? I don't follow the point about auth.WorkspaceIDHeaders. Can we make this comment clearer?
Contributor
There was a problem hiding this comment.
Is it because the streaming endpoint is not accessible via the SDK?
Collaborator
Author
There was a problem hiding this comment.
no the issue is only when the during auth login the --skip-workspace flag is passed, this causes WorkspaceID = "none" to be passed (which I could handle separately, but I wanted to keep the code change as small as possible)
Collaborator
Author
There was a problem hiding this comment.
Can check out libs/auth/arguments.go:8 for definition of this sentinel
Collaborator
Author
There was a problem hiding this comment.
Also modified the comment
shreyas-goenka
requested changes
Aug 13, 2026
shreyas-goenka
left a comment
shreyas-goenka
left a comment
Contributor
There was a problem hiding this comment.
Sorry, did not mean to approve the PR. The current PR is good but please lets clarify why SDK is not usable here.
Sankalp-Mittal
force-pushed
the
sankalp-mittal/workspace-export-migration
branch
2 times, most recently
from
August 18, 2026 13:52
301ea4b to
284f5af
Compare
…tion filer.go now reads state via /api/2.0/workspace/export (a2946f2b7); regenerate the acceptance goldens to match. Note: state/basic and force_pull_commands filter captured requests on the old workspace-files path, so their goldens are now empty and their state-read assertions need the filters repointed at /workspace/export (follow-up). Co-authored-by: Isaac
Sankalp-Mittal
force-pushed
the
sankalp-mittal/workspace-export-migration
branch
from
August 18, 2026 14:55
e4d4b68 to
d110a58
Compare
shreyas-goenka
left a comment
shreyas-goenka
left a comment
Contributor
There was a problem hiding this comment.
Did you confirmed that this works for files bigger than 10MB? PR looks good to me other than why the API requests are vanishing in those tests.
Collaborator
Author
Yes I did, also fixed the API requests vanishing issue |
Sankalp-Mittal
commented
Aug 19, 2026
| "path": "/Workspace/Users/[USERNAME]/.bundle/state/default/state/deploy.lock" | ||
| } | ||
| } | ||
| { |
Collaborator
Author
There was a problem hiding this comment.
We get these extra reads in the output because reading of locks and state files already used this endpoint, but then the grep was made on the old api, when we shift the grep to the new endpoint these extra reads also creep up
Collaborator
Author
There was a problem hiding this comment.
Since the reading of the locks is not what we are testing I am excluding this from the output grep
shreyas-goenka
self-requested a review
August 19, 2026 23:36
Collaborator
Integration test reportCommit: 8a72f2d
42 interesting tests: 36 FAIL, 2 KNOWN, 2 flaky, 2 SKIP
Top 50 slowest tests (at least 2 minutes):
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
bundle: read deploy state via /workspace/export instead of deprecated workspace-files API
Summary
Switch the DABs state filer read (
bundle/deploy/filer.go,stateFiler.Read) fromGET /api/2.0/workspace-files/{path}to the SDK'sWorkspace.Download(ctx, absPath), which issuesGET /api/2.0/workspace/export?path=<path>&direct_download=true. This reads the bundle state files(
terraform.tfstate,resources.json) from the workspace.Why
The
workspace-filesendpoint is deprecated and is mapped to no API scope. Under context-basedingress (CBI) with fine-grained scoped tokens it was only reachable via the
all_apismaster scope,so
bundle deployreturned 403s for customers on least-privilege scoped tokens./workspace/exportis properly scoped, so state reads now work under least-privilege auth.
Implementation
Reads go through the SDK's
Workspace.Downloadhelper, so there is no separate API client, no manualauth.WorkspaceIDHeadersrouting headers (the SDK sets these), and no intermediate buffer —Downloadreturns the streaming body directly.direct_download=trueis load-bearing: plain/workspace/exportreturns base64 JSON capped at 10 MB,while the streaming variant handles files up to 500 MB, so large state files stream through fine.
Testing
GET /api/2.0/workspace/exportwithdirect_download=true(user_agent,statesuites).end-to-end.
This pull request and its description were written by Isaac.