Add workspace experiment to maintain state between updates and capture success/failure of each #6693
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bdragon
force-pushed
the
bdragon/shared-workspace
branch
from
ee4b9d5
to
2057ea0
Compare
brrygrdn
reviewed
Feb 21, 2023
bdragon
force-pushed
the
bdragon/shared-workspace
branch
3 times, most recently
from
136c1d9
to
0c2a4f4
Compare
bdragon
force-pushed
the
bdragon/shared-workspace
branch
5 times, most recently
from
581606c
to
a1e6de5
Compare
bdragon
force-pushed
the
bdragon/shared-workspace
branch
4 times, most recently
from
3ed4c41
to
4976fae
Compare
bdragon
force-pushed
the
bdragon/shared-workspace
branch
14 times, most recently
from
bd64f0b
to
56069fb
Compare
bdragon
force-pushed
the
bdragon/shared-workspace
branch
2 times, most recently
from
8aaca08
to
1059a9c
Compare
bdragon
force-pushed
the
bdragon/shared-workspace
branch
2 times, most recently
from
55f34c5
to
5d47c09
Compare
bdragon
changed the title
bdragon
force-pushed
the
bdragon/shared-workspace
branch
from
5d47c09
to
0e428ce
Compare
brrygrdn
reviewed
Apr 6, 2023
updater/lib/dependabot/updater/operations/group_update_all_versions.rb
Outdated
Show resolved
Hide resolved
bdragon
force-pushed
the
bdragon/shared-workspace
branch
from
0e428ce
to
40ecc01
Compare
brrygrdn
reviewed
Jun 19, 2023
There was a problem hiding this comment.
This looks good to me - as I mentioned in a comment above I think we should reduce the scope of this PR to just introducing the workspace library code and then add in the call sites downstream of the changes in #7404.
That'll allow us to ship this without impacting any production jobs and have a smoke test in place for vendoring that proves this and #7404 fix the problem before we ship to production.
brrygrdn
reviewed
Jun 19, 2023
added 9 commits
June 20, 2023 16:10
Modifies `Dependabot::SharedHelpers.in_a_temporary_repo_directory` so that if the `:shared_workspace` experiment is enabled, a `Dependabot::Workspace::Git` will be initialized and used to execute the given block, unless an existing workspace is given via the `:workspace` kwarg, in which case the given block will be executed using the existing workspace.
Build a workspace for the in-flight job and set Dependabot::Workspace.active_workspace if the :shared_workspace experiment is enabled.
accepts an optional memo to be associated with each change attempt. We were looking for it in the kwargs given to but in order to avoid changes to that interface we will omit the memo for now.
Keeping the `Dependabot::Workspace` library code but holding off on integrating it until vendoring support for grouped updates is complete.
bdragon
force-pushed
the
bdragon/shared-workspace
branch
from
8fa1116
to
24f3bfd
Compare
brrygrdn
approved these changes
Jun 20, 2023
brettfo
pushed a commit
to brettfo/dependabot-core
that referenced
this pull request
Oct 11, 2023
…kspace Add workspace experiment to maintain state between updates and capture success/failure of each
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Over the course of a version update, Dependabot will typically use the
in_a_temporary_repo_directoryandin_a_temporary_directoryshared helpers (depending on whether the current ecosystem has been configured toalways_clone_for_package_manager?or not, respectively) to make changes to manifest files using the native package manager.This PR introduces the concept of a shared, or persistent workspace as an experiment behind the
shared_workspacefeature flag. When the experiment is enabled, it will only be used by always-clone package managers (in practice:go_modules,npm_and_yarn,pub, andterraform) or when vendoring dependencies. This is because the implementation is backed by git and requires a git repo to work. In the future, we may simplygit init(like this) before using thein_a_temporary_directoryshared helper so that the concept can be used across all ecosystems regardless of whether or not they clone.A workspace is simply a git repo: successful changes are commits and failed changes are stashes. When the experiment is enabled and
in_a_temporary_repo_directoryis invoked, its block argument will be passed to the workspace's#changemethod. If the block exits without failure, any changes to the repo contents will be captured as a successful change and committed to the repo; if the block raises, the state of the repo directory (and the exception that was raised) will be captured as a failed change attempt and stashed to the repo.This concept allows us to iteratively make changes to manifest files, each change starting from the state of the previous successful change, rather than starting from scratch each time. This has obvious potential benefits for grouped updates, in which several updates are made in succession over the course of a single version update job. As of this PR, if a change raises, the exception will be re-raised to mirror current behavior. In the future we may want to inspect and potentially suppress the failure and carry on with the next update in order to allow partially-successful grouped updates.
While this experiment may ultimately support grouped updates, it is being released separately so that we can start evaluating its viability independent of grouped updates work.