Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for backing up and restoring workspaces from snapshot volumes #9475

Merged
merged 8 commits into from May 25, 2022
Merged

Add support for backing up and restoring workspaces from snapshot volumes #9475

merged 8 commits into from May 25, 2022

Conversation

sagor999
Copy link
Contributor

@sagor999 sagor999 commented Apr 22, 2022 •
edited

Description

Add support for creating snapshot volume from PVC that was used by workspace to mount /workspace folder, and then use that snapshot volume to restore workspace from it.

Related Issue(s)

Fixes #9469
Fixes #9984

How to test

Loom video showing how this is working:
https://www.loom.com/share/feed9ce2d31c4e7ab71a7c094252fd15

Loom video walking through code changes and explaining them:
https://www.loom.com/share/46b5719f32894af7b3817d784ef6f608

Test steps if you have access to workspace preview environment:
You need to use https://github.com/gitpod-io/workspace-preview and use installer that was generated by this PR.
(Support for dev env for this feature will be a separate PR)
Then go into admin panel and enable persistent_volume_claim feature flag on your account.
Now create a new workspace. Edit some files. Do things in there.
Close workspace.
Observe that any pending changes are correctly showing up in dashboard.
In k8s, you can check that VolumeSnapshot object was created when workspace was stopped.
Now open that workspace again. All your files should be there with correct permissions, etc.
Additionally you can observe that no S3 backup was created for your workspace. All your data is now stored in snapshot volume.

Release Notes

[experimental] add support for backing up and restoring workspace's persistent volume claim via snapshot volume.

Documentation

@sagor999 sagor999 force-pushed the pavel/9469 branch 3 times, most recently from a8f8ab1 to 546fa5e Compare April 26, 2022 00:23
@sagor999
Copy link
Contributor Author

sagor999 commented May 3, 2022 •
edited by werft-gitpod-dev-com bot

/werft run

👍 started the job as gitpod-build-pavel-9469.34
(with .werft/ from main)

@sagor999 sagor999 force-pushed the pavel/9469 branch 5 times, most recently from 6351617 to d10f599 Compare May 5, 2022 21:07
This was referenced May 5, 2022
Merged
Merged
Merged
Merged
Closed
Merged
Closed
@sagor999 sagor999 changed the title Pavel/9469 Add support for backing up and restoring workspaces from snapshot volumes May 5, 2022
@sagor999
Copy link
Contributor Author

sagor999 commented May 9, 2022 •
edited by werft-gitpod-dev-com bot

/werft run with-clean-slate-deployment

👍 started the job as gitpod-build-pavel-9469.60
(with .werft/ from main)

@sagor999
Copy link
Contributor Author

sagor999 commented May 10, 2022 •
edited by werft-gitpod-dev-com bot

/werft run with-clean-slate-deployment

👍 started the job as gitpod-build-pavel-9469.61
(with .werft/ from main)

@sagor999 sagor999 force-pushed the pavel/9469 branch 4 times, most recently from 2bbd591 to 66b7cee Compare May 11, 2022 01:00
@sagor999
Copy link
Contributor Author

@jenting thank you for review! ❤️ PTAL

jenting
jenting reviewed May 19, 2022
View reviewed changes
Copy link
Contributor

@jenting jenting left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recently PRs deprecated the xerrors.Errorf, and use fmt.Errorf instead.
We could update it if you want or in the following PR.

components/ws-manager/pkg/manager/monitor.go Outdated Show resolved Hide resolved
components/ws-manager/pkg/manager/monitor.go Outdated Show resolved Hide resolved
components/ws-manager/pkg/manager/monitor.go Show resolved Hide resolved
components/ws-manager/pkg/manager/monitor.go Outdated
return false, err
}
if volumeSnapshot.Status != nil {
if volumeSnapshot.Status.ReadyToUse != nil && *(volumeSnapshot.Status.ReadyToUse) {
Copy link
Contributor

@jenting jenting May 19, 2022 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, personally I'd prefer to use a function to do check (is not nil and is true/false), refer to code.

I've check the code in the common-go, looks like currently we don't have the same function. We could enhance it in the future 🛹

components/ws-manager/pkg/manager/monitor.go Show resolved Hide resolved
components/ws-manager/pkg/manager/monitor.go Show resolved Hide resolved
components/ws-manager/pkg/manager/monitor.go Outdated Show resolved Hide resolved
@sagor999
Copy link
Contributor Author

@jenting PTAL ❤️

jenting
jenting approved these changes May 19, 2022
View reviewed changes
Copy link
Contributor

@jenting jenting left a comment •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did not launch a cluster to test (will do it tomorrow) but workspace related code LGTM.
Let's ship the 🛹 , thank you @sagor999 for the hard work.

corneliusludmann
corneliusludmann approved these changes May 19, 2022
View reviewed changes
Copy link
Member

@corneliusludmann corneliusludmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mainly focused on reviewing the self-hosted team owned files. These are looking awesome. 🕺 Ship it! 🚢

@jenting jenting mentioned this pull request May 20, 2022
Merged
@sagor999
Copy link
Contributor Author

sagor999 commented May 23, 2022 •
edited

@aledbf can you please review /components/ws-manager-api as you are code owner of that component.
@Furisto can you please review /components/ws-daemon-api, it is just formatting changes.
@geropl can you please review initializer.proto in /components/content-service-api. You are the owner of that component. And it just one line change.

Thank you!

easyCZ
easyCZ approved these changes May 24, 2022
View reviewed changes
Copy link
Member

@easyCZ easyCZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM based on my context. Also @geropl is out so hopefully this review unblocks you.

For next time, I'd recommend making the Proto changes in a separate PR.

@sagor999
Copy link
Contributor Author

/hold
to look at the issue that JenTing mentioned

@sagor999
Copy link
Contributor Author

/unhold
verified, works now.

@roboquat roboquat merged commit 905be0a into main May 25, 2022
@roboquat roboquat deleted the pavel/9469 branch May 25, 2022 07:47
@roboquat roboquat added the deployed: webapp Meta team change is running in production label May 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jenting jenting jenting approved these changes

@aledbf aledbf aledbf approved these changes

@easyCZ easyCZ easyCZ approved these changes

@corneliusludmann corneliusludmann corneliusludmann approved these changes

@Furisto Furisto Awaiting requested review from Furisto

@csweichel csweichel Awaiting requested review from csweichel

@geropl geropl Awaiting requested review from geropl

Assignees
No one assigned
Labels
deployed: webapp Meta team change is running in production release-note size/XXL team: delivery Issue belongs to the self-hosted team team: webapp Issue belongs to the WebApp team team: workspace Issue belongs to the Workspace team
Projects
None yet
Milestone
No milestone
6 participants
@sagor999 @jenting @aledbf @easyCZ @corneliusludmann @roboquat