[test] experimental inner loop with restart workspacekit

[iQQBot]

Description

this PR is an experimental inner loop with restart workspacekit, this PR focus on custom docker image

Related Issue(s)

Fixes #

How to test

1. start a workspace in this preview environment
2. create a Dockerfile in in this workspace
3. use sudo rm -rf /demo && sudo mkdir /demo && sudo chown -R gitpod:gitpod /demo && docker buildx build -o /demo . this command to export docker filesystem to /demo folder
4. use gp inner-loop 1 to restart ring1 and switch to new root filesystem
5. you can use Dashboard inner loop context menu to switch back origin filesystem or use gp inner-loop 0

Release Notes

NONE

Documentation

Werft options:

• /werft with-local-preview
  If enabled this will build install/preview
• /werft with-preview
• /werft with-large-vm
• /werft with-integration-tests=all
  Valid options are all, workspace, webapp, ide, jetbrains, vscode, ssh

[werft-gitpod-dev-com]

started the job as gitpod-build-pd-workspacekit.5 because the annotations in the pull request description changed
(with .werft/ from main)

[csweichel]

I'd expect this to be routed through the supervisor/workspacekit connection that already exists. A much shorter path compared to workspace -> server -> ws-manager -> ws-daemon -> IWS -> workspacekit.

[akosyakov]

We decided to go with docker run for now: #15795 We may reconsider it later again. @iQQBot thank you a lot ❤️

There were several issues with this approach:

• slow extraction of rootfs and increase consumption of the ephemeral space -> Chris mentioned that we possibly can reconstruct rootfs in faster way by learning how buildkit store images.
• invasive changes in workspacekit
• from UX standpoint: loose of original workspace state

We decided for now to explore less invasive approach and try to run along, not instead.