Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge Microsoft/opengcs and Microsoft/hcsshim #973

Merged
merged 722 commits into from
May 17, 2021

Conversation

dcantah
Copy link
Contributor

@dcantah dcantah commented Mar 17, 2021

This was done more or less like the following: https://www.nomachetejuggling.com/2011/09/12/moving-one-git-repo-into-another-as-subdirectory/

There's really no reason for opengcs to be it's own repository anymore. The gcs binary itself is built out of the repo but opengcs itself isn't used as a library anymore so positives like versioning aren't as important. The repository itself will live on and we cut a tag before removing all of the v1 codebase so I think the bases are covered.

The idea is to build the gcs binary (and tools binaries) out of cmd like we do now for everything else. Gcs binaries and rootfs should still be able to be built with Make and not much should change about the overall opengcs workflow. Almost all of the code for opengcs resides in /internal/guest to have a clear separation between what's already here and what's new.

There's a lot of things I like about this. Number one being that for work that involves changes both to the shim/host and in the guest, we can simply have one PR now that houses both of the changes so we can more easily see how things fit together instead of swapping back and forth between two PRs/repos. It's also nice to co-locate all of the container code in one place, even if the name of this repo is getting more and more outdated :P

Signed-off-by: Daniel Canter dcanter@microsoft.com

John Howard and others added 30 commits February 26, 2019 22:12
Signed-off-by: John Howard <jhoward@microsoft.com>
Signed-off-by: John Howard <jhoward@microsoft.com>
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
Fix comment for privileged container detection
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
…_append

Fix additional append in host device list
We don't actually fully support HotRemove NIC from namespace but since
this only happens on teardown we likely don't need it. For now just log
this as a warning rather than returning an error because we do
actually try to safe remove it from the host side on shutdown which is
returning an error that is silently ignored.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
Change HotRemove NIC to warning instead of error
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
Signed-off-by: John Howard <jhoward@microsoft.com>
Signed-off-by: John Howard <jhoward@microsoft.com>
LCOW(v1) Remove kernel/initrd options
Signed-off-by: John Howard <jhoward@microsoft.com>
Signed-off-by: John Howard <jhoward@microsoft.com>
Does a sweeping update to our vendor system and the vendor files to more
recent versions.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
Signed-off-by: John Howard <jhoward@microsoft.com>
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
* Move entire repo into opengcs subdir
* `gofmt -s -w .` all the files to satisfy linter

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
@dcantah dcantah force-pushed the merge-opengcs branch 2 times, most recently from 45de9eb to 3cff3f4 Compare April 29, 2021 09:43
@dcantah
Copy link
Contributor Author

dcantah commented Apr 29, 2021

@microsoft/containerplat It breathes. Ready for 👀

@dcantah
Copy link
Contributor Author

dcantah commented Apr 30, 2021

@dims Forgot to @ you also, let me know if you see anything of concern with this :)

@dims
Copy link
Contributor

dims commented Apr 30, 2021

@dcantah the root go.mod/sum looks good. thanks for the ping!

Copy link
Contributor

@anmaxvl anmaxvl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

structure LGTM. needs rebase and possibly another sync?

@@ -0,0 +1,34 @@

# Open Guest Compute Service (opengcs) [![Build Status](https://github.com/microsoft/opengcs/actions/workflows/ci.yml/badge.svg)](https://github.com/microsoft/opengcs/actions/workflows/ci.yml)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this won't work anymore :P did we update gh actions?

Copy link
Contributor Author

@dcantah dcantah May 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh thanks for pointing this out I almost forgot, I left only the README in the opengcs subdir as I didn't know what we'd want to do here. e.g. have a section on the main readme now that gives a quick blurb about opengcs or a link to where it used to be, or any other myriad of options haha

Copy link
Contributor Author

@dcantah dcantah May 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep gh actions has a build_gcs step now which will do exactly what the gh actions for opengcs does, so the badge we have on the hcsshim README will suffice

Copy link
Contributor

@msscotb msscotb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

dcantah added 15 commits May 17, 2021 15:07
Merge Microsoft/opengcs and Microsoft/hcsshim

This was done more or less like the following: https://www.nomachetejuggling.com/2011/09/12/moving-one-git-repo-into-another-as-subdirectory/

This is solely just adding the repo to hcsshim in an opengcs subdirectory with the history, nothing else.

The reason for this is really that there's no reason for opengcs to be it's own repository anymore. The gcs binary itself
is built out of the repo but opengcs itself isn't really used as a library anymore so positives like versioning aren't as important.
The repository itself will live on and we cut a tag before removing all of the v1 codebase so I think the bases are covered.
* Add gitignore rules to top level gitignore and remove files that have
duplicates and make no sense here anymore (LICENSE, CODEOWNERS, go.mod/sum)

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
No need for it, all the deps wil be going in the hcsshim vendor directory.

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
* Move hack/, init/, vsockexec/ and Makefile top level

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
These were just exact copies of the log and oc packages already in hcsshim.

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
The debug package is useful and not strictly related to guest behavior so I've moved this to
a new debug package in internal, however for everything that's either Linux specific or opengcs
specific I've created a new `guest` package that houses these. This includes the kmsg package
(linux specific) the hcs v2 runtime guest code, all of the storage functionality that ends up
leading to mount syscalls, and guest side vmbus/pci related code.

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
This commit moves the code for the main opengcs binary and the gcstools binary
to cmd so we can build the binaries the same way we do for the all of the other
binaries we care about.

The Makefile will be edited in a future commit to be aware of this rearrange.

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
Move all of the code that used to live in the opengcs service directory to the
internal guest package to live with the rest of the opengcs internal code.

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
Add new job to the CI to run the Makefile for opengcs and delete the existing
opengcs CI yml file living in /opengcs/.github

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
* Change import paths to actually reflect where the files currently reside.
* Put a linux build tag on the gcs binary
* Get rid of the secondary gitignore file in the service directory, afaict this isn't
actually needed from checking what it's actually exlcuding. This was likely a relic of
a much older version of the gcs and binaries/artifacts that used to be built out of it.

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
Bring in the opengcs dependencies:
github.com/linuxkit/virtsock
github.com/vishvananda/netlink
github.com/vishvananda/netns
github.com/opencontainers/runc
github.com/mattn/go-shellwords

Signed-off-by: Daniel Canter <dcanter@microsoft.com>

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
Signed-off-by: Daniel Canter <dcanter@microsoft.com>
The exported function we were using just calls a unix syscall, so just call
this directly.

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
Running our CI explodes as when it gets to anything in /internal/guest there's a whole
bunch of unix going on.

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
We're not using ginkgo for any of our tests anymore (that I can see) so for Makefile
test just swap to running every test in /internal/guest which comprises (mostly) all of the
Linux specific guest features.

This commit also adds linter tags to a couple things to fix up deadcode warnings as well as fixes
a casing mistake on the setSubreaper function in /internal/guest/runtime/runc/runc.go

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
@dcantah dcantah merged commit 3b82d41 into microsoft:master May 17, 2021
thaJeztah added a commit to thaJeztah/docker that referenced this pull request Feb 23, 2022
full diff: microsoft/hcsshim@v0.8.23...v0.9.2

diff is hard to compare on github, because Microsoft/opengcs was merged into
hcsshim; microsoft/hcsshim#973

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
thaJeztah added a commit to thaJeztah/docker that referenced this pull request Feb 24, 2022
full diff: microsoft/hcsshim@v0.8.23...v0.9.2

diff is hard to compare on github, because Microsoft/opengcs was merged into
hcsshim; microsoft/hcsshim#973

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet