Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add api Go module and move all protos under api #10151

Merged
merged 6 commits into from
May 3, 2024
Merged

Add api Go module and move all protos under api #10151

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
25a2886
Fix v2 migrate for testutil package
dmcgowan Apr 29, 2024
4a45507
Move runc options to api directory
dmcgowan Apr 29, 2024
3e9cace
Move runtimeoptions to api directory
dmcgowan Apr 29, 2024
e1b94c0
Move protobuf package under pkg
dmcgowan Apr 29, 2024
2ac2b9c
Make api a Go sub-module
dmcgowan Apr 29, 2024
171fc14
Update release doc to mention API versioning
dmcgowan Apr 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
14 changes: 6 additions & 8 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,6 @@ SHIM_GO_LDFLAGS=-ldflags '-X $(PKG)/version.Version=$(VERSION) -X $(PKG)/version
# Project packages.
PACKAGES=$(shell $(GO) list ${GO_TAGS} ./... | grep -v /vendor/ | grep -v /integration)
API_PACKAGES=$(shell (cd api && $(GO) list ${GO_TAGS} ./... | grep -v /vendor/ | grep -v /integration))
NON_API_PACKAGES=$(shell $(GO) list ${GO_TAGS} ./... | grep -v /vendor/ | grep -v /integration | grep -v "containerd/api")
TEST_REQUIRES_ROOT_PACKAGES=$(filter \
${PACKAGES}, \
$(shell \
Expand Down Expand Up @@ -178,12 +177,10 @@ protos: bin/protoc-gen-go-fieldpath
$(eval TMPDIR := $(shell mktemp -d))
@mv ${ROOTDIR}/vendor ${TMPDIR}
@(cd ${ROOTDIR}/api && PATH="${ROOTDIR}/bin:${PATH}" protobuild --quiet ${API_PACKAGES})
@(PATH="${ROOTDIR}/bin:${PATH}" protobuild --quiet ${NON_API_PACKAGES})
find v2 -name '*.pb.go' -exec sh -c 'f={}; mkdir -p $$(dirname "$${f#v2/}"); echo mv $$f $${f#v2/}; mv $$f $${f#v2/}' \;
@mv ${TMPDIR}/vendor ${ROOTDIR}
@rm -rf ${TMPDIR} v2
go-fix-acronym -w -a '^Os' $(shell find api/ core/runtime/ -name '*.pb.go')
go-fix-acronym -w -a '(Id|Io|Uuid|Os)$$' $(shell find api/ core/runtime/ -name '*.pb.go')
go-fix-acronym -w -a '^Os' $(shell find api/ -name '*.pb.go')
go-fix-acronym -w -a '(Id|Io|Uuid|Os)$$' $(shell find api/ -name '*.pb.go')

check-protos: protos ## check if protobufs needs to be generated again
@echo "$(WHALE) $@"
Expand Down Expand Up @@ -474,17 +471,18 @@ vendor: ## ensure all the go.mod/go.sum files are up-to-date including vendor/ d
@$(GO) mod tidy
@$(GO) mod vendor
@$(GO) mod verify
#@(cd ${ROOTDIR}/integration/client && ${GO} mod tidy)
@(cd ${ROOTDIR}/api && ${GO} mod tidy)

verify-vendor: ## verify if all the go.mod/go.sum files are up-to-date
@echo "$(WHALE) $@"
$(eval TMPDIR := $(shell mktemp -d))
@cp -R ${ROOTDIR} ${TMPDIR}
@(cd ${TMPDIR}/containerd && ${GO} mod tidy)
@(cd ${TMPDIR}/containerd/integration/client && ${GO} mod tidy)
@(cd ${TMPDIR}/containerd && ${GO} mod vendor)
@(cd ${TMPDIR}/containerd && ${GO} mod verify)
@(cd ${TMPDIR}/containerd/api && ${GO} mod tidy)
@diff -r -u -q ${ROOTDIR} ${TMPDIR}/containerd
@rm -rf ${TMPDIR}
#@${ROOTDIR}/script/verify-go-modules.sh integration/client


help: ## this help
Expand Down
21 changes: 0 additions & 21 deletions Protobuild.toml
View file
Open in desktop

This file was deleted.

30 changes: 30 additions & 0 deletions RELEASES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,36 @@ and new fields on messages may be added if they are optional.
to the API by having a diff that the CI can run. These files are not intended to be
consumed or used by clients.

As of containerd 2.0, the API version diverges from the main containerd version.
While containerd 2.0 is a _major_ version jump for containerd, the API will remain
on 1.x to remain backwards compatible with prior releases and existing clients.
The 2.0 release adds the API to a separate Go module which can remain as the
`github.com/containerd/containerd/api` Go package and imported separately from the
rest of containerd.

The API minor version will continue to be incremented for each major and minor
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this depends on whether there were changes in the package / module? (currently it reads as it will always be incremented, which may not always be needed)

Copy link
Member

@thaJeztah thaJeztah May 1, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(basically; versioning should depend on the changes in the api module itself to follow SemVer, which could mean that only a patch release is needed between containerd releases, and versions can be updated separate from containerd releases; order of releasing may still require the API to be tagged before doing a containerd release of course)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sure there will always be changes, we can change the doc if a situation comes up later that doesn't make sense.

It was an idea to remove, but I don't think that will be necessary. There are a few different ways we could handle the versioning, I think the cleanest will be just to not do patch releases for prior minor versions though. The logic is that the highest level API needs to be supported anyway and should not break any compatibility. The whole issue that made this change necessary is related to multiple Go packages in use and in that same scenario the API version will go up. I feel like we have a few different ways we could approach the versioning though and this change gives us more options than we have without it.

version release of containerd. However, the API is tagged directly out of the
main branch with the minor version incrementing earlier in the next release cycle
rather than at the end. This means that after the containerd 2.0 release, the next
API change is tagged as `api/v1.9.0` prior to any containerd 2.1 release. The
latest API version should be backported to all supported versions and patch
releases for prior API versions should be avoided if possible.


| Containerd Version | API Version at Release |
|--------------------|------------------------|
| v1.0 | 1.0 |
| v1.1 | 1.1 |
| v1.2 | 1.2 |
| v1.3 | 1.3 |
| v1.4 | 1.4 |
| v1.5 | 1.5 |
| v1.6 | 1.6 |
| v1.7 | 1.7 |
| v2.0 | 1.8 |
| next | 1.9 |


### Metrics API

The metrics API that outputs prometheus style metrics will be versioned independently,
Expand Down
10 changes: 5 additions & 5 deletions api/events/container.pb.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion api/events/container.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ package containerd.events;
import "google/protobuf/any.proto";
import "github.com/containerd/containerd/api/types/fieldpath.proto";

option go_package = "github.com/containerd/containerd/v2/api/events;events";
option go_package = "github.com/containerd/containerd/api/events;events";
option (containerd.types.fieldpath_all) = true;

message ContainerCreate {
Expand Down
10 changes: 5 additions & 5 deletions api/events/content.pb.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion api/events/content.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ package containerd.events;

import "github.com/containerd/containerd/api/types/fieldpath.proto";

option go_package = "github.com/containerd/containerd/v2/api/events;events";
option go_package = "github.com/containerd/containerd/api/events;events";
option (containerd.types.fieldpath_all) = true;

message ContentDelete {
Expand Down
10 changes: 5 additions & 5 deletions api/events/image.pb.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion api/events/image.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ package containerd.services.images.v1;

import "github.com/containerd/containerd/api/types/fieldpath.proto";

option go_package = "github.com/containerd/containerd/v2/api/events;events";
option go_package = "github.com/containerd/containerd/api/events;events";
option (containerd.types.fieldpath_all) = true;

message ImageCreate {
Expand Down
10 changes: 5 additions & 5 deletions api/events/namespace.pb.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion api/events/namespace.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ package containerd.events;

import "github.com/containerd/containerd/api/types/fieldpath.proto";

option go_package = "github.com/containerd/containerd/v2/api/events;events";
option go_package = "github.com/containerd/containerd/api/events;events";
option (containerd.types.fieldpath_all) = true;

message NamespaceCreate {
Expand Down
8 changes: 4 additions & 4 deletions api/events/sandbox.pb.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion api/events/sandbox.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ package containerd.events;

import "google/protobuf/timestamp.proto";

option go_package = "github.com/containerd/containerd/v2/api/events;events";
option go_package = "github.com/containerd/containerd/api/events;events";

message SandboxCreate {
string sandbox_id = 1;
Expand Down
10 changes: 5 additions & 5 deletions api/events/snapshot.pb.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion api/events/snapshot.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ package containerd.events;

import "github.com/containerd/containerd/api/types/fieldpath.proto";

option go_package = "github.com/containerd/containerd/v2/api/events;events";
option go_package = "github.com/containerd/containerd/api/events;events";
option (containerd.types.fieldpath_all) = true;

message SnapshotPrepare {
Expand Down
10 changes: 5 additions & 5 deletions api/events/task.pb.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion api/events/task.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ import "google/protobuf/timestamp.proto";
import "github.com/containerd/containerd/api/types/mount.proto";
import "github.com/containerd/containerd/api/types/fieldpath.proto";

option go_package = "github.com/containerd/containerd/v2/api/events;events";
option go_package = "github.com/containerd/containerd/api/events;events";
option (containerd.types.fieldpath_all) = true;

message TaskCreate {
Expand Down
22 changes: 22 additions & 0 deletions api/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
module github.com/containerd/containerd/api

go 1.22.0

require (
github.com/containerd/ttrpc v1.2.3
github.com/containerd/typeurl/v2 v2.1.1
github.com/opencontainers/image-spec v1.1.0
google.golang.org/genproto/googleapis/rpc v0.0.0-20240401170217-c3f982113cda
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The verions of dependencies in the root go.mod and in the api should be same so it dont cause conflicts later. Currently its at google.golang.org/genproto/googleapis/rpc v0.0.0-20240415180920-8c6c420018be https://github.com/containerd/containerd/pull/10151/files#diff-33ef32bf6c23acb95f5902d7097b7a1d5128ca061167ec0716715b0b9eeaa5f6R74

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not change the version in the main repository, the go.mod only needs to specify a minimum version. Since API isn't compiled directly, the version isn't too important here. I think it would only be an issue for API to be ahead of the main repository, but also don't want it to be ahead of 1.6/1.7 since we intend to backport there.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree with @dmcgowan here

google.golang.org/grpc v1.59.0
google.golang.org/protobuf v1.33.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only thing worth considering is to see if those versions are required to be "current" version; some release branches may be on older image-spec and/or grpc; the API module should define minimum required version so that those branches can use the version they need (usually higher than minimum required version)

)

require (
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/sirupsen/logrus v1.8.1 // indirect
golang.org/x/net v0.21.0 // indirect
golang.org/x/sys v0.17.0 // indirect
golang.org/x/text v0.14.0 // indirect
)
Loading