go modules adoption checklist

[sigma]

/kind feature

Description:

This issue is to track the list of issues for a go modules (aka vgo) adoption. The list itself is a work in progress, and will be updated as issues popup or get resolved.

First class of issues: the current godep snapshot of packages is not a valid solution as far as go modules are concerned

• gonum and cadvisor conflict. details in gonum.org/v1/gonum module conflicts with use of golang.org/x/exp/inotify #68478
• heketi and k/k conflict. details in adjust to newer k8s APIs heketi/heketi#1272

This list needs to be empty to actually perform the migration. Post-migration, the burden will move to the dependency update workflow (similar issues will block the update).

Some more issues are to be expected, due to mainly 2 factors:

• go modules reason about test dependencies in addition to "regular" dependencies, which we currently completely ignore
• a large chunk of our dependencies are not versioned at all, meaning that no semver guarantees apply. It'll be easy to get incompatibilities once our dependencies start using go modules themselves.

Second class of issues: tooling needed

• it takes a non-negligible amount of effort to convince go to vendor everything we need at the top-level in k/k, given that some of our dependencies are actually only used by our staging sub-projects.
• since go modules don't track modifications at the sub-package level, we need to maintain Godeps.json files for a while, and there's no official tooling for that.
• our go modules dependencies need to be kept in sync across k/k and all staging repos.

https://github.com/sigma/vgo-k8s-tools has some (ugly) code to address all these, but is very much a work in progress

Third class of issues: go modules usability

• extremely opaque about what it does, and why. Took me some grepping the cache to understand why golang.org/x/exp was bumped in the first place in gonum.org/v1/gonum module conflicts with use of golang.org/x/exp/inotify #68478. With our kind of dependency graph complexity, lack of clarity is hard to handle.
• we'll still need some scripting around go modules to maintain our dependency graph. Hopefully that should be after running regular module operations in the main or staging repos.

[sigma]

@kubernetes/sig-testing-proposals

[k8s-ci-robot]

@sigma: Reiterating the mentions to trigger a notification:
@kubernetes/sig-testing-proposals

In response to this:

@kubernetes/sig-testing-proposals

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[BenTheElder]

/cc

[spiffxp]

FYI @kubernetes/sig-architecture-proposals

[bgrant0607]

cc @thockin @lavalamp @mattfarina

[lavalamp]

Excited to have someone looking at this!

[sdboyer]

It's worth noting that the path of least resistance (to the module system) will ultimately be to treat each currently-discrete staging repo as its own go module, and abandon the separate staging repos entirely.

ive also been thinking about how useful some kind of declaration would be to automatically treat all imports under a particular root as though there were a local replace declaration. As the number of Go modules within k8s increases, this will be increasingly useful, as it will obviate the need to litter replace statements everywhere.