Updates dependencies to get us pretty close to current #336

tnthornton · 2023-06-23T22:33:26Z

Description of your changes

It's been a long time since dependencies were updated in this repo. Some of the consequences of that are:

The crossplane/crossplane dependency hasn't been updated since 1.6 resulting in issues such as:
- Latest up version doesn't add regexp patch transforms to the package.yaml file #329
Any security issues that may have cropped over the last "months" time would remain unresolved.
The general PR, as seen here, is much larger changeset than we'd tend to like.

The overarching goal of this PR is to get us roughly caught up on the crossplane/crossplane dependency which results in a cascade affect of the other dependencies needing to be updated.

In addition:

The minimum Go version is being updated to 1.20 (which has affects on the GolangCI Lint version).
The build submodule is also updated.

Fixes #329

I have:

Read and followed Upbound's contribution process.
Run make reviewable to ensure this PR is ready for review.

How has this code been tested

Basic validation: make build test reviewable
More extensive tests are in progress:

xpkg APIs, ensuring above linked issues are resolved.
xpls still works the way it did prior to this update

Testing xpls

make build
move up into terminal path.
Check version

up --version
v0.18.0-rc.0.11.g6bceab2

See below screenshots, prior functionality appears to continue to work

Testing xpkg commands

a. Checking if #329 is resolved

Followed repro steps in Latest up version doesn't add regexp patch transforms to the package.yaml file #329 and ended up with an ezgi-platform-ref-gcp.xpkg file.
Using up, ran the following command to extract the package up xpkg xp-extract --from-xpkg ezgi-platform-ref-gcp.xpkg
Used gzip to uncompress the resulting out.gz: gzip --uncompress out.gz
Opened the out file, then searched for status.atProvider.id and found:

    - fromFieldPath: status.atProvider.id
      policy:
        fromFieldPath: Required
      toFieldPath: status.gke.project
      transforms:
      - string:
          regexp:
            group: 1
            match: projects\/(.+)\/serviceAccounts\/.*
          type: Regexp
        type: string
      type: ToCompositeFieldPath

b. Verified that I can push xpkg to registry

up xpkg push xpkg.upbound.io/taylor/tt-test:v0.0.1 -f ezgi-platform-ref-gcp.xpkg
xpkg pushed to xpkg.upbound.io/taylor/tt-test:v0.0.1

c. Verified that I can pull that xpkg into a control plane

[kubedebug] / # cat c.yaml
apiVersion: pkg.crossplane.io/v1
kind: Configuration
metadata:
  name: platform-ref-gcp
spec:
  package: xpkg.upbound.io/taylor/tt-test:v0.0.1


[kubedebug] / # kubectl get providers
NAME                                     INSTALLED   HEALTHY   PACKAGE                                                          AGE
crossplane-contrib-provider-helm         True        True      xpkg.upbound.io/crossplane-contrib/provider-helm:v0.15.0         5m35s
upbound-provider-aws                     True        Unknown   xpkg.upbound.io/upbound/provider-aws:v0.36.0                     5m52s
upbound-provider-family-gcp              True        True      xpkg.upbound.io/upbound/provider-family-gcp:v0.33.1              5m11s
upbound-provider-gcp-cloudplatform       True        True      xpkg.upbound.io/upbound/provider-gcp-cloudplatform:v0.33.0       5m32s
upbound-provider-gcp-compute             True        True      xpkg.upbound.io/upbound/provider-gcp-compute:v0.33.0             5m29s
upbound-provider-gcp-container           True        True      xpkg.upbound.io/upbound/provider-gcp-container:v0.33.0           5m23s
upbound-provider-gcp-servicenetworking   True        True      xpkg.upbound.io/upbound/provider-gcp-servicenetworking:v0.33.0   5m19s
upbound-provider-gcp-sql                 True        True      xpkg.upbound.io/upbound/provider-gcp-sql:v0.33.0                 5m15s


[kubedebug] / # kubectl get providerrevisions
NAME                                                  HEALTHY   REVISION   IMAGE                                                            STATE    DEP-FOUND   DEP-INSTALLED   AGE
crossplane-contrib-provider-helm-503c3591121b         True      1          xpkg.upbound.io/crossplane-contrib/provider-helm:v0.15.0         Active                               5m30s
upbound-provider-aws-3f7547b28ca6                               1          xpkg.upbound.io/upbound/provider-aws:v0.36.0                     Active                               5m48s
upbound-provider-family-gcp-b0fb8e3d4a7a              True      1          xpkg.upbound.io/upbound/provider-family-gcp:v0.33.1              Active                               5m5s
upbound-provider-gcp-cloudplatform-78133325933e       True      1          xpkg.upbound.io/upbound/provider-gcp-cloudplatform:v0.33.0       Active   1           1               5m27s
upbound-provider-gcp-compute-308b9eb8c1d3             True      1          xpkg.upbound.io/upbound/provider-gcp-compute:v0.33.0             Active   1           1               5m24s
upbound-provider-gcp-container-fbc4b73cd0e7           True      1          xpkg.upbound.io/upbound/provider-gcp-container:v0.33.0           Active   1           1               5m18s
upbound-provider-gcp-servicenetworking-3b85e9b62814   True      1          xpkg.upbound.io/upbound/provider-gcp-servicenetworking:v0.33.0   Active   1           1               5m14s
upbound-provider-gcp-sql-d88e7407bac4                 True      1          xpkg.upbound.io/upbound/provider-gcp-sql:v0.33.0                 Active   1           1               5m10s

Signed-off-by: Taylor Thornton <thornton.tn@gmail.com>

- update vendored packages to meet new functionality from upstream Signed-off-by: Taylor Thornton <thornton.tn@gmail.com>

Signed-off-by: Taylor Thornton <thornton.tn@gmail.com>

AlainRoy · 2023-06-23T22:39:38Z

cmd/up/main.go

+	// We don't need this choice to be cryptographically random.
+	rand.New(rand.NewSource(time.Now().UTC().UnixNano())) // nolint:gosec


Just because I'm curious: If we don't need it to be cryptographically strong, why do we need random? What do we use it for?

Funny enough, in searching for the answer to your question it looks like it was being used for some now dead code: c4c3627. I just removed this line to remove the confusion.

Signed-off-by: Taylor Thornton <thornton.tn@gmail.com>

AlainRoy · 2023-06-23T22:43:36Z

go.mod

-	// The following replacements should be kept in sync with upstream
-	// Kubernetes version.
-	// xref: https://github.com/kubernetes/kubernetes/blob/4ce5a8954017644c5420bae81d72b09b735c21f0/go.mod#L394


Yay! I'm happy to see these replacements removed.

Signed-off-by: Taylor Thornton <thornton.tn@gmail.com>

AlainRoy · 2023-06-23T22:45:37Z

go.mod

+	// https://github.com/helm/helm/issues/11821
+	github.com/docker/cli => github.com/docker/cli v20.10.19+incompatible
+	github.com/docker/docker => github.com/docker/docker v20.10.19+incompatible


Ewww.
I know it's not your fault and it looks like we have to do this, but still. Eww.

AlainRoy · 2023-06-23T22:48:25Z

LGTM! I'll wait to approve until you test, but it looks reasonable.

My only concern is about what happens for new releases of Crossplane. Will it be a lot of work to update the vendored code? Would it help to document anything about the process you followed to do the update, or is just something that we need to work through?

tnthornton · 2023-06-23T22:51:14Z

My only concern is about what happens for new releases of Crossplane. Will it be a lot of work to update the vendored code? Would it help to document anything about the process you followed to do the update, or is just something that we need to work through?

It's incredibly unclear what the LOE is to update the next time around. Regarding documentation, I don't really know what to document:

I updated a dependency.
Fixed compilation errors.
Copied code over and tore out things that "we didn't need". Still TBD if that plan worked.

AlainRoy · 2023-06-24T00:40:19Z

My only concern is about what happens for new releases of Crossplane. Will it be a lot of work to update the vendored code? Would it help to document anything about the process you followed to do the update, or is just something that we need to work through?

It's incredibly unclear what the LOE is to update the next time around. Regarding documentation, I don't really know what to document:
* I updated a dependency.

* Fixed compilation errors.

* Copied code over and tore out things that "we didn't need". Still TBD if that plan worked.

OK, that's cool. No need to do anything to document it.

AlainRoy · 2023-06-24T01:22:24Z

Should we do a new release of up in the near future?

tnthornton · 2023-06-24T01:37:53Z

Should we do a new release of up in the near future

That makes sense to me. Maybe sometime next week so that we can at the very least get fixes for #329 out.

jastang

Thanks @tnthornton ! Did a few more tests with xpkg xp-extract with a local build and looks good 👍🏼

@phisco did you want this backported?

phisco · 2023-06-26T13:34:57Z

@jastang I don't think we support multiple releases, looking at https://github.com/upbound/up/tags it looks like we are always going forward with versions, so nothing to backport to

tnthornton added 6 commits June 22, 2023 20:02

update deps across the board. get make build to pass

76b1f9a

Signed-off-by: Taylor Thornton <thornton.tn@gmail.com>

update crossplane-runtime, make build and make test pass

7efaf1f

Signed-off-by: Taylor Thornton <thornton.tn@gmail.com>

upgrade to crossplane v1.10.0

c6d0508

Signed-off-by: Taylor Thornton <thornton.tn@gmail.com>

update to crossplane v1.11.0

fdb1e76

- update vendored packages to meet new functionality from upstream Signed-off-by: Taylor Thornton <thornton.tn@gmail.com>

update to crossplane 1.12.0

1257869

Signed-off-by: Taylor Thornton <thornton.tn@gmail.com>

fix some new lint errors from going to 1.20

b1a8259

Signed-off-by: Taylor Thornton <thornton.tn@gmail.com>

tnthornton requested review from AlainRoy, jastang and phisco June 23, 2023 22:34

AlainRoy reviewed Jun 23, 2023

View reviewed changes

bump ci.yaml go version to 1.20

ed10f94

Signed-off-by: Taylor Thornton <thornton.tn@gmail.com>

AlainRoy reviewed Jun 23, 2023

View reviewed changes

remove dead code

6bceab2

Signed-off-by: Taylor Thornton <thornton.tn@gmail.com>

AlainRoy reviewed Jun 23, 2023

View reviewed changes

tnthornton marked this pull request as ready for review June 24, 2023 00:54

tnthornton changed the title ~~[WIP]: Updates dependencies to get us pretty close to current~~ Updates dependencies to get us pretty close to current Jun 24, 2023

AlainRoy approved these changes Jun 24, 2023

View reviewed changes

jastang approved these changes Jun 26, 2023

View reviewed changes

phisco approved these changes Jun 26, 2023

View reviewed changes

tnthornton merged commit e1e24a6 into upbound:main Jun 26, 2023
6 checks passed

tnthornton deleted the update-deps branch June 26, 2023 16:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updates dependencies to get us pretty close to current #336

Updates dependencies to get us pretty close to current #336

tnthornton commented Jun 23, 2023 •

edited

AlainRoy Jun 23, 2023

tnthornton Jun 23, 2023

AlainRoy Jun 23, 2023

AlainRoy Jun 23, 2023

AlainRoy commented Jun 23, 2023

tnthornton commented Jun 23, 2023

AlainRoy commented Jun 24, 2023

AlainRoy commented Jun 24, 2023

tnthornton commented Jun 24, 2023

jastang left a comment

phisco commented Jun 26, 2023

		// We don't need this choice to be cryptographically random.
		rand.New(rand.NewSource(time.Now().UTC().UnixNano())) // nolint:gosec

Updates dependencies to get us pretty close to current #336

Updates dependencies to get us pretty close to current #336

Conversation

tnthornton commented Jun 23, 2023 • edited

Description of your changes

How has this code been tested

Testing xpls

Testing xpkg commands

AlainRoy Jun 23, 2023

Choose a reason for hiding this comment

tnthornton Jun 23, 2023

Choose a reason for hiding this comment

AlainRoy Jun 23, 2023

Choose a reason for hiding this comment

AlainRoy Jun 23, 2023

Choose a reason for hiding this comment

AlainRoy commented Jun 23, 2023

tnthornton commented Jun 23, 2023

AlainRoy commented Jun 24, 2023

AlainRoy commented Jun 24, 2023

tnthornton commented Jun 24, 2023

jastang left a comment

Choose a reason for hiding this comment

phisco commented Jun 26, 2023

tnthornton commented Jun 23, 2023 •

edited