Use custom jammy paketo builder #1911

matejvasek · 2023-08-07T16:27:25Z

Changes

Use custom jammy paketo builder. The builders are modified so they include custom Go and Rust buildpacks. This enables on cluster builds for Go and Rust functions. Where possible (Go, Quarkus) the "tiny" builder is used, other runtimes use "base" builder.

Since jammy builder uses different uid/gid updated task is required, but for sake of compatibility this is done by creating new task file instead of modifying the existing tekton task yaml. Please see

diff pkg/pipelines/resources/tekton/task/func-buildpacks/0.1/func-buildpacks.yaml \ 
     pkg/pipelines/resources/tekton/task/func-buildpacks/0.2/func-buildpacks.yaml

knative-prow · 2023-08-07T16:27:27Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

codecov · 2023-08-07T16:35:47Z

Codecov Report

Patch coverage: 80.00% and project coverage change: +0.95% 🎉

Comparison is base (89cf6b7) 61.82% compared to head (2603e6a) 62.78%.
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1911      +/-   ##
==========================================
+ Coverage   61.82%   62.78%   +0.95%     
==========================================
  Files         107      107              
  Lines       13831    13813      -18     
==========================================
+ Hits         8551     8672     +121     
+ Misses       4423     4262     -161     
- Partials      857      879      +22

Flag	Coverage Δ
e2e-test	`35.50% <0.00%> (+0.04%)`	⬆️
e2e-test-oncluster	`30.74% <40.00%> (+0.02%)`	⬆️
e2e-test-oncluster-runtime	`25.65% <30.00%> (?)`
e2e-test-runtime-go	`25.93% <0.00%> (?)`
e2e-test-runtime-node	`26.93% <0.00%> (?)`
e2e-test-runtime-python	`26.93% <0.00%> (?)`
e2e-test-runtime-quarkus	`27.05% <0.00%> (?)`
e2e-test-runtime-rust	`25.93% <0.00%> (?)`
e2e-test-runtime-springboot	`26.07% <0.00%> (?)`
e2e-test-runtime-typescript	`27.05% <0.00%> (?)`
integration-tests	`51.47% <40.00%> (+2.22%)`	⬆️
unit-tests-macos-latest	`48.28% <60.00%> (+0.04%)`	⬆️
unit-tests-ubuntu-latest	`49.07% <60.00%> (+0.02%)`	⬆️
unit-tests-windows-latest	`48.32% <60.00%> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
pkg/builders/buildpacks/builder.go	`65.73% <ø> (ø)`
pkg/pipelines/tekton/pipelines_pac_provider.go	`43.20% <0.00%> (+37.78%)`	⬆️
pkg/pipelines/tekton/pipelines_provider.go	`57.54% <0.00%> (+2.56%)`	⬆️
pkg/pipelines/tekton/templates.go	`70.74% <ø> (+1.36%)`	⬆️
pkg/k8s/security_context.go	`89.28% <100.00%> (ø)`
pkg/pipelines/tekton/validate.go	`88.23% <100.00%> (+10.45%)`	⬆️

... and 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

matejvasek · 2023-08-09T00:31:39Z

Note this also contain commit from #1914 .

matejvasek · 2023-08-09T00:52:45Z

PTAL @lkingland @jrangelramos @Shashankft9

lkingland

/hold for tests

matejvasek · 2023-08-09T01:12:53Z

@Shashankft9 this should enable build of Go and Rust on cluster.

matejvasek · 2023-08-09T17:26:18Z

/unhold

matejvasek · 2023-08-09T17:43:23Z

/retest

jrangelramos · 2023-08-09T18:11:30Z

/lgtm

pkg/pipelines/tekton/validate.go

matejvasek · 2023-08-09T20:58:08Z

/retest-required

Use our own modified jammy builder with additional buildpacks for GoFunc and Rust. This enables on cluster build for Go and Rust functions. Where possible (Go, Java) we use "tiny" variant, other runtimes use "base" variant. The updated task is new file instead of modifying existing task this is done for sake of keeping compatiblility. Signed-off-by: Matej Vasek <mvasek@redhat.com>

Signed-off-by: Matej Vasek <mvasek@redhat.com>

matejvasek · 2023-08-09T21:09:26Z

PTAL @lkingland @jrangelramos @Shashankft9

Signed-off-by: Matej Vasek <mvasek@redhat.com>

knative-prow · 2023-08-09T21:14:40Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lkingland, matejvasek, Shashankft9

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [lkingland,matejvasek]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Signed-off-by: Matej Vasek <mvasek@redhat.com>

knative-prow · 2023-08-09T21:58:03Z

New changes are detected. LGTM label has been removed.

matejvasek · 2023-08-10T03:22:31Z

@lkingland please re-lgtm

* chore: use custom jammy paketo builder Use our own modified jammy builder with additional buildpacks for GoFunc and Rust. This enables on cluster build for Go and Rust functions. Where possible (Go, Java) we use "tiny" variant, other runtimes use "base" variant. The updated task is new file instead of modifying existing task this is done for sake of keeping compatiblility. Signed-off-by: Matej Vasek <mvasek@redhat.com> * fixup: remove unnecessary code per review request Signed-off-by: Matej Vasek <mvasek@redhat.com> * fixup Signed-off-by: Matej Vasek <mvasek@redhat.com> * fixup: podman test refers correct tkn task yamls Signed-off-by: Matej Vasek <mvasek@redhat.com> --------- Signed-off-by: Matej Vasek <mvasek@redhat.com>

* chore: use tkn tasks from PR branch in CI (#1914) Signed-off-by: Matej Vasek <mvasek@redhat.com> * chore: released binaries refer correct task yamls (#1916) Signed-off-by: Matej Vasek <mvasek@redhat.com> * chore: refer correct tkn yaml in prow test (#1918) Without this change prow test will refer tkn yamls from the main branch not from the PR head branch. Signed-off-by: Matej Vasek <mvasek@redhat.com> * chore: update release generation (#1924) * Minimize release binary size. * Release latest version of buildpack tekton task. Signed-off-by: Matej Vasek <mvasek@redhat.com> * test: update github ref used on e2e oncluster tests (#1917) * test: Split of GH oncluster tests by builder. Added FUNC_BUILDER env var for e2e oncluter tests (#1963) * Use our own s2i image (#1971) Our image is much more recent and it is multiarch. Signed-off-by: Matej Vasek <mvasek@redhat.com> * chore: update buildah image ref (#1960) Signed-off-by: Matej Vasek <mvasek@redhat.com> * chore: update CA certs (#1944) Signed-off-by: Matej Vasek <mvasek@redhat.com> * chore: update Quarkus platform version to 3.4.1 (#1989) Co-authored-by: Knative Automation <automation@knative.team> Signed-off-by: Matej Vasek <mvasek@redhat.com> * chore: update mvn wrapper in Quarkus template (#1987) Signed-off-by: Matej Vasek <mvasek@redhat.com> * chore: use ./mvnw not mvn in tests (#1988) Signed-off-by: Matej Vasek <mvasek@redhat.com> * chore: update Springboot platform version Signed-off-by: Matej Vasek <mvasek@redhat.com> * fix: docker registry/repository parsing (#1929) * fix: docker registry/repository parsing Use go-containerregistry to do parsing. Signed-off-by: Matej Vasek <mvasek@redhat.com> * fix: use kebab-case instead of camelCase Signed-off-by: Matej Vasek <mvasek@redhat.com> * fix: use kebab-case instead of camelCase Signed-off-by: Matej Vasek <mvasek@redhat.com> --------- Signed-off-by: Matej Vasek <mvasek@redhat.com> * Fix failing concurrent test on Windows (#1890) * src: better debugging Signed-off-by: Matej Vasek <mvasek@redhat.com> * fix: wait for both builds Signed-off-by: Matej Vasek <mvasek@redhat.com> * fixup Signed-off-by: Matej Vasek <mvasek@redhat.com> * fixup Signed-off-by: Matej Vasek <mvasek@redhat.com> * fixup Signed-off-by: Matej Vasek <mvasek@redhat.com> * fix: detection of process liveness on Windows Signed-off-by: Matej Vasek <mvasek@redhat.com> * fix: make symlink relative Signed-off-by: Matej Vasek <mvasek@redhat.com> * fixup: cleanup Signed-off-by: Matej Vasek <mvasek@redhat.com> --------- Signed-off-by: Matej Vasek <mvasek@redhat.com> * allowing on cluster build for go runtime (#1445) * allowing on cluster build for go runtime * warning message added for go and rust builder * gofmt * fixups Signed-off-by: Matej Vasek <mvasek@redhat.com> --------- Signed-off-by: Matej Vasek <mvasek@redhat.com> Co-authored-by: Matej Vasek <mvasek@redhat.com> * Use custom jammy paketo builder (#1911) * chore: use custom jammy paketo builder Use our own modified jammy builder with additional buildpacks for GoFunc and Rust. This enables on cluster build for Go and Rust functions. Where possible (Go, Java) we use "tiny" variant, other runtimes use "base" variant. The updated task is new file instead of modifying existing task this is done for sake of keeping compatiblility. Signed-off-by: Matej Vasek <mvasek@redhat.com> * fixup: remove unnecessary code per review request Signed-off-by: Matej Vasek <mvasek@redhat.com> * fixup Signed-off-by: Matej Vasek <mvasek@redhat.com> * fixup: podman test refers correct tkn task yamls Signed-off-by: Matej Vasek <mvasek@redhat.com> --------- Signed-off-by: Matej Vasek <mvasek@redhat.com> * doc: Go and Rust on cluster build is supported (#1923) * doc: Go and Rust on cluster build is supported Signed-off-by: Matej Vasek <mvasek@redhat.com> * doc: build envvars Signed-off-by: Matej Vasek <mvasek@redhat.com> --------- Signed-off-by: Matej Vasek <mvasek@redhat.com> * fix: direct upload ppc64le, x390x (#1958) Signed-off-by: Matej Vasek <mvasek@redhat.com> * fix: report correct error when task doesn't exist (#1915) Signed-off-by: Matej Vasek <mvasek@redhat.com> * feat: tekton task urls in the env sub-cmd output (#1925) Signed-off-by: Matej Vasek <mvasek@redhat.com> --------- Signed-off-by: Matej Vasek <mvasek@redhat.com> Co-authored-by: Jefferson Ramos <jeramos@redhat.com> Co-authored-by: Knative Automation <automation@knative.team> Co-authored-by: Shashank Sharma <48708039+Shashankft9@users.noreply.github.com>

knative-prow bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Aug 7, 2023

knative-prow bot requested review from dsimansk and rhuss August 7, 2023 16:27

knative-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 7, 2023

matejvasek force-pushed the test-builder-img branch 2 times, most recently from 1256907 to ccc6886 Compare August 8, 2023 23:52

matejvasek requested review from salaboy, aslom, jrangelramos, Shashankft9, gauron99 and lkingland and removed request for rhuss and dsimansk August 8, 2023 23:53

matejvasek changed the title ~~[DO NOT MERGE] testing jammy builder~~ Use custom jammy paketo builder Aug 8, 2023

matejvasek marked this pull request as ready for review August 9, 2023 00:06

knative-prow bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 9, 2023

knative-prow bot requested a review from maximilien August 9, 2023 00:06

matejvasek removed the request for review from maximilien August 9, 2023 00:09

matejvasek force-pushed the test-builder-img branch from ccc6886 to b2f9dc5 Compare August 9, 2023 00:51

lkingland approved these changes Aug 9, 2023

View reviewed changes

knative-prow bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 9, 2023

knative-prow bot assigned jrangelramos Aug 9, 2023

Shashankft9 reviewed Aug 9, 2023

View reviewed changes

pkg/pipelines/tekton/validate.go Outdated Show resolved Hide resolved

matejvasek force-pushed the test-builder-img branch from b2f9dc5 to 755fa2e Compare August 9, 2023 21:02

knative-prow bot removed the lgtm Indicates that a PR is ready to be merged. label Aug 9, 2023

fixup: remove unnecessary code per review request

e1c9f49

Signed-off-by: Matej Vasek <mvasek@redhat.com>

matejvasek requested review from lkingland and Shashankft9 August 9, 2023 21:09

fixup

0e23489

Signed-off-by: Matej Vasek <mvasek@redhat.com>

Shashankft9 approved these changes Aug 9, 2023

View reviewed changes

knative-prow bot assigned Shashankft9 Aug 9, 2023

knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label Aug 9, 2023

fixup: podman test refers correct tkn task yamls

2603e6a

Signed-off-by: Matej Vasek <mvasek@redhat.com>

knative-prow bot removed the lgtm Indicates that a PR is ready to be merged. label Aug 9, 2023

lkingland added the lgtm Indicates that a PR is ready to be merged. label Aug 10, 2023

knative-prow bot merged commit efb04be into knative:main Aug 10, 2023
41 checks passed

wladimiiir mentioned this pull request Mar 19, 2024

Cannot write into node_modules directory #2237

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use custom jammy paketo builder #1911

Use custom jammy paketo builder #1911

matejvasek commented Aug 7, 2023 •

edited

knative-prow bot commented Aug 7, 2023

codecov bot commented Aug 7, 2023 •

edited

matejvasek commented Aug 9, 2023

matejvasek commented Aug 9, 2023

lkingland left a comment

matejvasek commented Aug 9, 2023

matejvasek commented Aug 9, 2023

matejvasek commented Aug 9, 2023

jrangelramos commented Aug 9, 2023

matejvasek commented Aug 9, 2023

matejvasek commented Aug 9, 2023

knative-prow bot commented Aug 9, 2023

knative-prow bot commented Aug 9, 2023

matejvasek commented Aug 10, 2023

Use custom jammy paketo builder #1911

Use custom jammy paketo builder #1911

Conversation

matejvasek commented Aug 7, 2023 • edited

Changes

knative-prow bot commented Aug 7, 2023

codecov bot commented Aug 7, 2023 • edited

Codecov Report

matejvasek commented Aug 9, 2023

matejvasek commented Aug 9, 2023

lkingland left a comment

Choose a reason for hiding this comment

matejvasek commented Aug 9, 2023

matejvasek commented Aug 9, 2023

matejvasek commented Aug 9, 2023

jrangelramos commented Aug 9, 2023

matejvasek commented Aug 9, 2023

matejvasek commented Aug 9, 2023

knative-prow bot commented Aug 9, 2023

knative-prow bot commented Aug 9, 2023

matejvasek commented Aug 10, 2023

matejvasek commented Aug 7, 2023 •

edited

codecov bot commented Aug 7, 2023 •

edited