Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[release-0.8]: backports from master #1048

Merged
merged 36 commits into from Aug 23, 2023
Merged

Conversation

marquiz
Copy link
Contributor

@marquiz marquiz commented Aug 18, 2023

This PR cherry-picks and backports a big pile of patches from the master branch, all in preparation for the next patch release. The patches can be divided into two major categories:

  1. updates and fixes to the build/CI infra, enabling us to build container images and other release assets as GitHub workflows
  2. Fixes and updates to dependencies

The --container command like argument was skipped because of incorrect
shift in the script which makes sure that it's really not used.

(cherry picked from commit 365ed6b)
Speed up builds considerably by reducing the download times (the golang
tarball will be cached by docker) and skipping the build of golang
itself.

(cherry picked from commit 4576695)
When cross-building binary packages (rpm, deb, tarball), create a
separate packages/release-assets directory where all binary packages are
copied with a distintive distro-specific name. Make cutting releases way
easier by removing the need for manual renaming (because without the
distro-specific naming package names would clash). Also opens up the
possibility to automatic uploading of release assets.

(cherry picked from commit ddd266f)
This make target is basically just for CI. Decoupling image-push from
image makes it easier to separate the image build and publish steps,
without unnecessary extra runs of docker build.

(cherry picked from commit 9094faf)
Makes it possible to run dockerized targets in CI, for example.

(cherry picked from commit d7338a4)
We definitely don't want 'make' to be mangling our code tree.
Plus, this is useless as we take the golang version from go.mod
nowadays.

(cherry picked from commit 7898a9c)
(cherry picked from commit 086954b)
@marquiz marquiz changed the base branch from master to release-0.8 August 18, 2023 16:33
klihub
klihub previously approved these changes Aug 18, 2023
Fix the verify workflow to actually install the version of Go specified
in go.mod. Previously, because of the yaml parsing peculiarities go v1.2 was
installed.

Also changes the order of steps so that go.mod is available.

NOTE: This expects that the practice from now on is to keep the Go version
in go.mod and Makefile in sync.

(cherry picked from commit b5128e8)
Use github workflows' built-in concurrency instead of the 3rd party
plygin for managing Github pages update jobs (only one job allowed to be
running at the time).

(cherry picked from commit 16494e8)
Buildability of docs and gh-pages update are verified in the following
steps anyway. Dropping the containerized build of docs speeds up CI.
Refactor the docs-build into a re-usable job called from both the verify
and publish workflows.

(cherry picked from commit 207aa32)
Split the security scanning workflow into separate parameterized jobs.
This makes them re-usable for diiferent workflows, e.g. CI on PRs and
image-publishing.

Also drop unneeded and broken parts from the CodeQL scanning job.

(cherry picked from commit 40b9761)
Use Ubuntu 22.04 explicitly.

(cherry picked from commit 59ca57f)
Change the verify workflow to run on PRs only. Add concurrency rule so
that in-progress runs for a PR are canceled if the PR is updated before
their completion.

Add a nightly verify workflow that runs on the master branch every day
at 2:30 UTC.

(cherry picked from commit 300b402)
Add two new workflows, "Release" and "Publish devel images". The first
one is intended for building and publishing all release artefacts - we
now add only container images. The second one is for image-publishing
only from the master branch.

The patch also adds a common re-usable image building job.

(cherry picked from commit 5ecc089)
Fix the usage of github environments in the container image building.
Environment cannot be specifiec in the "caller" job of re-usable "on
workflow_call" workflows.

(cherry picked from commit d3a4187)
(cherry picked from commit 7dcd31a)
Be slightly smarter in what test jobs to run, e.g. don't build
documentation if nothing related changes.

Also, enable uploading of trivy scan results to the GitHub security tab.

(cherry picked from commit 0012a19)
marquiz and others added 10 commits August 18, 2023 20:22
Use path filter for only building/updating docs from the mainline and
release branches when something in the documentation actually changes.

(cherry picked from commit 6131005)
Speeds up subsequent builds as the tarball will be cached.

(cherry picked from commit 9cea559)
(cherry picked from commit b5f4528)
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
(cherry picked from commit 24ab5f7)
Go v1.20 supports wrapping multiple errors into one.

(cherry picked from commit cce97cb)
Instead of importing cpuset.CPUSet directly from the kubelet,
switch to using k8s.io/utils/cpuset. However, don't import it
directly all over the map. Instead wrap it in a single place
(pkg/utils/cpuset) and import it everywhere else from there.

Signed-off-by: Krisztian Litkey <krisztian.litkey@intel.com>
(cherry picked from commit 68a8ade)
Don't import kubernetes/kubelet for namespace and label
constants or milli-CPU conversion constants and functions.
Instead, define our own constants and functions for those
few things we need.

Signed-off-by: Krisztian Litkey <krisztian.litkey@intel.com>
(cherry picked from commit bf7c902)
(cherry picked from commit 8595859)
klihub
klihub previously approved these changes Aug 21, 2023
@askervin
Copy link
Contributor

Guys, hold this for a little while, I'm still debugging why some e2e tests failed on this branch...

"sleep" in the latest busybox does not create a child process named
"sleep". Change process matching from sleep to matching pod0c0
parent/child instead.

(cherry picked from commit 922c5a6)
kubelet command line parameter --container-runtime has been deprecated
and removed. --container-runtime-endpoint is enough.

(cherry picked from commit 8c4d39b)
Run tests in the latest Ubuntu LTS by default.

(cherry picked from commit f1e1d30)
Strict requirement makes the test flaky. Give two more rounds time for
detecting first memory moves.

(cherry picked from commit 6699d6b)
Found processes are allowed to exit during reading their memory
maps. This is not a fatal error, demotion can continue from the next
pid.

(cherry picked from commit ce9f696)
@codecov-commenter
Copy link

Codecov Report

Merging #1048 (24d0251) into release-0.8 (adb0945) will decrease coverage by 0.14%.
The diff coverage is 62.80%.

@@               Coverage Diff               @@
##           release-0.8    #1048      +/-   ##
===============================================
- Coverage        33.58%   33.44%   -0.14%     
===============================================
  Files               61       60       -1     
  Lines             9159     9140      -19     
===============================================
- Hits              3076     3057      -19     
+ Misses            5798     5797       -1     
- Partials           285      286       +1     
Files Changed Coverage Δ
pkg/cgroups/cgroupstats.go 0.00% <0.00%> (ø)
pkg/cri/resource-manager/cache/utils.go 23.20% <0.00%> (-11.50%) ⬇️
...i/resource-manager/control/page-migrate/demoter.go 17.95% <0.00%> (ø)
...manager/policy/builtin/balloons/balloons-policy.go 1.70% <0.00%> (ø)
...esource-manager/policy/builtin/balloons/metrics.go 0.00% <ø> (ø)
...esource-manager/policy/builtin/podpools/metrics.go 0.00% <0.00%> (ø)
...manager/policy/builtin/podpools/podpools-policy.go 23.78% <0.00%> (ø)
...urce-manager/policy/builtin/static-pools/config.go 5.88% <0.00%> (ø)
...rce-manager/policy/builtin/topology-aware/cache.go 41.22% <ø> (ø)
...urce-manager/policy/builtin/topology-aware/hint.go 80.76% <ø> (ø)
... and 11 more

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Contributor

@askervin askervin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

All e2e tests PASS.

@klihub klihub merged commit 246d9f1 into intel:release-0.8 Aug 23, 2023
7 checks passed
@marquiz marquiz deleted the release-0.8 branch August 23, 2023 06:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants