Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: test the shedulers with the latest sched-ext kernel #95

Merged
merged 1 commit into from
Jan 18, 2024
Merged

Conversation

arighi
Copy link
Collaborator

@arighi arighi commented Jan 18, 2024

Instead of downloading a precompiled sched-ext enabled kernel from the Ubuntu ppa, fetch the latest kernel directly from the sched-ext git repository and recompile it on-the-fly using virtme-ng.

This allows to get rid of the Ubuntu ppa dependency, take out from the equation potential Ubuntu-specific patches, and ensures testing all the schedulers with the most up-to-date sched-ext kernel (that should also help to detect potential kernel-related issues in advance).

The downside is that the CI runs will take a bit longer now, because we are recompiling the kernel from scratch. However, the kernel built with virtme-ng is relatively quick to compile and includes all the sched-ext features required for testing.

It's worth noting that this method aligns with the current sched-ext kernel CI, where we test only the in-kernel schedulers (as intended).

This change allows to extend the test coverage, using the same kernel to test also the schedulers included in this repository.

- run: mkdir -p kernel
- run: for f in *.deb; do dpkg -x $f kernel; done
# Get the latest sched-ext enabled kernel directly from the git repository
- run: git clone https://github.com/sched-ext/sched_ext.git linux
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dev branch is sched_ext. One worry is that when we make breaking changes in the kernel, this would immediately break CI for scx repo. Would it be possible to make it try sched_ext-ci first and then if that's not present try sched_ext so that we have a way to lag a bit when necessary?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we could have a release source tarball for CI that we get and untar here. That would also speed up CI as we wouldn't have to clone the entire repo.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we could just go with git clone --single-branch -b sched-ext --depth 1 and it would be like fetching a tarball.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...and git ls-remote or similar to check if sched-ext-ci or sched-ext exist.

@arighi arighi force-pushed the github-ci branch 3 times, most recently from 6a4e197 to 9e866c4 Compare January 18, 2024 19:48
Instead of downloading a precompiled sched-ext enabled kernel from the
Ubuntu ppa, fetch the latest kernel directly from the sched-ext git
repository and recompile it on-the-fly using virtme-ng.

This allows to get rid of the Ubuntu ppa dependency, take out from the
equation potential Ubuntu-specific patches, and ensures testing all the
schedulers with the most up-to-date sched-ext kernel (that should also
help to detect potential kernel-related issues in advance).

The downside is that the CI runs will take a bit longer now, because we
are recompiling the kernel from scratch. However, the kernel built with
virtme-ng is relatively quick to compile and includes all the sched-ext
features required for testing.

It's worth noting that this method aligns with the current sched-ext
kernel CI, where we test only the in-kernel schedulers (as intended).

This change allows to extend the test coverage, using the same kernel to
test also the schedulers included in this repository.

Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
Copy link
Contributor

@Byte-Lab Byte-Lab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic, thanks!!

@arighi arighi merged commit af11da2 into main Jan 18, 2024
2 checks passed
@Byte-Lab Byte-Lab deleted the github-ci branch March 14, 2024 18:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants