Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.cirrus.yml: Add Cirrus CI for Arm64 tests. #4875

Merged
merged 1 commit into from
Sep 22, 2021

Conversation

junaruga
Copy link
Member

@junaruga junaruga commented Sep 20, 2021

This PR is based on #4772 to enable Cirrus CI to test Arm cases. Initially there are 2 cases, the latest clang 12 and gcc 11 available on our CI image: http://ghcr.io/ruby/ruby-ci-image .

I would like to use the Cirrus Ci to cover some cases on Arm. We can add maximum 16 cases. The entire testing design is inspired from .github/workflows/compilers.yml .

Note

  • As I found an issue in rubygems tests with the environment variable make, I am using make_cmd for now, You can see rubygems tests failing with customized environment variable "make" rubygems/rubygems#4921 for detail.
  • I added the badge image to the README. The updated README is here.
  • This CI is also executed at the timing of push on forked repo like other CIs. It's good to test on your forked repo without sending PR.
  • The tool/disable_ipv6.sh is what we used in ruby/ruby in the past. We reused the script again.

@junaruga
Copy link
Member Author

The total running time is

So, it finishes faster than other CIs. That's good to add this CI.

My concern is the following part in make test-all. Some tests are retried. I don't know why the tests are retried while other CI results, such as the logs by .github/workflows/ubuntu.yml and .travis.yml don't print this kind of message.

https://cirrus-ci.com/task/5219912575614976

$make_cmd test-all
config.status: creating ruby-runner.h
making mjit_build_dir.so
Run options: 
  --seed=27671
  "--ruby=./miniruby -I./lib -I. -I.ext/common  ./tool/runruby.rb --extout=.ext  -- --disable-gems"
  --excludes-dir=./test/excludes
  --name=!/memory_leak/
  -q
  --color=always
  --tty=no
# Running tests:

Retrying...
[  1/127] TestDefined#test_autoloaded_noload = 0.00 s
[  2/127] TestDefined#test_autoloaded_subclass = 0.00 s
...
[126/127] TestKeywordArguments#test_zsuper_only_anonymous_kwrest = 0.00 s
[127/127] TestKeywordArguments#test_zsuper_only_named_kwrest = 0.00 s
Finished tests in 84.888216s, 247.7493 tests/s, 31488.2456 assertions/s.
21031 tests, 2672981 assertions, 0 failures, 0 errors, 190 skips
ruby -v: ruby 3.1.0dev (2021-09-20) [aarch64-linux]

tool/disable_ipv6.sh Outdated Show resolved Hide resolved
@junaruga junaruga marked this pull request as ready for review September 21, 2021 06:02
@junaruga
Copy link
Member Author

@shyouhei is it possible to review this PR, as it includes the new file .cirrus.yml? Thanks.

.cirrus.yml Outdated Show resolved Hide resolved
@junaruga
Copy link
Member Author

Rebased with using GNUMAkEFLAGS.

echo "GNUMAKEFLAGS=$GNUMAKEFLAGS"
GNUMAKEFLAGS=-s -j9

The total running time that I tested on my forked repo is
https://cirrus-ci.com/build/4609835590221824

  • clang-12: 05:01
  • gcc-11: 03:34
    It's exactly same time with previous one.

This PR's running time is like this. Maybe the start was queued and delayed.
https://cirrus-ci.com/build/5755328030572544

  • clang-12: 06:04
  • gcc-11: 04:08

As a reference, I also tested the total running time without concurrent jobs -jN by executing the make without setting the GNUMAKEFLAGS. The running time is much longer as expected.
https://cirrus-ci.com/build/5528881986797568

  • clang-12: 23:30
  • gcc-11: 16:49

.cirrus.yml Outdated Show resolved Hide resolved
Co-authored-by: fedor <fedor@cirruslabs.org>
@junaruga
Copy link
Member Author

junaruga commented Sep 21, 2021

Note the 1 failing by the failed JIT tests on Visual Studio 20219 on Windows is what randomly happens according to kokubun's research yesterday. https://github.com/ruby/ruby/pull/4875/checks?check_run_id=3664926540 . It's not related to this PR.

Copy link
Member

@shyouhei shyouhei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

@shyouhei
Copy link
Member

We might also want to fix .github/workflows/compilers.yml's use of $make variable in a separate pull request.

@junaruga junaruga merged commit dd78084 into ruby:master Sep 22, 2021
@junaruga junaruga deleted the wip/cirrus-ci-arm branch September 22, 2021 07:01
@junaruga
Copy link
Member Author

@shyouhei Thanks for the review! For the .github/workflows/compilers.yml to fix the $make, I sent the PR #4880 now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants