Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ci deploy #5328

Merged
merged 20 commits into from Sep 26, 2018

Conversation

Projects
None yet
3 participants
@headius
Copy link
Member

headius commented Sep 26, 2018

Numeric changes here to improve the overall time of our CI builds.

  • Using -j for MRI suites to parallelize tests.
  • Moved MRI test_syntax to a separate run, since it appears to be the slowest test file.
  • Eliminated pre-build maven install; it does not appear to be necessary anymore.
  • Move deploy of snapshot to a post-test build stage so it does not fire for any of the test builds.
  • Reduce git clone to depth 1.

Some changes were necessary but unrelated to the work above:

  • Removed allow_failures group of tests because of difficulties getting them to match with the new "jobs" matrix.

With these changes in place plus @enebo's tweaks to use --dev, the most recent build on this branch took 22 minutes to complete, versus 45 minute for a build before any of this work began. The most recent master build, with @enebo's changes, took 25 minutes.

There's still room to improve.

  • More tests could be moved to the "syntax" MRI suite, since that is still shorter than the long version.
  • Running "int" and "fullint" for MRI may be excessive. JIT runs off the same IR as "fullint", even if it does not test the interpreter, and "int" will progress stdlib methods to "fullint" as a result of lazy IR optimization.
  • Obviously any improvement to startup time would continue to reduce the longest test runs. Many tiny scripts are launched in test_syntax, for example.

headius added some commits Sep 25, 2018

More tweaks to reduce build time.
* Remove probably-unnecessary install step.
* Tweak IRC notification so it might not be blocked.
Use oraclejdk9 for SSL support.
Specifying openjdk9 appears to break the fetching of some gem
resources in the build, perhaps because the openjdk9 build has
crippled crypto or stale certificates. The error looks like this:

```
[ERROR] Failed to execute goal on project jruby-stdlib: Could not resolve dependencies for project org.jruby:jruby-stdlib:jar:9.2.1.0-SNAPSHOT: Failed to collect dependencies at rubygems:cmath💎1.0.0: Failed to read artifact descriptor for rubygems:cmath💎1.0.0: Could not transfer artifact rubygems:cmath:pom:1.0.0 from/to mavengems (mavengem:https://rubygems.org): Error transferring file: RubygemsFile[type=POM, storage=/maven/releases/rubygems/cmath/1.0.0/cmath-1.0.0.pom, remote=/maven/releases/rubygems/cmath/1.0.0/cmath-1.0.0.pom, name=cmath, state=ERROR, exception=IOException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target] on https://rubygems.org -> [Help 1]
```

This commit returns to openjdk9 in an attempt to fix this error.
More tweaks to speed CI.
* Move slowest tests to top, so they'll run while others do.
* Reduce git depth to 1.
Move MRI test_syntax to its own run.
The test_syntax test from MRI runs a number of tests that launch
lots of subprocesses, which is still rather slow under JRuby. This
commit moves the entire script to its own build so it can run in
parallel with other suites and not hold up test:mri builds as
much.

@headius headius requested review from enebo and kares Sep 26, 2018

@headius

This comment has been minimized.

Copy link
Member Author

headius commented Sep 26, 2018

Additional changes: reordered test suites so the longer ones come at the top; they will start earlier and run while the shorter suites complete. This does mean fewer short suites will run in parallel but given that the MRI suites are nearly as long as the entire CI run, I think the new orientation is better.

I reverted a last-minute bump from j2 to j3 for the remaining core MRI tests. It appeared to slow things down.

@headius

This comment has been minimized.

Copy link
Member Author

headius commented Sep 26, 2018

We could also just remove the deploy stage, which would cut another couple minutes off the build. We do not link to the sonatype snapshot repos from anywhere, and have never officially used these builds as our snapshots. The CI builds run periodically on Cloudbees serve as our snapshots at present.

@mkristian I don't think anything in our builds is using the snapshot deploys either, right?

@mkristian

This comment has been minimized.

Copy link
Member

mkristian commented Sep 26, 2018

@headius install is all the build needs !

@kares

This comment has been minimized.

Copy link
Member

kares commented Sep 26, 2018

LGTM

@kares

kares approved these changes Sep 26, 2018

headius added some commits Sep 26, 2018

@headius

This comment has been minimized.

Copy link
Member Author

headius commented Sep 26, 2018

spec:regression remains unstable but I will examine that separately. This PR looks ok.

@headius headius merged commit cd9d179 into master Sep 26, 2018

1 of 2 checks passed

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@headius headius deleted the fix_ci_deploy branch Sep 26, 2018

@headius headius added this to the JRuby 9.2.1.0 milestone Sep 26, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.