Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ci deploy #5328

Merged
merged 20 commits into from Sep 26, 2018
Merged

Fix ci deploy #5328

merged 20 commits into from Sep 26, 2018

Conversation

@headius
Copy link
Member

@headius headius commented Sep 26, 2018

Numeric changes here to improve the overall time of our CI builds.

  • Using -j for MRI suites to parallelize tests.
  • Moved MRI test_syntax to a separate run, since it appears to be the slowest test file.
  • Eliminated pre-build maven install; it does not appear to be necessary anymore.
  • Move deploy of snapshot to a post-test build stage so it does not fire for any of the test builds.
  • Reduce git clone to depth 1.

Some changes were necessary but unrelated to the work above:

  • Removed allow_failures group of tests because of difficulties getting them to match with the new "jobs" matrix.

With these changes in place plus @enebo's tweaks to use --dev, the most recent build on this branch took 22 minutes to complete, versus 45 minute for a build before any of this work began. The most recent master build, with @enebo's changes, took 25 minutes.

There's still room to improve.

  • More tests could be moved to the "syntax" MRI suite, since that is still shorter than the long version.
  • Running "int" and "fullint" for MRI may be excessive. JIT runs off the same IR as "fullint", even if it does not test the interpreter, and "int" will progress stdlib methods to "fullint" as a result of lazy IR optimization.
  • Obviously any improvement to startup time would continue to reduce the longest test runs. Many tiny scripts are launched in test_syntax, for example.
headius added 16 commits Sep 25, 2018
* Remove probably-unnecessary install step.
* Tweak IRC notification so it might not be blocked.
Specifying openjdk9 appears to break the fetching of some gem
resources in the build, perhaps because the openjdk9 build has
crippled crypto or stale certificates. The error looks like this:

```
[ERROR] Failed to execute goal on project jruby-stdlib: Could not resolve dependencies for project org.jruby:jruby-stdlib:jar:9.2.1.0-SNAPSHOT: Failed to collect dependencies at rubygems:cmath💎1.0.0: Failed to read artifact descriptor for rubygems:cmath💎1.0.0: Could not transfer artifact rubygems:cmath:pom:1.0.0 from/to mavengems (mavengem:https://rubygems.org): Error transferring file: RubygemsFile[type=POM, storage=/maven/releases/rubygems/cmath/1.0.0/cmath-1.0.0.pom, remote=/maven/releases/rubygems/cmath/1.0.0/cmath-1.0.0.pom, name=cmath, state=ERROR, exception=IOException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target] on https://rubygems.org -> [Help 1]
```

This commit returns to openjdk9 in an attempt to fix this error.
* Move slowest tests to top, so they'll run while others do.
* Reduce git depth to 1.
The test_syntax test from MRI runs a number of tests that launch
lots of subprocesses, which is still rather slow under JRuby. This
commit moves the entire script to its own build so it can run in
parallel with other suites and not hold up test:mri builds as
much.
@headius headius requested review from enebo and kares Sep 26, 2018
@headius
Copy link
Member Author

@headius headius commented Sep 26, 2018

Additional changes: reordered test suites so the longer ones come at the top; they will start earlier and run while the shorter suites complete. This does mean fewer short suites will run in parallel but given that the MRI suites are nearly as long as the entire CI run, I think the new orientation is better.

I reverted a last-minute bump from j2 to j3 for the remaining core MRI tests. It appeared to slow things down.

@headius
Copy link
Member Author

@headius headius commented Sep 26, 2018

We could also just remove the deploy stage, which would cut another couple minutes off the build. We do not link to the sonatype snapshot repos from anywhere, and have never officially used these builds as our snapshots. The CI builds run periodically on Cloudbees serve as our snapshots at present.

@mkristian I don't think anything in our builds is using the snapshot deploys either, right?

@mkristian
Copy link
Member

@mkristian mkristian commented Sep 26, 2018

@headius install is all the build needs !

@kares
Copy link
Member

@kares kares commented Sep 26, 2018

LGTM

@kares
kares approved these changes Sep 26, 2018
@headius
Copy link
Member Author

@headius headius commented Sep 26, 2018

spec:regression remains unstable but I will examine that separately. This PR looks ok.

@headius headius merged commit cd9d179 into master Sep 26, 2018
1 of 2 checks passed
1 of 2 checks passed
continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@headius headius deleted the fix_ci_deploy branch Sep 26, 2018
@headius headius added this to the JRuby 9.2.1.0 milestone Sep 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants