Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update travis matrix (add truffleruby, remove rubinius) #695

Merged
merged 4 commits into from May 22, 2020

Conversation

@iliabylich
Copy link
Collaborator

iliabylich commented May 22, 2020

No description provided.

@bbatsov
Copy link
Contributor

bbatsov commented May 22, 2020

Nice! I didn't even know that Rbx was still being developed.

@iliabylich
Copy link
Collaborator Author

iliabylich commented May 22, 2020

looks like rbx 5.0 has been released a week ago 😄 However, I wasn't able to find any announcements or articles about it (maybe they are still WIP)

@iliabylich iliabylich force-pushed the iliabylich:update-travis-matrix branch from 7a5c5df to 12eac91 May 22, 2020
@iliabylich iliabylich changed the title update travis matrix (add truffleruby, bump rubinius) update travis matrix (add truffleruby, remove rubinius) May 22, 2020
@iliabylich
Copy link
Collaborator Author

iliabylich commented May 22, 2020

@eregon It takes ~15 minutes to run a test suite (for MRI 2.7 it's only 30s). Is there any way to make it faster?

@eregon
Copy link

eregon commented May 22, 2020

I'll take a look.

@eregon
Copy link

eregon commented May 22, 2020

An easy gain that makes it about 2x faster is to not run with COVERAGE=1.
That can be done by swapping bundle exec rake test_cov with bundle exec rake test.
Locally, that reduce the time from 250s specs 5min28s total to 135s specs 2min24s total.

Coverage is usually not that expensive, but here there is a custom :line TracePoint and it seems TruffleRuby splits & compiles it many times which is really slow (something to fix).

BTW on MRI I also see that COVERAGE=1 is >2x slower (11.3s with, 3.5s without for specs), but of course it doesn't matter with those times.

@iliabylich iliabylich force-pushed the iliabylich:update-travis-matrix branch from d3e969a to e8276b8 May 22, 2020
@iliabylich iliabylich merged commit 521054b into whitequark:master May 22, 2020
1 check passed
1 check passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@iliabylich iliabylich deleted the iliabylich:update-travis-matrix branch May 22, 2020
@eregon
Copy link

eregon commented May 22, 2020

Something else that also speeds it up about 2x is to use the latency Mode, which improves warmup (i.e., JIT faster) by reducing splitting and inlining (at the cost of peak performance, but rarely relevant for tests).

TRUFFLERUBYOPT=--engine.Mode=latency bundle exec rake test

gives me locally 68s specs 1min17s total.

For the record,
initial TravisCI run: 617s specs https://travis-ci.org/github/whitequark/parser/jobs/690086457#L439
rake test: 290s specs https://travis-ci.org/github/whitequark/parser/jobs/690130316

@iliabylich
Copy link
Collaborator Author

iliabylich commented May 22, 2020

@eregon Thanks! I'll play with locally. Compiling parsers using racc is very slow atm. Also, does it make sense to use JIT at all? I mean, we run different codepaths on purpose, is it possible to completely disable runtime optimizations?

@eregon
Copy link

eregon commented May 22, 2020

Compiling parsers using racc is very slow atm.

It takes about 5s per file with TRUFFLERUBYOPT=--engine.Mode=latency and 10s without for me locally.

Also, does it make sense to use JIT at all? I mean, we run different codepaths on purpose, is it possible to completely disable runtime optimizations?

Yes, and in fact it's significantly slower with the JIT disabled. It's always useful to e.g. speculate that Integer#+ doesn't overflow, etc.

@iliabylich
Copy link
Collaborator Author

iliabylich commented May 22, 2020

Yes, I've got similar results locally (16s -> 8s). Also, I've noticed that using pure ruby version of racc saves extra second, but there's no clean way to enforce it. Thanks a lot, I'll add this option to CI config.

Side question, is it applicable to all pure/ext libraries like json/ext vs json/pure?

@eregon
Copy link

eregon commented May 22, 2020

Interesting, how did you force using the Ruby version of racc?

Side question, is it applicable to all pure/ext libraries like json/ext vs json/pure?

No, it depends. Sometimes the Ruby version is faster and sometimes the C extension is faster.

@iliabylich
Copy link
Collaborator Author

iliabylich commented May 22, 2020

well, there's no simple way to do it. Here - https://github.com/ruby/racc/blob/master/lib/racc/parser.rb#L197 - I added raise LoadError to pretend that there's no compiled cparse.[so|bundle|dll] and to come to the rescue branch that configures racc to use pure ruby version. At the end you should see the difference in the output of

$ racc --runtime-version
racc runtime version 1.4.15 (rev. $Id: 2bd697d1146b280d8d3878e21e556ef769484a37 $); c core version 1.4.15 (rev. 1.8)

# vs

$ racc --runtime-version
racc runtime version 1.4.15 (rev. $Id: 2bd697d1146b280d8d3878e21e556ef769484a37 $); ruby core version 1.4.15 (rev. 2bd697d1146b280d8d3878e21e556ef769484a37)

also, IIRC racc ... uses parsefilegenerator.rb, so 1s is spent to load C version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.