Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

YJIT: Add compilation time counter #8417

Merged
merged 2 commits into from Sep 12, 2023
Merged

Conversation

k0kubun
Copy link
Member

@k0kubun k0kubun commented Sep 12, 2023

This PR adds a counter of cumulative time taken for compiling JIT code. Similar to GC.stat(:time), if you call this before and after your request, you can measure how long you spent in YJIT compilation for each request.

When we compare multiple Ruby versions, we could monitor GC.stat(:time) and this to check how much impact each of GC and YJIT had on their difference in performance.

Example

benchmark compile_time_ms
30k_ifelse 1,234ms
30k_methods 429ms
railsbench 621ms
lobsters 1,699ms

@matzbot matzbot requested a review from a team September 12, 2023 00:52
yjit/src/stats.rs Outdated Show resolved Hide resolved
@maximecb
Copy link
Contributor

Once the switch to Instant is made, could you also measure if there is any impact on total run time for the first iteration of e.g. 30k_ifelse. I would expect not, but just want to validate that this doesn't slow down benchmarks due to context switches. Always good to double-check.

@k0kubun
Copy link
Member Author

k0kubun commented Sep 12, 2023

could you also measure if there is any impact on total run time for the first iteration of e.g. 30k_ifelse

I ran one iteration of the following benchmarks:

--yjit

before: ruby 3.3.0dev (2023-09-11T17:09:22Z master 1ae5dd6f32) +YJIT [x86_64-linux]
after: ruby 3.3.0dev (2023-09-12T16:02:16Z yjit-compile-time 42b9e18d96) +YJIT [x86_64-linux]

-----------  -----------  ----------  ----------  ----------  -------------  ------------
bench        before (ms)  stddev (%)  after (ms)  stddev (%)  after 1st itr  before/after
lobsters     1738.2       0.0         1722.0      0.0         1.01           1.01
railsbench   1498.7       0.0         1493.5      0.0         1.00           1.00
30k_ifelse   1345.2       0.0         1327.1      0.0         1.01           1.01
-----------  -----------  ----------  ----------  ----------  -------------  ------------

--yjit-stats

before: ruby 3.3.0dev (2023-09-11T17:09:22Z master 1ae5dd6f32) +YJIT [x86_64-linux]
after: ruby 3.3.0dev (2023-09-12T16:02:16Z yjit-compile-time 42b9e18d96) +YJIT [x86_64-linux]

----------  -----------  ----------  ----------  ----------  -------------  ------------
bench       before (ms)  stddev (%)  after (ms)  stddev (%)  after 1st itr  before/after
lobsters    1918.4       0.0         1909.9      0.0         1.00           1.00
railsbench  1606.6       0.0         1603.7      0.0         1.00           1.00
30k_ifelse  1725.2       0.0         1705.3      0.0         1.01           1.01
----------  -----------  ----------  ----------  ----------  -------------  ------------

This PR doesn't seem to impact the 1st itr performance whether this feature is enabled or not.

Copy link
Contributor

@maximecb maximecb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice 👌

@maximecb maximecb merged commit 0ae7f2d into ruby:master Sep 12, 2023
94 checks passed
@k0kubun k0kubun deleted the yjit-compile-time branch September 12, 2023 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
4 participants