-
Notifications
You must be signed in to change notification settings - Fork 325
7903740: JMH: Perf event validation not working with skid options #132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
7903740: JMH: Perf event validation not working with skid options #132
Conversation
|
👋 Welcome back galder! A progress list of the required criteria for merging this PR into |
|
@galderz This change now passes all automated pre-integration checks. After integration, the commit message for the final commit will be: You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed. At the time when this comment was updated there had been 1 new commit pushed to the
Please see this link for an up-to-date comparison between the source branch of this pull request and the As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@shipilev) but any other Committer may sponsor as well. ➡️ To flag this PR as ready for integration with the above commit message, type |
Webrevs
|
|
Mailing list message from Ian Rogers on jmh-dev: An alternative to cycles:p is to use cycles:P (note the case) which Performance counter stats for 'true': 900,546 cycles:P 0.001261127 seconds time elapsed 0.001401000 seconds user Thanks, On Fri, May 31, 2024 at 10:53?AM Galder Zamarre?o <galder at openjdk.org> wrote: |
|
@galderz This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, so the point for initialization checks to fail early, before we actually get to executing benchmarks. What would happen when event with option is not supported by perf record? Suppose events is supported and events:ppp is not. We check events and it works, we proceed to using events:ppp at runtime and fail.
Since there is an apparent discrepancy between what perf record and perf stat accept, I think we need to validate with perf record then. Something easy, like perf record -e ... echo 1 would likely work well?
|
@shipilev Makes sense, I'll look into validating that. |
|
@shipilev Updated the PR with your proposed fix. |
|
There are GHA test failures, not sure if they are due to this fix, please check? |
|
Sure, checking... |
|
Hmmm, I think the error is caused because profiler supported check now passes when running in CI, whereas in the past this tests didn't attempt to run. Although you cannot see the With my change, it seems now the test thinks the profiler is supported, but then no secondary results are obtained. I'm unsure how to deal with this yet. |
|
@shipilev Are you sure these issues are also present in the main branch? I don't see any main branch CI runs and I don't see any recent CI jobs other than the one here. I wonder if something has changed in the way CI environments run with regards to perf. In the May run I pointed above linux perf asm tests where not really executing. That means that It would be useful to have a way to see if |
|
Actually my test shows the issue with switching to Since The Looking at the |
|
I'm playing around with this idea to see if it works: do a |
|
I think my idea works. On my linux box, for For Then on CI (see test) I see: |
|
Ok, let's see what CI thinks about this. It checks if the |
|
Hmmmm that doesn't fully work on CI :\ (see output here). Doing further investigations. |
|
Ian, I've pushed changes based on your feedback. I think that does the job very nicely. |
|
Wooo hoo, all tests passing, @shipilev can you have another look? |
|
Well, I just went to my It works without the patch. I think checking code is being too smart for its own good? Reiterating my previous suggestion: let's not try to parse the recordings, and instead "just" parse the output |
|
That is to say, I am willing to accept the simple patch like: ...and everything else needs a very hard justification and a lot of testing to make sure it does not break in the cases it is supposed to work. The fact the PR code fails on my first actual try does not inspire confidence, to be honest. Remember: the capabilities check in constructor is an opportunistic check, it can pass by accident, and users would discover their profiler does not really work later when looking at the results. The check should NOT fail by accident, when the profiler would actually work, if not for a misbehaving capability check. |
Makes sense and I agree with you, but my previous efforts on this PR show that the CI is getting in the way of what you said above. E.g. see what happened in https://github.com/galderz/jmh/actions/runs/10384414105/job/28751371192, where the check passes by accident but then the profiler contains no data, so the test fails. |
|
Closed the PR by mistake. |
I will give your suggestion a go and see if |
|
Just pushed this:
The time it takes to run the test hints that something actually happened, rather than skipping the test, but we can verify it in the test output: Also, running with skid option works fine: Now let's wait and see what CI says. |
|
@shipilev CI on GH actions doesn't like the patch: Do you have suggestions on how to address this? This is what I've been battling with all throughout this patch, finding a way for the test to pass on both CI on GH actions and locally, and as you can see, it's not easy to find something that works in all environments. |
|
Maybe the GH runner does not have PMU enabled? Given that it works in manual runs, I think it is fine if we just fix the test. It is not worth the hassle to fight GH infra, IMO. If there are no events gathered, don't fail the test. See how Windows perfasm test does it: jmh/jmh-core-it/src/test/java/org/openjdk/jmh/it/profilers/WinPerfAsmProfilerTest.java Line 60 in c2931c9
I am going to test the patch locally too. |
My local tests work well too. So the only thing left is to fix the test, and we are done! |
|
/integrate |
|
/sponsor |
Fixes https://bugs.openjdk.org/browse/CODETOOLS-7903740
Progress
Issue
Reviewers
Reviewing
Using
gitCheckout this PR locally:
$ git fetch https://git.openjdk.org/jmh.git pull/132/head:pull/132$ git checkout pull/132Update a local copy of the PR:
$ git checkout pull/132$ git pull https://git.openjdk.org/jmh.git pull/132/headUsing Skara CLI tools
Checkout this PR locally:
$ git pr checkout 132View PR using the GUI difftool:
$ git pr show -t 132Using diff file
Download this PR as a diff file:
https://git.openjdk.org/jmh/pull/132.diff
Webrev
Link to Webrev Comment