-
Notifications
You must be signed in to change notification settings - Fork 619
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make ci-bench more deterministic #1444
Conversation
I have another improvement in mind, which is to automatically output the cachegrind diff for the benchmarks that have significant instruction differences. I'll leave that for tomorrow, though |
ci-bench/src/main.rs
Outdated
@@ -116,8 +117,8 @@ fn main() -> anyhow::Result<()> { | |||
.get(index as usize) | |||
.ok_or(anyhow::anyhow!("Benchmark not found: {index}"))?; | |||
|
|||
let mut stdin = io::stdin().lock(); | |||
let mut stdout = io::stdout().lock(); | |||
let mut stdin = util::UnbufferedStdinReader; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if i've understood this correctly, it would be equivalent to do let mut stdin = unsafe { File::from_raw_fd(0) };
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like that! The only drawback I see is that each File
will be closed upon drop, meaning that stdin
and stdout
will get closed too. I think that's not a real problem, though, because if we ever want to use stdio elsewhere, we'll want to do so in an unbuffered way (reusing the files).
(Just pushed the changes use File
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could do a mem::forget()
for the files, I suppose?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
True, I'll add that for ultimate correctness
1691a73
to
78a8b31
Compare
Codecov Report
@@ Coverage Diff @@
## main #1444 +/- ##
=======================================
Coverage 96.49% 96.49%
=======================================
Files 71 71
Lines 15160 15160
=======================================
Hits 14628 14628
Misses 532 532 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
78a8b31
to
8370f0c
Compare
The failing CI jobs seem transient |
8370f0c
to
c8be5c8
Compare
* Switch to unbuffered stdio * Use FxHasher where possible
Head branch was pushed to by a user without write access
c8be5c8
to
d440dbc
Compare
Note: the CI failure is expected because the benchmarking code has changed between main and this branch (and because main is noisy).