Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark results are not useful because of formatting mis-alignment #772

Open
nyurik opened this issue Feb 8, 2024 · 2 comments
Open
Labels
bug Something isn't working

Comments

@nyurik
Copy link
Contributor

nyurik commented Feb 8, 2024

Looking at any performance reports, I see that the results of the main branch are not on the same line as the results of the current PR. Is this a configuration bug of some sort? For example, here, these two lines should be written as one

Benchmark main nyurik:lints Change
🆕 load BSBM explore 1000 in on disk with bulk load N/A 2.9 s N/A
⁉️ load BSBM explore 1000 in on disk with bulk load 2.9 s N/A N/A
@nyurik nyurik added the bug Something isn't working label Feb 8, 2024
@Tpt
Copy link
Collaborator

Tpt commented Feb 8, 2024

I believe it's because of #741: the bench paths have changed and so they are not considered to be the same by the bench tool. And because the reference benchmarks are computed every night and I have merged #741 this morning the comparison point is still before #741. I hope this will sort out itself tomorrow

@nyurik
Copy link
Contributor Author

nyurik commented Feb 8, 2024

Ah, thx, so this is not a permanent thing, thanks for looking into it! I'll close it in a day just to make sure

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants