-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Workflow run benchmarks #6
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left a few small suggestions for improvements, but looks good.
Looking at the benchmark JSON: "group": "Add-2-dense-numpy",
"name": "test_add[2-dense-numpy]",
"fullname": "test_linear_algebra.py::test_add[2-dense-numpy]",
"params": {
"size": 2,
"density": "dense",
"dtype": "numpy"
},
"param": "2-dense-numpy", These groups, names, etc are going to be tricky to adjust later, so they should be conceptually clean, not tailored to the current use cases. The To me I'm fine with this being address in a later PR, but let's at least create an issue on this repo for it now, and keep in mind that we will likely have to throw away or migrate all the benchmark JSON files later if we adjust the groups. Note that there might be multiple tests that populate the same group (e.g. two different implementations might require rather different implementations). E.g. We could imagine |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I left a long comment about the "group" that each result belongs to, but I'm happy for an issue to be opened to discuss that and for this to be merged now.
Regarding the group I will create an issue to discuss it. |
PR containing the first part of the github workflow.
This workflow runs the benchmarks for add and matmul and stores the result in an artefact.
The workflow will not work yet since the previous PR hasn´t been merged yet.