Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add scripts for comprehensive benchmark TensorIterator #30248

Open
zasdfgbnm opened this issue Nov 21, 2019 · 0 comments
Open

Add scripts for comprehensive benchmark TensorIterator #30248

zasdfgbnm opened this issue Nov 21, 2019 · 0 comments
Assignees
Labels
feature A request for a proper, new feature. module: performance Issues related to performance, either of kernel code or framework glue triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@zasdfgbnm
Copy link
Collaborator

zasdfgbnm commented Nov 21, 2019

馃殌 Feature

Proposed in: #29743

TensorIterator is performance-critical but complicated. Changing some parts of it might
cause hard-to-realize regression on the other parts. It would be very convenient to have a script that benchmark all the case we could imagine:

Memory layout: trivial 1D, (contiguous dim, non-contiguous dims)
Problem size: small, medium, large
Type of computation: unary ops, binary ops, compare ops, reduction
Data type: all dtypes with/without promotion
Inplace: True, False
Device: CPU, CUDA

The designed usage of the script could be:

Step 1:
Install a PyTorch build of the master branch, and run

python main.py benchmark baseline.json

Step 2:
Go to your branch, build install and run

python main.py benchmark new.json

Step 3:
Run the following command to get the report:

python main.py compare baseline.json new.json report.html

cc: @ngimel @VitalyFedyunin
cc: @csarofeen

cc @VitalyFedyunin @ngimel @mruberry

@cpuhrsch cpuhrsch added feature A request for a proper, new feature. module: performance Issues related to performance, either of kernel code or framework glue triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels Nov 21, 2019
@zasdfgbnm zasdfgbnm self-assigned this Nov 21, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature A request for a proper, new feature. module: performance Issues related to performance, either of kernel code or framework glue triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants