Add activation functions (ReLU and SiLU for now) for structured sparse linear operator

[alexsamardzic]

Stack from ghstack (oldest at bottom):

• -> Add activation functions (ReLU and SiLU for now) for structured sparse linear operator #101339

cc @nikitaved @pearu @cpuhrsch @amjames @bhosmer

Differential Revision: D46453476

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/101339

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ 3 Unrelated Failures

As of commit 4dd1e44:

UNSTABLE - The following jobs failed but were likely due to flakiness present on trunk and has been marked as unstable:

• linux-bionic-cuda11.8-py3.10-gcc9 / test (distributed, 1, 3, linux.8xlarge.nvidia.gpu, unstable) (gh)
• linux-bionic-cuda11.8-py3.10-gcc9 / test (distributed, 2, 3, linux.8xlarge.nvidia.gpu, unstable) (gh)
• linux-bionic-cuda12.1-py3.10-gcc9 / test (deploy, 1, 1, linux.4xlarge.nvidia.gpu, unstable) (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[cpuhrsch]

Overall the changes look reasonable. Can you check how much this increases our binary size? You can look at the size of libtorch_cuda.so. I'm worried we'll start adding a lot of size.

[alexsamardzic]

It doesn't seem too much of a difference: without my PRs, the size of libtorch_cuda.so is 152.51MB, and with them it's 153.15 MB.

[cpuhrsch]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[pytorchmergebot]

Merge failed

Reason: PR #100881 has not been reviewed yet

Details for Dev Infra team

Raised by workflow job

Failing merge rule: Core Maintainers

[amjames]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[pytorchmergebot]

Merge failed

Reason: This PR is too stale; the last push date was more than 3 days ago. Please rebase and try again. You can rebase and merge by leaving the following comment on this PR:
@pytorchbot merge -r
Or just rebase by leaving @pytorchbot rebase comment

Details for Dev Infra team

Raised by workflow job

[cpuhrsch]

@pytorchbot merge -r

[pytorchmergebot]

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[pytorchmergebot]

Successfully rebased gh/alexsamardzic/12/orig onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via ghstack checkout https://github.com/pytorch/pytorch/pull/101339)

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[izaitsevfb]

@pytorchbot revert -m "Depends on #100881, which has to be reverted due to internal build breakage." -c ghfirst

[pytorchmergebot]

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

[pytorchmergebot]

@alexsamardzic your PR has been successfully reverted.

[izaitsevfb]

@izaitsevfb has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[cpuhrsch]

@pytorchbot rebase

[pytorchmergebot]

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[pytorchmergebot]

Rebase failed due to Command git -C /home/runner/work/pytorch/pytorch rebase refs/remotes/origin/viable/strict gh/alexsamardzic/12/orig returned non-zero exit code 1

Rebasing (1/2)
Auto-merging aten/src/ATen/native/native_functions.yaml
Auto-merging test/forward_backward_compatibility/check_forward_backward_compatibility.py
CONFLICT (content): Merge conflict in test/forward_backward_compatibility/check_forward_backward_compatibility.py
Auto-merging test/test_sparse.py
error: could not apply 66d2638bc35... Implement adding bias vector into structured sparse linear operator
hint: Resolve all conflicts manually, mark them as resolved with
hint: "git add/rm <conflicted_files>", then run "git rebase --continue".
hint: You can instead skip this commit: run "git rebase --skip".
hint: To abort and get back to the state before "git rebase", run "git rebase --abort".
Could not apply 66d2638bc35... Implement adding bias vector into structured sparse linear operator

Raised by https://github.com/pytorch/pytorch/actions/runs/5284684057

[alexsamardzic]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here