[pruning][docs] Update README.md for structured pruning #90403
Conversation
Summary: I wrote a tutorial of how to use structured pruning flow as part of BE week Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/90403
Note: Links to docs will display an error until the docs builds have been completed. ❌ 7 FailuresAs of commit a2957d3: NEW FAILURES - The following jobs have failed:
FLAKY - The following jobs failed but were likely due to flakiness present on master:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
Summary: I wrote a tutorial of how to use structured pruning flow as part of BE week Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: bc6e1c76039066785c02d67c717707173c8d43bd Pull Request resolved: #90403
| * **Structured pruning** refers to pruning by removing entire rows / columns of a weight. | ||
|
|
||
| Structured pruning offers several advantages over unstructured pruning at the cost of lower granuality: | ||
| 1. We can store the masks more efficeintly in structured pruning. |
There was a problem hiding this comment.
its a bit confusing to suddenly introduce terminology, we haven't said what masks are, the user would not be aware that this was a problem that needed to be solve when reading this line
|
|
||
| ## What do I need to use Structured Pruning? | ||
|
|
||
| **Your model must be FX symbolically traceable** |
There was a problem hiding this comment.
how can a user check this? or get more info about it?
|
|
||
| **Your model must be FX symbolically traceable** | ||
|
|
||
| We support structured pruning for the following patterns out of the box. |
There was a problem hiding this comment.
pattern hasn't been defined
|
|
||
| The Base Pruner inherits from the Base Sparsifier. | ||
| Let's walk through this line by line to understand what the different parts are doing. |
There was a problem hiding this comment.
may be better to walk line by line with comments in the code block, as is, this section seems less line-by-line and more just going over the main parts of the API, i.e. most of the lines are not covered
|
|
||
| # Motivation | ||
| To specify the layers to prune we just need the fully qualified name (FQN) of the tensor you are looking to prune in the module. | ||
| You can get the FQN of a tensor by printing out `model.named_parameters()`. |
There was a problem hiding this comment.
may want to define what fqn means
|
|
||
|  | ||
| One common pruning criteria is to use the **saliency** of a row, which is defined as the sum of all the L1 norms of the weights in the row. |
There was a problem hiding this comment.
We've used weights to mean weight tensors previously now it looks like this means weight tensor values should keep it consistent. Also you can just say L1 norm of the row, L1 norm is an operator on tensors, otherwise I would normally say sum of absolute value of all values in the row
|
|
||
| **PruningParametrization:** After pruning, the shape of the weight changes (some of the output channels are pruned). That means the output of the current layer will have less output layers compared to the original. This means that the next layer should have less input channels. | ||
| Now that we've defined the sparse config for our model, we need to specify the pruning criteria that we will use. | ||
| To do this, we need to extend a `BaseStructuredSparsifier` with a custom `update_mask` function. |
There was a problem hiding this comment.
I think ideally users will not need to implement their own Pruners, I'm not sure if this is necessary for an introduction
|
|
||
| Once this is done for all matches, `prune` returns your pruned model: |
There was a problem hiding this comment.
I would maybe clarify that this is not actually what is being returned, it returns a pruned model, the following are useful stats to compare the before/after
| ``` | ||
| pruner.squash_mask() | ||
| ``` | ||
| If you're working with linear/conv2d layers, it's very probable that you just need to add an entry to the pattern dict mapping your pattern to an existing prune_function. |
There was a problem hiding this comment.
unrelated to the documentation, this API seems a bit painful for the average user, if all the user is doing most of the time is indicating that in their (conv-x-conv) pattern that the x op doesn't need to be pruned, the API should probably be able to handle that more elegantly than requiring a user to define a new function that essentially just calls the normal conv2d pruner. If thats not true, i.e. the average user actually needs to carefully handle the x op pruning in most anticipated cases, I think this guide should probably have an example going through that
There was a problem hiding this comment.
I think the goal is that for most of these patterns, they are already present in the default pruning pattern dict. So for most users this section is unnecessary, and all they have to interact with is the code in the section above.
I think I might split this section and the Implementing a pruner section out, these are more for power users and probably don't need to be in the tutorial.
Summary: I wrote a tutorial of how to use structured pruning flow as part of BE week Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
Summary: I wrote a tutorial of how to use structured pruning flow as part of BE week Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 084fec91a5578d525f7ae105543463106a7c8b80 Pull Request resolved: #90403
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: The following mandatory check(s) failed (Rule Dig deeper by viewing the failures on hud Details for Dev Infra teamRaised by workflow job |
|
@pytorchbot merge -f "documentation changes only" |
Merge startedYour change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Summary: I wrote a tutorial of how to use structured pruning flow as part of BE week Test Plan: Reviewers: Subscribers: Tasks: Tags: Pull Request resolved: pytorch#90403 Approved by: https://github.com/HDCharles
Stack from ghstack (oldest at bottom):
Summary:
I wrote a tutorial of how to use structured pruning flow as part of BE week
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags: