Transformer Compression using SliceGPT #1052
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
shaahji
force-pushed
the
shaahji/slicegpt
branch
from
1b1a556
to
450a963
Compare
shaahji
changed the title
shaahji
force-pushed
the
shaahji/slicegpt
branch
from
450a963
to
28b0060
Compare
devang-ml
reviewed
Apr 4, 2024
shaahji
force-pushed
the
shaahji/slicegpt
branch
from
28b0060
to
b3f02de
Compare
shaahji
force-pushed
the
shaahji/slicegpt
branch
2 times, most recently
from
d069691
to
147545d
Compare
devang-ml
reviewed
Apr 8, 2024
shaahji
force-pushed
the
shaahji/slicegpt
branch
6 times, most recently
from
31cd97a
to
88556c5
Compare
devang-ml
previously approved these changes
Apr 9, 2024
jambayk
reviewed
Apr 9, 2024
jambayk
reviewed
Apr 9, 2024
jambayk
reviewed
Apr 9, 2024
Adding new pass to use SliceGPT compression technique to improve performance and reduce memory footprint. Updated phi2 example with a new workflow that uses the implemented pass.
shaahji
force-pushed
the
shaahji/slicegpt
branch
from
88556c5
to
e19acb8
Compare
devang-ml
approved these changes
Apr 9, 2024
jambayk
reviewed
Apr 9, 2024
| @staticmethod | ||
| def _default_config(accelerator_spec: AcceleratorSpec) -> Dict[str, PassConfigParam]: | ||
| return { | ||
| "calibration_data_config": PassConfigParam( |
There was a problem hiding this comment.
only the data_config.name is used from this param. Like we discussed offline previously, wouldn't it be simpler to just have string param called calibration_dataset_name or something?
The actual data config is not used at all. Having it here makes the config unnecessarily complicated. It also makes it appear any data config is supported when only three data names are supported by the tool.
There was a problem hiding this comment.
I will follow up on this once I have a discussion with Devang. I think a full data_config keeps the option open for future extension but again it can always be changed in future. Let me follow up after discussion.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Transformer Compression using SliceGPT
Adding new pass to use SliceGPT compression technique to improve performance and reduce memory footprint.
Updated phi2 example with a new workflow that uses the implemented pass.
Release Note: New pass
SliceGPTto compress transformer to improve performance and reduce memory footprint.Checklist before requesting a review
lintrunner -a(Optional) Issue link