This repository has been archived by the owner on May 1, 2023. It is now read-only.
Post-train quantization based on stats + additional modules quantized #136
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Expose command line arguments for collecting and loading stats
* Integrate in image classification sample
* range_linear.py:
* Load stats from YAML file or dict in post-train quantizer
* Refactor layer wrappers to handle both dynamic and stats-based
(aka "static") cases
* collector.py:
* Add dedicated collector for quantization stats
* Allow None classes filter, in which case collect stats for all types
* Expose abstract 'save' function instead of existing 'to_xlsx'
* Fixes to typos and some comments
* Extract code for setting deterministic execution into function, move
to utils.py
* Add ability to shuffle test dataset (required since we don't collect
stats on the whole test set)
* Move YAML loading functionality to utils.py
* Add unit-tests for quantization based on stats
…d control * Command line argument to configure post-train quantizer from file * Integrated in image classification sample * Add ability to load specific "component" from YAML file, without a full- blown scheduler * Minor change in Quantizer initialization - change default of bits_overrides to None
* Concat + element-wise add / mult supported in post-training * Embeddings supported in quantization-aware training * Wrapped PyTorch concat + element-wise add/mult ops in Modules so they could be recognized by the Quantizer * Modified our ResNet implementation to use element-wise add modules instead of operator * Unit tests or concat, add and mult
galnov
approved these changes
Jan 29, 2019
nzmora
suggested changes
Jan 29, 2019
| return dict_config(model, optimizer, sched_dict, scheduler) | ||
| except yaml.YAMLError as exc: | ||
| print("\nFATAL parsing error while parsing the schedule configuration file %s" % filename) | ||
| raise | ||
| raise |
There was a problem hiding this comment.
This raise is not needed...
There was a problem hiding this comment.
This is a weird merge artifact in GitHub, the code on the branch doesn't actually have the redundant statement. So I can't fix it now. If it sticks after the merge I'll remove it then.
* Change semantics of 'qe_calibration' argument to match the new implementation of using part of dataset
michaelbeale-IL
pushed a commit
that referenced
this pull request
Apr 24, 2023
Summary of changes: (1) Post-train quantization based on pre-collected statistics (2) Quantized concat, element-wise addition / multiplication and embeddings (3) Move post-train quantization command line args out of sample code (4) Configure post-train quantization from YAML for more fine-grained control (See PR #136 for more detailed changes descriptions)
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a big PR, will be easiest to review commit-by-commit. Not exactly best-practice, but as it is I prefer this to separate PRs.
Summary of changes:
(1) Post-train quantization based on pre-collected statistics
(2) Quantized concat, element-wise addition / multiplication and embeddings
(3) Move post-train quantization command line args out of sample code
(4) Configure post-train quantization from YAML for more fine-grained control