Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

removed ScheduledOptimizer, moved logic to ScheduledModifierManager #77

Merged
merged 2 commits into from
Feb 18, 2021
Merged

removed ScheduledOptimizer, moved logic to ScheduledModifierManager #77

merged 2 commits into from
Feb 18, 2021

Conversation

@bfineran
Copy link
Contributor

@bfineran bfineran commented Feb 17, 2021 •
edited
Loading

  • Deleted ScheduledOptimizer
  • added steps in ScheduledModifierManager initialization to wrap the existing optimizer's step function to do manager pre/post step updates
  • Added PyTorch modifier finalize function to clean up any unwanted state or hooks from PyTorch modifiers, including the overwritten step function
  • updated pruning modifiers with finalize
  • updated documentation, scripts, and examples to use the new Manager initialization flow

udpated PyTorch integration flow:

from sparseml.pytorch.optim import ScheduledModifierManager

# setup
model = None  # your model definition
optimizer = None  # your optimizer definition
num_train_batches = len(train_data) / batch_size  # your number of batches per training epoch

# integration
manager = ScheduledModifierManager.from_yaml("/PATH/TO/recipe.yaml")
manager.initialize(model, optimizer, steps_per_epoch=num_train_batches)

# PyTorch training code...

# finalize cleans up any added parameters or hooks
manager.finalize(model, optimizer)

@bfineran bfineran requested review from a team and markurtz February 17, 2021 19:58
@bfineran bfineran self-assigned this Feb 17, 2021
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@bfineran bfineran force-pushed the remove-scheduled-optimizer branch from 3350f3f to 6e1dd72 Compare February 17, 2021 20:38
markurtz
markurtz previously approved these changes Feb 17, 2021
View reviewed changes
Copy link
Member

@markurtz markurtz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, excited for how this is going to turn out

@markurtz markurtz mentioned this pull request Feb 17, 2021
Merged
@bfineran bfineran requested a review from a team February 17, 2021 23:23
@bfineran bfineran changed the title removing ScheduledOptimizer, moving logic to ScheduledModifierManager removed ScheduledOptimizer, moved logic to ScheduledModifierManager Feb 17, 2021
bfineran
bfineran commented Feb 17, 2021
View reviewed changes
@bfineran bfineran requested a review from a team February 17, 2021 23:57
@bfineran bfineran merged commit 6073abb into main Feb 18, 2021
@bfineran bfineran deleted the remove-scheduled-optimizer branch February 18, 2021 22:44
bfineran pushed a commit that referenced this pull request Feb 20, 2021
Revert "removed ScheduledOptimizer, moved logic to ScheduledModifierM…
1b9187f 
…anager (#77)"

This reverts commit 6073abb.
@bfineran bfineran mentioned this pull request Feb 20, 2021
Merged
bfineran added a commit that referenced this pull request Feb 20, 2021
@bfineran
Revert "removed ScheduledOptimizer, moved logic to ScheduledModifierM…
6fe3b59 
…anager (#77)" (#80)

This reverts commit 6073abb.
markurtz added a commit that referenced this pull request Feb 25, 2021
@markurtz @jeanniefinks
@bfineran @eldarkurtic @natuan
Rebase release/0.1 off of main for 0.1.1 (#87)
edcce80 
* GA code, toctree links (#61)

- added tracking for docs output
- added help links for docs output

* Update README.md (#64)

removed placeholder reference to comingsoon repo in favor of active repo

* add decorator for flaky tf sparsity tests (#65)

* enable modifier groups in SparseML recipes (#66)

* enable modifier groups in SparseML recipes
* unit tests for YAML modifier list loading

* make build argument for nightly builds (#63)

* rst url syntax correction (#67)

correcting double slash at the end of URLs with updates to index.rst pre-compilation

* match types explicitly in torch qat quant observer wrappers (#68)

* docs updates (#71)

enhancing left nav for Help; after this merge, the docs need to be rebuilt for this repo so docs.neuralmagic.com can be refreshed. cc @markurtz

* Rename KSModifier to PruningModifier (#76)

* Rename ConstantKSModifier to ConstantPruningModifier

* Rename GradualKSModifier to GMPruningModifier

* Fix broken link for Optimization Recipes (#75)

* Serialize/deserialize MaskedLayer (#69)

* Serialize/deserialize MaskedLayer

* Remove unused symbols

* Register pruning scheduler classes for serialization

* removed ScheduledOptimizer, moved logic to ScheduledModifierManager (#77)

* load recipes directly from SparseZoo (#72)

* load recipes into Managers from sparsezoo stubs
* moving recipe_type handling to SparseZoo only, supporting loading SparseZoo recipe objects

* Revert "removed ScheduledOptimizer, moved logic to ScheduledModifierManager (#77)" (#80)

This reverts commit 6073abb.

* Update for 0.1.1 release (#82)

* Update for 0.1.1 release
- update python version to 0.1.1
- setup.py add in version parts and _VERSION_MAJOR_MINOR for more flexibility with dependencies between neural magic packages
- add in deepsparse optional install pathway

* missed updating version to 0.1.1

* rename examples directory to integrations (#78)

* rwightman/pytorch-image-models integration (#70)

* load checkpoint file based on sparsezoo recipe in pytorch_vision script (#83)

* ultralytics/yolov5 integration (#73)

* pytorch sparse quantized transfer learning notebook (#81)

* load qat onnx models for conversion from file path (#86)

* Sparsification update (#84)

* Sparsification update
- update sparsification descriptions and move to preferred verbage

* update from comments on deepsparse for sparsification

* Update README.md

Co-authored-by: Jeannie Finks <74554921+jeanniefinks@users.noreply.github.com>

* Update README.md

Co-authored-by: Jeannie Finks <74554921+jeanniefinks@users.noreply.github.com>

* Update README.md

Co-authored-by: Jeannie Finks <74554921+jeanniefinks@users.noreply.github.com>

* Update README.md

Co-authored-by: Jeannie Finks <74554921+jeanniefinks@users.noreply.github.com>

* Update docs/source/index.rst

Co-authored-by: Jeannie Finks <74554921+jeanniefinks@users.noreply.github.com>

* Update docs/source/index.rst

Co-authored-by: Jeannie Finks <74554921+jeanniefinks@users.noreply.github.com>

* Update docs/source/index.rst

Co-authored-by: Jeannie Finks <74554921+jeanniefinks@users.noreply.github.com>

* Update docs/source/index.rst

Co-authored-by: Jeannie Finks <74554921+jeanniefinks@users.noreply.github.com>

* Update docs/source/recipes.md

Co-authored-by: Jeannie Finks <74554921+jeanniefinks@users.noreply.github.com>

* fix links in index.rst from reviewed content

* update overviews and taglines from doc

Co-authored-by: Jeannie Finks <74554921+jeanniefinks@users.noreply.github.com>

* blog style readme for torch sparse-quant TL notebook (#85)

Co-authored-by: Jeannie Finks (NM) <74554921+jeanniefinks@users.noreply.github.com>
Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>
Co-authored-by: Eldar Kurtic <eldar.ciki@gmail.com>
Co-authored-by: Tuan Nguyen <tuan@neuralmagic.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

2 more reviewers

@natuan natuan natuan approved these changes

@markurtz markurtz markurtz approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@bfineran bfineran

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@bfineran @natuan @markurtz