Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handle NonDynamicallyQuantizableLinear in smoothquant module #30

Closed
ELanning opened this issue Dec 31, 2023 · 2 comments
Closed

Handle NonDynamicallyQuantizableLinear in smoothquant module #30

ELanning opened this issue Dec 31, 2023 · 2 comments

Comments

@ELanning
Copy link 

Traceback (most recent call last):
  File "C:/Program Files/JetBrains/PyCharm Community Edition 2023.2.1/plugins/python-ce/helpers/pydev/pydevd.py", line 1527, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2023.2.1\plugins\python-ce\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "C:\code\foobar\scripts\quantize.py", line 37, in <module>
    swap_linear_with_smooth_fq_linear(model)
  File "C:\code\py-envs\foobar\lib\site-packages\torchao\quantization\smoothquant.py", line 219, in swap_linear_with_smooth_fq_linear
    swap_linear_with_smooth_fq_linear(child, skip_fqn_list, new_fqn, alpha)
  File "C:\code\py-envs\foobar\lib\site-packages\torchao\quantization\smoothquant.py", line 219, in swap_linear_with_smooth_fq_linear
    swap_linear_with_smooth_fq_linear(child, skip_fqn_list, new_fqn, alpha)
  File "C:\code\py-envs\foobar\lib\site-packages\torchao\quantization\smoothquant.py", line 219, in swap_linear_with_smooth_fq_linear
    swap_linear_with_smooth_fq_linear(child, skip_fqn_list, new_fqn, alpha)
  [Previous line repeated 1 more time]
  File "C:\code\py-envs\foobar\lib\site-packages\torchao\quantization\smoothquant.py", line 215, in swap_linear_with_smooth_fq_linear
    target_cls = source_cls_to_target_cls[type(child)]
KeyError: <class 'torch.nn.modules.linear.NonDynamicallyQuantizableLinear'>
python-BaseException

Expected: NonDynamicallyQuantizableLinear layer is skipped (possibly with a warning), or properly handled.
Actual: exception.

It sounds like HDCharles was planning on fixing this more generally: pytorch/pytorch#58969
@HDCharles
Copy link
Contributor

Hey, this is unrelated, its a different api. there's just a bug in the swap_linear_with_smooth_fq_linear.

@HDCharles HDCharles mentioned this issue Dec 20, 2023
Merged
HDCharles added a commit that referenced this issue Jan 3, 2024
@HDCharles
Update on "smoothquant fixes"
9a5487e 
Summary: certain custom linear modules add additional inputs to the
forward that need to be handled but can be otherwise ignored.
Additionally swap_linear_with_smooth_fq_linear had a bug where linear
subclasses would get past the if statement and error on the dict key
lookup since the actual class wasn't expected.
(#30) enabled
NonDynamicallyQuantizableLinear to work with smoothquant and fixed bug
for other subclasses. At some point this should be brought in line with
the other APIs if its getting use.

Test Plan: python test/test.py

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
HDCharles added a commit that referenced this issue Jan 3, 2024
@HDCharles
smoothquant fixes
500596a 
Summary: certain custom linear modules add additional inputs to the
forward that need to be handled but can be otherwise ignored.
Additionally swap_linear_with_smooth_fq_linear had a bug where linear
subclasses would get past the if statement and error on the dict key
lookup since the actual class wasn't expected.
(#30) enabled
NonDynamicallyQuantizableLinear to work with smoothquant and fixed bug
for other subclasses. At some point this should be brought in line with
the other APIs if its getting use.

Test Plan: python test/test.py

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: 059b392376f36bf13be16ec992a52063707b5abc
Pull Request resolved: #28
HDCharles added a commit that referenced this issue Jan 3, 2024
@HDCharles
smoothquant fixes
e670872 
Summary: certain custom linear modules add additional inputs to the
forward that need to be handled but can be otherwise ignored.
Additionally swap_linear_with_smooth_fq_linear had a bug where linear
subclasses would get past the if statement and error on the dict key
lookup since the actual class wasn't expected.
(#30) enabled
NonDynamicallyQuantizableLinear to work with smoothquant and fixed bug
for other subclasses. At some point this should be brought in line with
the other APIs if its getting use.

Test Plan: python test/test.py

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: 059b392376f36bf13be16ec992a52063707b5abc
Pull Request resolved: #28
@HDCharles
Copy link
Contributor

fixoed in linked PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ELanning @HDCharles