Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update default NNCF configurations #824

Merged
merged 6 commits into from
Jul 17, 2024
Merged

Update default NNCF configurations #824

merged 6 commits into from
Jul 17, 2024

Conversation

KodiaqQ
Copy link
Contributor

@KodiaqQ KodiaqQ commented Jul 15, 2024

What does this PR do?

  • Updates default NNCF configuration according to experiments.
  • Fixes wrong AWQ parameter usage.

@KodiaqQ
Copy link
Contributor Author

KodiaqQ commented Jul 15, 2024

@nikita-savelyevv, @eaidova, review, please.

@KodiaqQ
Copy link
Contributor Author

KodiaqQ commented Jul 15, 2024 •
edited
Loading

@echarlaix, please do not merge, until the tests for the NNCF configurations wouldn't be added (after discussion with @eaidova).

@HuggingFaceDocBuilderDev
Copy link

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

nikita-savelyevv
nikita-savelyevv reviewed Jul 15, 2024
View reviewed changes
Copy link
Collaborator

@nikita-savelyevv nikita-savelyevv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

optimum/intel/openvino/configuration.py Outdated
"togethercomputer/RedPajama-INCITE-7B-Instruct": {"bits": 4, "sym": False, "group_size": 128},
"HuggingFaceH4/zephyr-7b-beta": {
"bits": 4,
"sym": True,
"group_size": 128,
"ratio": 0.8,
"dataset": "wikitext2",
"awq": True,
"quant_method": "awq",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"quant_method": "awq",
"quant_method": OVQuantizationMethod.AWQ,

Here and below

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This requires to replace the OVQuantizationMethod definition before _DEFAULT_4BIT_CONFIGS. Do you agree with that change?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This requires to replace the OVQuantizationMethod definition before _DEFAULT_4BIT_CONFIGS. Do you agree with that change?

Yes, sure

@KodiaqQ
Copy link
Contributor Author

KodiaqQ commented Jul 15, 2024

@echarlaix, @nikita-savelyevv, review again, please.

nikita-savelyevv
nikita-savelyevv reviewed Jul 15, 2024
View reviewed changes
tests/openvino/test_quantization.py Outdated
@@ -849,6 +849,14 @@ def test_config_from_dict(self, quantization_config: dict, config_type: type, wa
if hasattr(ov_config.quantization_config, k):
self.assertEqual(getattr(ov_config.quantization_config, k), v)

@parameterized.expand(_DEFAULT_4BIT_CONFIGS)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also please include _DEFAULT_4BIT_CONFIG here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do that, but the test wouldn't look acceptable then, since _DEFAULT_4BIT_CONFIGS is a dictionary with the model_id as key and _DEFAULT_4BIT_CONFIG is not.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will not work?
@prametrized(_DEFAULT_4BIT_CONFIGS.update({"default": _DEFAULT_4BIT_CONFIG))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, no. .update method doesn't return any value.

@KodiaqQ KodiaqQ requested a review from nikita-savelyevv July 15, 2024 14:48
@KodiaqQ
Copy link
Contributor Author

KodiaqQ commented Jul 17, 2024

@echarlaix, could you review this again, please?

@MaximProshin
Copy link
Contributor

@echarlaix , can you please help to merge it?

@echarlaix echarlaix merged commit 31f49a2 into huggingface:main Jul 17, 2024
13 of 17 checks passed
@nikita-savelyevv nikita-savelyevv mentioned this pull request Jul 23, 2024
Closed
KodiaqQ added a commit to KodiaqQ/optimum-intel that referenced this pull request Jul 25, 2024
@KodiaqQ
Revert "Update default NNCF configurations (huggingface#824)"
358ef01 
This reverts commit 31f49a2.
@KodiaqQ KodiaqQ mentioned this pull request Jul 25, 2024
Merged
echarlaix pushed a commit that referenced this pull request Jul 26, 2024
@KodiaqQ
Partial revert of #824 (#842)
cad4c4f 
* Revert "Update default NNCF configurations (#824)"

This reverts commit 31f49a2.

* Partial revert scale algorithm apply

* Missed changes

* Delete missed config

* Added missed config
@nikita-savelyevv nikita-savelyevv mentioned this pull request Jul 28, 2024
Merged
3 tasks
IlyasMoutawwakil pushed a commit that referenced this pull request Aug 6, 2024
@KodiaqQ @IlyasMoutawwakil
Update default NNCF configurations (#824)
45b7291 
* Add configs from 143530

* Fix wrong AWQ option

* Apply comment

* Add test

* Add missed configuration

* Apply comment
IlyasMoutawwakil pushed a commit that referenced this pull request Aug 6, 2024
@KodiaqQ @IlyasMoutawwakil
Partial revert of #824 (#842)
2a2f082 
* Revert "Update default NNCF configurations (#824)"

This reverts commit 31f49a2.

* Partial revert scale algorithm apply

* Missed changes

* Delete missed config

* Added missed config
IlyasMoutawwakil pushed a commit that referenced this pull request Aug 6, 2024
@KodiaqQ @IlyasMoutawwakil
Update default NNCF configurations (#824)
4fb3848 
* Add configs from 143530

* Fix wrong AWQ option

* Apply comment

* Add test

* Add missed configuration

* Apply comment
IlyasMoutawwakil pushed a commit that referenced this pull request Aug 6, 2024
@KodiaqQ @IlyasMoutawwakil
Partial revert of #824 (#842)
3395168 
* Revert "Update default NNCF configurations (#824)"

This reverts commit 31f49a2.

* Partial revert scale algorithm apply

* Missed changes

* Delete missed config

* Added missed config
IlyasMoutawwakil pushed a commit that referenced this pull request Aug 6, 2024
@KodiaqQ @IlyasMoutawwakil
Partial revert of #824 (#842)
abfb4df 
* Revert "Update default NNCF configurations (#824)"

This reverts commit 31f49a2.

* Partial revert scale algorithm apply

* Missed changes

* Delete missed config

* Added missed config
IlyasMoutawwakil pushed a commit that referenced this pull request Aug 6, 2024
@KodiaqQ @IlyasMoutawwakil
Update default NNCF configurations (#824)
81c6ff7 
* Add configs from 143530

* Fix wrong AWQ option

* Apply comment

* Add test

* Add missed configuration

* Apply comment
IlyasMoutawwakil pushed a commit that referenced this pull request Aug 6, 2024
@KodiaqQ @IlyasMoutawwakil
Partial revert of #824 (#842)
e3f9d12 
* Revert "Update default NNCF configurations (#824)"

This reverts commit 31f49a2.

* Partial revert scale algorithm apply

* Missed changes

* Delete missed config

* Added missed config
IlyasMoutawwakil pushed a commit that referenced this pull request Aug 6, 2024
@KodiaqQ @IlyasMoutawwakil
Update default NNCF configurations (#824)
2e6ff25 
* Add configs from 143530

* Fix wrong AWQ option

* Apply comment

* Add test

* Add missed configuration

* Apply comment
IlyasMoutawwakil pushed a commit that referenced this pull request Aug 6, 2024
@KodiaqQ @IlyasMoutawwakil
Update default NNCF configurations (#824)
e513cb3 
* Add configs from 143530

* Fix wrong AWQ option

* Apply comment

* Add test

* Add missed configuration

* Apply comment
IlyasMoutawwakil pushed a commit that referenced this pull request Aug 6, 2024
@KodiaqQ @IlyasMoutawwakil
Partial revert of #824 (#842)
28e7fe8 
* Revert "Update default NNCF configurations (#824)"

This reverts commit 31f49a2.

* Partial revert scale algorithm apply

* Missed changes

* Delete missed config

* Added missed config
@KodiaqQ KodiaqQ deleted the nm/update_defaults branch August 12, 2024 15:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@eaidova eaidova eaidova approved these changes

@nikita-savelyevv nikita-savelyevv nikita-savelyevv approved these changes

@echarlaix echarlaix Awaiting requested review from echarlaix

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@KodiaqQ @HuggingFaceDocBuilderDev @MaximProshin @nikita-savelyevv @eaidova @echarlaix