OV configurations alignment #787
Conversation
KodiaqQ
commented
Jun 27, 2024
KodiaqQ
commented
- Updated OV configurations to align with openvino.genai nncf_utils.py.
| "ratio": 0.8, | ||
| "dataset": "wikitext2", | ||
| "awq": True, | ||
| }, | ||
| "stablelm-epoch-3b-preview": {"bits": 4, "sym": True, "group_size": 64, "ratio": 0.8}, |
There was a problem hiding this comment.
can this line be removed, it looks like it is accidentally copied with wrong naming format?
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
@KodiaqQ , I still see some misalignment with genai. In genai we have 28 models, but only 26 here. Can you explain it? |
As was noticed, the |
| "sym": True, | ||
| "group_size": 64, | ||
| "ratio": 0.8, | ||
| "dataset": "wikitext2", |
There was a problem hiding this comment.
Is there a chance that using a specific dataset in the default configs causes better accuracy when evaluating on wikitext, but worse for actual use-cases?
There was a problem hiding this comment.
I think that this is possible.
But this is only alignment with the default state from here openvino.genai, configurations aren't new.
cc @AlexKoff88
There was a problem hiding this comment.
Let's skip it here. This task is not to find better configs but rather align optimum-intel configs with ones are currently used in the OV validation which will also switch to optimum-intel. Without it, we can't switch.
|
@echarlaix , can you please help to merge it? |
* Align with genai * Added stable-code-3b * Change order * Update configuration.py
