Skip to content

Enable FunctionHookMode tracing by default #3338

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Apr 11, 2025
Merged

Enable FunctionHookMode tracing by default #3338

merged 6 commits into from
Apr 11, 2025

Conversation

AlexanderDokuchaev
Copy link
Collaborator

@AlexanderDokuchaev AlexanderDokuchaev commented Mar 11, 2025 •
edited
Loading

Changes

  • Enable FunctionHookMode tracing by default
  • Update examples:
    • torch/ssd300_vgg16: remove disable tracing for normalization (is works correctly but changed final metric 0.5148 vs 0.5145)
    • qat/torch/anomalib: in model added new FQ in result changes graph, on windows changed metric (win: 0.9692, linux: 0.9767)
  • check model devices to insert modules in load_from_config
  • Update lora example:
    • changes to support new tracing
    • add filter for tracing warning to clean outputs
    • update gitignore to ignore output files

Related tickets

152996

Tests

https://github.com/openvinotoolkit/nncf/actions/runs/14352342788/job/40233792090
https://github.com/openvinotoolkit/nncf/actions/runs/14353482882/job/40237562175
NNCF/job/manual/job/post_training_quantization/643/

@github-actions github-actions bot added NNCF PT Pull requests that updates NNCF PyTorch experimental NNCF Common Pull request that updates NNCF Common NNCF PTQ Pull requests that updates NNCF PTQ API Public API-impacting changes labels Mar 11, 2025
@github-actions github-actions bot removed the API Public API-impacting changes label Mar 13, 2025
@AlexanderDokuchaev AlexanderDokuchaev changed the title enable experimental tracing by default Enable FunctionHookMode tracing by default Mar 13, 2025
@AlexanderDokuchaev
Copy link
Collaborator Author

Diff in ssd300:
image

@AlexanderDokuchaev
Copy link
Collaborator Author

Diff in anomalib
image
image

@AlexanderDokuchaev AlexanderDokuchaev marked this pull request as ready for review March 13, 2025 23:31
@AlexanderDokuchaev AlexanderDokuchaev requested a review from a team as a code owner March 13, 2025 23:31
@alexsu52
Copy link
Contributor

Diff in anomalib image image

Is the left operand of the multiply ops quantized?

alexsu52
alexsu52 requested changes Mar 14, 2025
View reviewed changes
alexsu52
alexsu52 reviewed Mar 14, 2025
View reviewed changes
Copy link
Contributor

@alexsu52 alexsu52 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check your PR with #3322 and run the nightly training test for Torch backend.
cc' @ljaljushkin

ljaljushkin
ljaljushkin reviewed Mar 14, 2025
View reviewed changes
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Mar 14, 2025
alexsu52
alexsu52 reviewed Mar 17, 2025
View reviewed changes
@AlexanderDokuchaev AlexanderDokuchaev marked this pull request as draft March 19, 2025 14:24
@github-actions github-actions bot added NNCF Common Pull request that updates NNCF Common NNCF PTQ Pull requests that updates NNCF PTQ and removed documentation Improvements or additions to documentation NNCF Common Pull request that updates NNCF Common NNCF PTQ Pull requests that updates NNCF PTQ labels Apr 9, 2025
@AlexanderDokuchaev AlexanderDokuchaev marked this pull request as ready for review April 9, 2025 12:19
@ljaljushkin ljaljushkin mentioned this pull request Apr 10, 2025
Merged
@AlexanderDokuchaev
Copy link
Collaborator Author

Example qat_with_lora main.py --epochs=1 --pretrained=HuggingFaceTB/SmolLM2-135M-Instruct --num_train_samples=128 --seqlen=128 --lora_rank=8 --batch_size=16 --microbatch_size=4 --lr=5e-4
Develop:
image

New best WWB similarity = 0.7795
torch.cuda.max_memory_allocated()  # 1189900800`

PR:
image

New best WWB similarity = 0.7795
torch.cuda.max_memory_allocated()  # 1189900800`

@ljaljushkin
Copy link
Contributor

ljaljushkin commented Apr 11, 2025 •
edited
Loading

Was able to reproduce results on wikitext with new tracing.


Model
Epochs
Micro

batch

size
Mode
Tracing
Max allocated

CUDA memory

per GPU, Gb
Elapsed

Time,

hour

HuggingFaceTB/SmolLM-1.7B-Instruct
10
8
Single GPU
New
48.4
0.5

HuggingFaceTB/SmolLM-1.7B-Instruct
10
8
Single GPU
Old
49.2
0.5

meta-llama/Meta-Llama-3-8B-Instruct
10
2
Two GPU
New
49.1
3

meta-llama/Meta-Llama-3-8B-Instruct
10
2
Two GPU
Old
49.1
2.8

Final perplexity is within the margin of error
image

image

@AlexanderDokuchaev AlexanderDokuchaev merged commit 77f0bf6 into openvinotoolkit:develop Apr 11, 2025
19 checks passed
@AlexanderDokuchaev AlexanderDokuchaev mentioned this pull request Apr 15, 2025
Merged
AlexanderDokuchaev added a commit that referenced this pull request Apr 15, 2025
@AlexanderDokuchaev
Remove disable patching for fx examples and docs (#3435)
09269bf 
### Reason for changes

No need after #3338

###
https://github.com/openvinotoolkit/nncf/actions/runs/14470409240
manual/job/post_training_quantization/648
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alexsu52 alexsu52 alexsu52 approved these changes

@ljaljushkin ljaljushkin ljaljushkin approved these changes

Assignees

@alexsu52 alexsu52

Labels
experimental NNCF Common Pull request that updates NNCF Common NNCF PT Pull requests that updates NNCF PyTorch NNCF PTQ Pull requests that updates NNCF PTQ
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@AlexanderDokuchaev @alexsu52 @ljaljushkin