Skip to content

Add convert_llama3_config_to_decilm_config + unit test #465

Merged
danielkorzekwa merged 30 commits intofeature/compressfrom
dkorzekwa/llama3_to_decilm_convertion
Oct 29, 2025
Merged

Add convert_llama3_config_to_decilm_config + unit test #465
danielkorzekwa merged 30 commits intofeature/compressfrom
dkorzekwa/llama3_to_decilm_convertion

Conversation

@danielkorzekwa
Copy link
Contributor

@danielkorzekwa danielkorzekwa commented Oct 27, 2025

What does this PR do?

  • Add a converter to convert a llama3 model to decilm format
  • Add an integration test

Additional Information

First review/merge #464 and merge it back to this feature branch.

@danielkorzekwa
The main compression function for a model
c758ad5 
using MIP-based NAS search algorithm.

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Code formatting
8af9903 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Model search space configuration used by test_compress.py test.
5ba6c27 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Tokenizer used by test_compress.py test.
0bc5d84 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Tokenizer utility used by test_compress.py test
87d4fa5 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
e2e tests for compress.py
ced1e99 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Add convert_llama3_config_to_decilm_config + unit test
5de0bdc 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa danielkorzekwa requested a review from a team as a code owner October 27, 2025 11:48
@danielkorzekwa danielkorzekwa requested review from kevalmorabia97 and realAsma and removed request for a team October 27, 2025 11:48
@copy-pr-bot
Copy link

copy-pr-bot bot commented Oct 27, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@codecov
Copy link

codecov bot commented Oct 27, 2025 •
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.40%. Comparing base (ad1d18e) to head (ae61644).
⚠️ Report is 1 commits behind head on feature/compress.

Additional details and impacted files 
@@                Coverage Diff                @@
##           feature/compress     #465   +/-   ##
=================================================
  Coverage             73.40%   73.40%           
=================================================
  Files                   180      180           
  Lines                 18127    18127           
=================================================
  Hits                  13306    13306           
  Misses                 4821     4821

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kevalmorabia97 kevalmorabia97 requested review from ChenhanYu and removed request for realAsma October 27, 2025 12:11
kevalmorabia97
kevalmorabia97 reviewed Oct 27, 2025
View reviewed changes
...xperimental/torch/_compress/decilm/converters/test_convert_llama3_config_to_decilm_config.py
return Path(request.config.rootpath)


def test_convert_llama3_config_to_decilm_config(project_root_path: Path, tmp_path: Path):
Copy link
Collaborator

@kevalmorabia97 kevalmorabia97 Oct 27, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we extend this and take a dummy input (available via model.dummy_inputs) and run forward pass on llama model (output1 = model(**model.dummy_inputs)) and converted DeciLM model and assert they are same / almost same (torch.allclose)? This will be a more robuts check that conversion is correct

Copy link
Collaborator

@kevalmorabia97 kevalmorabia97 Oct 27, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a util function to do this. from _test_utils.torch.transformers_models import tf_output_tester

Copy link
Contributor Author

@danielkorzekwa danielkorzekwa Oct 27, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

regarding validating on dummy input (this is planned in the internal nvidia repo, let's migrate it later), creating an issue to migrate it. issues/14

@danielkorzekwa
Remove unused bypass distillation config files.
800414c 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Moving integration tests to tests/experimental to not trigger CICD
16abcc9 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
update docs
a5ba1c7 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Replace mprint with print and replace osp.join with path1 / path2 not…
1bda391 
…ation.

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Refactor file checking assertions to use .is_file() and .exists()
bb38401 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Add a new dependency section to setyp.py for the modelopt.torch._comp…
8415548 
…ress module.

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa danielkorzekwa requested review from a team as code owners October 27, 2025 16:42
danielkorzekwa and others added 4 commits October 27, 2025 18:43
@danielkorzekwa
Move test_convert_llama3_config_to_decilm_config.py to tests/experime…
b1b1833 
…ntal/ folder to not be run by CICD yet.

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@kevalmorabia97
Merge branch 'feature/compress' into dkorzekwa/e2e_compression_test
d4ffc91
@kevalmorabia97
Fix: Add missing LICENSE headers
6f28e4a 
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
@danielkorzekwa
Use spawn_multiprocess_job for test_compress test (to be able to use …
016fb63 
…tmp_path.

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Add comments.
0ccf1c4 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Add _save_dummy_dataset to the test_compress.py
58439ca 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Refactoring: Move torch distributed env variables to dist_utils.py
2e5f776 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Refactoring: move torch distributed variables to dist_utils
6274db5 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Move os.environ["WANDB_DISABLED"] = "true" to dist_utils.py
d942e0a 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Merge branch 'dkorzekwa/e2e_compression_test' into dkorzekwa/llama3_t…
72bdc7a 
…o_decilm_convertion
@danielkorzekwa
Fix broken test - incorrect package names.
f7fe23c 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Improve docs
739f868 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Merge branch 'dkorzekwa/e2e_compression_test' into dkorzekwa/llama3_t…
b06d22b 
…o_decilm_convertion
@danielkorzekwa
Merge branch 'feature/compress' into dkorzekwa/llama3_to_decilm_conve…
18cb88b 
…rtion

Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
Fix import
1033c81 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
kevalmorabia97
kevalmorabia97 reviewed Oct 28, 2025
View reviewed changes
...xperimental/torch/_compress/decilm/converters/test_convert_llama3_config_to_decilm_config.py Outdated
Comment on lines +29 to +31
@pytest.fixture
def project_root_path(request: pytest.FixtureRequest) -> Path:
return Path(request.config.rootpath)
Copy link
Collaborator

@kevalmorabia97 kevalmorabia97 Oct 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can define this in tests/experimental/torch/_compress/conftest.py so it can be reused in both tests

Copy link
Contributor Author

@danielkorzekwa danielkorzekwa Oct 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@danielkorzekwa
create conftest.py with shared test logic for compress tests.
86e04a0 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa
code cleanup
ae61644 
Signed-off-by: Daniel Korzekwa <dkorzekwa@nvidia.com>
@danielkorzekwa danielkorzekwa merged commit cef3655 into feature/compress Oct 29, 2025
21 checks passed
@danielkorzekwa danielkorzekwa deleted the dkorzekwa/llama3_to_decilm_convertion branch October 29, 2025 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kevalmorabia97 kevalmorabia97 kevalmorabia97 approved these changes

@ChenhanYu ChenhanYu Awaiting requested review from ChenhanYu

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@danielkorzekwa @kevalmorabia97