Multi node settings for evaluation nemo-run script #13568
Conversation
Signed-off-by: Abhishree <abhishreetm@gmail.com>
Signed-off-by: Abhishree <abhishreetm@gmail.com>
|
@chtruong814 can we fast-merge this once @marta-sd approves ? It just has changes for the example nemo-run script. Thanks! |
Signed-off-by: Abhishree <abhishreetm@gmail.com>
athitten
force-pushed
the
athitten/eval_mnode_runs
branch
from
57e4235 to
0bfb1c5
Compare
|
@jgerh can you please review/approve a small addition to |
|
@chtruong814 getting a link check error on docs unrelated to this PR. Can you please help ? Thanks! |
| executors (your local workstation) and Slurm-based executors like clusters. In this setup, the deploy and evaluate | ||
| processes are launched as two separate jobs with NeMo-Run. The evaluate method waits until the PyTriton server is | ||
| accessible and the model is deployed before starting the evaluations. | ||
|
|
There was a problem hiding this comment.
please revise the file name by replacing the underscore character with a hyphen: change evaluation_doc.rst to evaluation-doc.rst. this is now an seo requirement. Ensure that you also make the same filename change to the toc tree in the index file.
jgerh
left a comment
jgerh
left a comment
There was a problem hiding this comment.
Completed review of change in docs/source/evaluation/evaluation_doc.rst. There were no copyedits needed, but a filename change is needed to evaluation-doc.rst.
Signed-off-by: Abhishree <abhishreetm@gmail.com>
athitten
force-pushed
the
athitten/eval_mnode_runs
branch
from
20cb903 to
f92d409
Compare
Signed-off-by: athitten <athitten@users.noreply.github.com>
* Add multi node settings for eval run Signed-off-by: Abhishree <abhishreetm@gmail.com> * Edit to eval nemo-run script Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add HF_TOKEN to docs Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove underscore in eval doc name Signed-off-by: Abhishree <abhishreetm@gmail.com> * Apply isort and black reformatting Signed-off-by: athitten <athitten@users.noreply.github.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: athitten <athitten@users.noreply.github.com> Co-authored-by: athitten <athitten@users.noreply.github.com> Signed-off-by: jianbinc <shjwudp@gmail.com>
Important
The
Update branchbutton must only be pressed in very rare occassions.An outdated branch is never blocking the merge of a PR.
Please reach out to the automation team before pressing that button.
What does this PR do ?
Minor edits to
scripts/llm/evaluation.pyto support evaluation of model on multi node.Collection: [Note which collection this PR will affect]
Changelog
Usage
# Add a code snippet demonstrating how to use thisGitHub Actions CI
The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.
The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".
Before your PR is "Ready for review"
Pre checks:
PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.
Who can review?
Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.
Additional Information