-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update llama-3 PEFT notebook to download model from NGC #9667
Update llama-3 PEFT notebook to download model from NGC #9667
Conversation
shashank3959
commented
Jul 10, 2024
- Updates the llama-3 PEFT tutorial to download the model from NVIDIA NGC to skip conversion
Signed-off-by: Shashank Verma <shashank3959@gmail.com>
Signed-off-by: Shashank Verma <shashank3959@gmail.com>
Signed-off-by: Shashank Verma <shashank3959@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me
I am providing additional copyedits to the file, ReadMe.rst, herein. I was not able to provide inline edits because part of the file was read-only. Line 4 fix punctuation
Line 6 fix punctuation
Line 19 revise for clarity NIM enables seamless deployment of multiple LoRA adapters (referred to as ‘multi-LoRA’) on the same base model. It dynamically loads adapter weights based on incoming requests at runtime. This flexibility allows handling inputs from various tasks or use cases without requiring a unique model for each individual scenario. For further details, consult the NIM documentation for LLMs. Line 24 delete the following sentence, it is sufficient to retain the bullets In order to proceed, ensure that you have met the following requirements: Line 103/109 fix punctuation
Line 110/116 fix grammar To ensure the model store is organized as expected, create a folder named llama3-8b-pubmed-qa and move your .nemo checkpoint there. Line 122/128 revise for clarity Ensure that the LoRA model store directory follows this structure: the model name should be a sub-folder containing the .nemo file. Line 138/144 fix punctuation
Line 170/176 fix grammar The first time you run the command, it will download the model and cache it in $NIM_CACHE_PATH so subsequent deployments are even faster. There are several options to configure NIM other than the ones listed above. You can find a full list in the NIM configuration documentation. Line 173/179 fix punctuation
Line 175/181 fix grammar From another terminal, follow the same instructions as the previous notebook to launch Jupyter Lab, and then navigate to this notebook. Line 178/185 revise for clarity You can use the same NeMo Framework docker container in which Jupyter Lab was previously installed. |
Signed-off-by: Shashank Verma <shashank3959@gmail.com>
Signed-off-by: Shashank Verma <shashank3959@gmail.com>
Signed-off-by: Shashank Verma <shashank3959@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
* Update llama-3 PEFT notebook to download model from NGC Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Fix broken link in llama-3 PEFT tutorial README Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Fix broken code block in llama 3 PEFT tutorial README Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Copy-edits to Llama-3 8B PEFT tutorial README Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Fix broken link Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Minor formatting fixes Signed-off-by: Shashank Verma <shashank3959@gmail.com> --------- Signed-off-by: Shashank Verma <shashank3959@gmail.com>
* Update llama-3 PEFT notebook to download model from NGC Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Fix broken link in llama-3 PEFT tutorial README Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Fix broken code block in llama 3 PEFT tutorial README Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Copy-edits to Llama-3 8B PEFT tutorial README Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Fix broken link Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Minor formatting fixes Signed-off-by: Shashank Verma <shashank3959@gmail.com> --------- Signed-off-by: Shashank Verma <shashank3959@gmail.com>
* Update llama-3 PEFT notebook to download model from NGC Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Fix broken link in llama-3 PEFT tutorial README Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Fix broken code block in llama 3 PEFT tutorial README Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Copy-edits to Llama-3 8B PEFT tutorial README Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Fix broken link Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Minor formatting fixes Signed-off-by: Shashank Verma <shashank3959@gmail.com> --------- Signed-off-by: Shashank Verma <shashank3959@gmail.com>
* Update llama-3 PEFT notebook to download model from NGC Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Fix broken link in llama-3 PEFT tutorial README Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Fix broken code block in llama 3 PEFT tutorial README Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Copy-edits to Llama-3 8B PEFT tutorial README Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Fix broken link Signed-off-by: Shashank Verma <shashank3959@gmail.com> * Minor formatting fixes Signed-off-by: Shashank Verma <shashank3959@gmail.com> --------- Signed-off-by: Shashank Verma <shashank3959@gmail.com> Signed-off-by: Tugrul Konuk <ertkonuk@gmail.com>