Skip to content

Commit f17e38b

Browse files
authored
Add SMP v2 notebooks using latest PyTorch 2.3.1, TSM 2.5.0 release following new directory structure (#4739)
Rename notebooks and all badge paths to follow new directory structure. Add context parallel example notebook Update model parallel v2 shared_scripts Update pysdk>=2.240 Add FP8 Mixtral notebook Update configs and cleanup
1 parent 329b976 commit f17e38b

37 files changed

+6906
-105
lines changed
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
## Sagemaker Model Parallelism
2+
3+
This directory contains example scripts to train or fine-tune large scale models,
4+
with the Sagemaker distributed model parallelism library.
5+
When using one of the ipynb notebooks within the folders of this directory please
6+
make sure to use the `./shared-scripts/` directory as the source directory when submitting a job.
7+
8+
For example, if one wanted to submit a llama finetune job on Sagemaker using the `/llama_v2/smp-finetuning-llama-fsdp-tp.ipynb`
9+
notebook, they would have to copy that notebook within the `./shared-scripts/` directory to make sure it can access all the accompanied files.
10+
11+
After cloning this repository run the following command to setup a copy of the notebook associated with your desired model into the `./shared-scripts/` directory.
12+
13+
- `cp [RELATIVE PATH TO ipynb] shared-scripts/`
14+
15+
Finally, upload the `./shared-scripts/` directory to a Sagemaker notebook to submit your training/finetuning job.

0 commit comments

Comments
 (0)