Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update container to 22.05 #4329

Merged
merged 12 commits into from
Jun 7, 2022
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

ARG BASE_IMAGE=nvcr.io/nvidia/pytorch:22.04-py3
ARG BASE_IMAGE=nvcr.io/nvidia/pytorch:22.05-py3


# build an image that includes only the nemo dependencies, ensures that dependencies
Expand Down
8 changes: 7 additions & 1 deletion Jenkinsfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
pipeline {
agent {
docker {
image 'nvcr.io/nvidia/pytorch:22.04-py3'
image 'nvcr.io/nvidia/pytorch:22.05-py3'
args '--device=/dev/nvidia0 --gpus all -e TRANSFORMERS_OFFLINE=1 --user 0:128 -v /home/TestData:/home/TestData -v $HOME/.cache:/root/.cache --shm-size=8g'
}
}
Expand All @@ -12,6 +12,12 @@ pipeline {

stages {

stage('Add git safe directory'){
steps{
sh 'git config --global --add safe.directory /var/lib/jenkins/workspace/NeMo_$GIT_BRANCH'
}
}

stage('nvidia-smi'){
steps{
sh 'nvidia-smi'
Expand Down
6 changes: 3 additions & 3 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ Megatron GPT training requires NVIDIA Apex to be installed.

git clone https://github.com/NVIDIA/apex
cd apex
git checkout 9263bc8c6c16555bd55dd759f1a1b8c0cd187d10
git checkout 5d8c8a8eedaf567d56f0762a45431baf9c0e800e
pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" --global-option="--fast_layer_norm" ./

Docker containers:
Expand All @@ -214,13 +214,13 @@ To build a nemo container with Dockerfile from a branch, please run
DOCKER_BUILDKIT=1 docker build -f Dockerfile -t nemo:latest .


If you chose to work with main branch, we recommend using NVIDIA's PyTorch container version 22.04-py3 and then installing from GitHub.
If you chose to work with main branch, we recommend using NVIDIA's PyTorch container version 22.05-py3 and then installing from GitHub.

.. code-block:: bash

docker run --gpus all -it --rm -v <nemo_github_folder>:/NeMo --shm-size=8g \
-p 8888:8888 -p 6006:6006 --ulimit memlock=-1 --ulimit \
stack=67108864 --device=/dev/snd nvcr.io/nvidia/pytorch:22.04-py3
stack=67108864 --device=/dev/snd nvcr.io/nvidia/pytorch:22.05-py3

Examples
--------
Expand Down
2 changes: 1 addition & 1 deletion ci.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ spec:
path: /vol/scratch1/scratch.okuchaiev_blossom
containers:
- name: cuda
image: nvcr.io/nvidia/pytorch:22.04-py3
image: nvcr.io/nvidia/pytorch:22.05-py3
command:
- cat
volumeMounts:
Expand Down