Allow to override PyTorch git revision and set it to v2.1.0-rc2 for 2.1 releases #5543

mateuszlewko · 2023-09-06T10:08:06Z

This change allows to overwrite pytorch_git_rev in the artifacts.auto.tfvars. I saw that you may want to use a different revision for pytorch.

Additionally, I cleaned up arguments passed to the Dockerfile, as only python_version is used. The rest of the parameters are passed to ansible_vars build arg, which is then passed directly to ansible.

In the diff below you can see: "pytorch_git_rev":"v2.1.0-rc2", and that some build args were removed.

Terraform plan:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.versioned_builds["r1.12_3.7_cuda_11.2"].module.cloud_build.google_cloudbuild_trigger.trigger will be updated in-place
  ~ resource "google_cloudbuild_trigger" "trigger" {
        id                 = "projects/tpu-pytorch-releases/locations/us-central1/triggers/83d9fe75-563d-4c4d-8a20-33b09f9fdb17"
        name               = "r1-12-3-7-cuda-11-2"
        tags               = []
        # (10 unchanged attributes hidden)

      ~ build {
            tags          = []
            # (3 unchanged attributes hidden)

          ~ step {
              ~ args       = [
                    "-c",
                  - "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=accelerator=cuda --build-arg=arch=amd64 --build-arg=bundle_libtpu=1 --build-arg=cuda_version=11.2 --build-arg=git_tag=v1.12.0 --build-arg=package_version=1.12 --build-arg=python_version=3.7 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.2\",\"git_tag\":\"v1.12.0\",\"package_version\":\"1.12\",\"python_version\":\"3.7\",\"pytorch_git_rev\":\"v1.12.0\",\"xla_git_rev\":\"v1.12.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r1.12_3.7_cuda_11.2)\" -t=local_image",
                  + "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=python_version=3.7 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.2\",\"git_tag\":\"v1.12.0\",\"package_version\":\"1.12\",\"python_version\":\"3.7\",\"pytorch_git_rev\":\"v1.12.0\",\"xla_git_rev\":\"v1.12.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r1.12_3.7_cuda_11.2)\" -t=local_image",
                ]
                id         = "build_xla_docker_image"
                name       = "gcr.io/cloud-builders/docker"
                # (5 unchanged attributes hidden)
            }

            # (6 unchanged blocks hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # module.versioned_builds["r1.13_3.7_cuda_11.2"].module.cloud_build.google_cloudbuild_trigger.trigger will be updated in-place
  ~ resource "google_cloudbuild_trigger" "trigger" {
        id                 = "projects/tpu-pytorch-releases/locations/us-central1/triggers/9c7d2e24-701e-4fe4-94fe-1a5bf57a4aae"
        name               = "r1-13-3-7-cuda-11-2"
        tags               = []
        # (10 unchanged attributes hidden)

      ~ build {
            tags          = []
            # (3 unchanged attributes hidden)

          ~ step {
              ~ args       = [
                    "-c",
                  - "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=accelerator=cuda --build-arg=arch=amd64 --build-arg=bundle_libtpu=1 --build-arg=cuda_version=11.2 --build-arg=git_tag=v1.13.0 --build-arg=package_version=1.13 --build-arg=python_version=3.7 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.2\",\"git_tag\":\"v1.13.0\",\"package_version\":\"1.13\",\"python_version\":\"3.7\",\"pytorch_git_rev\":\"v1.13.0\",\"xla_git_rev\":\"v1.13.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r1.13_3.7_cuda_11.2)\" -t=local_image",
                  + "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=python_version=3.7 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.2\",\"git_tag\":\"v1.13.0\",\"package_version\":\"1.13\",\"python_version\":\"3.7\",\"pytorch_git_rev\":\"v1.13.0\",\"xla_git_rev\":\"v1.13.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r1.13_3.7_cuda_11.2)\" -t=local_image",
                ]
                id         = "build_xla_docker_image"
                name       = "gcr.io/cloud-builders/docker"
                # (5 unchanged attributes hidden)
            }

            # (6 unchanged blocks hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # module.versioned_builds["r1.13_3.8_cuda_11.2"].module.cloud_build.google_cloudbuild_trigger.trigger will be updated in-place
  ~ resource "google_cloudbuild_trigger" "trigger" {
        id                 = "projects/tpu-pytorch-releases/locations/us-central1/triggers/ebb0bce0-f86d-46bd-8064-dfcdae031d96"
        name               = "r1-13-3-8-cuda-11-2"
        tags               = []
        # (10 unchanged attributes hidden)

      ~ build {
            tags          = []
            # (3 unchanged attributes hidden)

          ~ step {
              ~ args       = [
                    "-c",
                  - "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=accelerator=cuda --build-arg=arch=amd64 --build-arg=bundle_libtpu=1 --build-arg=cuda_version=11.2 --build-arg=git_tag=v1.13.0 --build-arg=package_version=1.13 --build-arg=python_version=3.8 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.2\",\"git_tag\":\"v1.13.0\",\"package_version\":\"1.13\",\"python_version\":\"3.8\",\"pytorch_git_rev\":\"v1.13.0\",\"xla_git_rev\":\"v1.13.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r1.13_3.8_cuda_11.2)\" -t=local_image",
                  + "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=python_version=3.8 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.2\",\"git_tag\":\"v1.13.0\",\"package_version\":\"1.13\",\"python_version\":\"3.8\",\"pytorch_git_rev\":\"v1.13.0\",\"xla_git_rev\":\"v1.13.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r1.13_3.8_cuda_11.2)\" -t=local_image",
                ]
                id         = "build_xla_docker_image"
                name       = "gcr.io/cloud-builders/docker"
                # (5 unchanged attributes hidden)
            }

            # (6 unchanged blocks hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # module.versioned_builds["r1.13_3.8_tpuvm"].module.cloud_build.google_cloudbuild_trigger.trigger will be updated in-place
  ~ resource "google_cloudbuild_trigger" "trigger" {
        id                 = "projects/tpu-pytorch-releases/locations/us-central1/triggers/6c01b84d-28fb-4128-9b2a-a903298e234f"
        name               = "r1-13-3-8-tpuvm"
        tags               = []
        # (10 unchanged attributes hidden)

      ~ build {
            tags          = []
            # (3 unchanged attributes hidden)

          ~ step {
              ~ args       = [
                    "-c",
                  - "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=accelerator=tpu --build-arg=arch=amd64 --build-arg=bundle_libtpu=1 --build-arg=cuda_version=11.8 --build-arg=git_tag=v1.13.0 --build-arg=package_version=1.13 --build-arg=python_version=3.8 --build-arg=ansible_vars='{\"accelerator\":\"tpu\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.8\",\"git_tag\":\"v1.13.0\",\"package_version\":\"1.13\",\"python_version\":\"3.8\",\"pytorch_git_rev\":\"v1.13.0\",\"xla_git_rev\":\"v1.13.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r1.13_3.8_tpuvm)\" -t=local_image",
                  + "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=python_version=3.8 --build-arg=ansible_vars='{\"accelerator\":\"tpu\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.8\",\"git_tag\":\"v1.13.0\",\"package_version\":\"1.13\",\"python_version\":\"3.8\",\"pytorch_git_rev\":\"v1.13.0\",\"xla_git_rev\":\"v1.13.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r1.13_3.8_tpuvm)\" -t=local_image",
                ]
                id         = "build_xla_docker_image"
                name       = "gcr.io/cloud-builders/docker"
                # (5 unchanged attributes hidden)
            }

            # (6 unchanged blocks hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # module.versioned_builds["r2.0_3.10_cuda_11.8"].module.cloud_build.google_cloudbuild_trigger.trigger will be updated in-place
  ~ resource "google_cloudbuild_trigger" "trigger" {
        id                 = "projects/tpu-pytorch-releases/locations/us-central1/triggers/98c54ed4-d414-48be-a1b8-031dea702b96"
        name               = "r2-0-3-10-cuda-11-8"
        tags               = []
        # (10 unchanged attributes hidden)

      ~ build {
            tags          = []
            # (3 unchanged attributes hidden)

          ~ step {
              ~ args       = [
                    "-c",
                  - "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=accelerator=cuda --build-arg=arch=amd64 --build-arg=bundle_libtpu=1 --build-arg=cuda_version=11.8 --build-arg=git_tag=v2.0.0 --build-arg=package_version=2.0 --build-arg=python_version=3.10 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.8\",\"git_tag\":\"v2.0.0\",\"package_version\":\"2.0\",\"python_version\":\"3.10\",\"pytorch_git_rev\":\"v2.0.0\",\"xla_git_rev\":\"v2.0.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.0_3.10_cuda_11.8)\" -t=local_image",
                  + "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=python_version=3.10 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.8\",\"git_tag\":\"v2.0.0\",\"package_version\":\"2.0\",\"python_version\":\"3.10\",\"pytorch_git_rev\":\"v2.0.0\",\"xla_git_rev\":\"v2.0.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.0_3.10_cuda_11.8)\" -t=local_image",
                ]
                id         = "build_xla_docker_image"
                name       = "gcr.io/cloud-builders/docker"
                # (5 unchanged attributes hidden)
            }

            # (6 unchanged blocks hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # module.versioned_builds["r2.0_3.8_cuda_11.7"].module.cloud_build.google_cloudbuild_trigger.trigger will be updated in-place
  ~ resource "google_cloudbuild_trigger" "trigger" {
        id                 = "projects/tpu-pytorch-releases/locations/us-central1/triggers/1151b31d-04c8-4dbb-b1c8-be7d0eab6fed"
        name               = "r2-0-3-8-cuda-11-7"
        tags               = []
        # (10 unchanged attributes hidden)

      ~ build {
            tags          = []
            # (3 unchanged attributes hidden)

          ~ step {
              ~ args       = [
                    "-c",
                  - "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=accelerator=cuda --build-arg=arch=amd64 --build-arg=bundle_libtpu=1 --build-arg=cuda_version=11.7 --build-arg=git_tag=v2.0.0 --build-arg=package_version=2.0 --build-arg=python_version=3.8 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.7\",\"git_tag\":\"v2.0.0\",\"package_version\":\"2.0\",\"python_version\":\"3.8\",\"pytorch_git_rev\":\"v2.0.0\",\"xla_git_rev\":\"v2.0.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.0_3.8_cuda_11.7)\" -t=local_image",
                  + "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=python_version=3.8 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.7\",\"git_tag\":\"v2.0.0\",\"package_version\":\"2.0\",\"python_version\":\"3.8\",\"pytorch_git_rev\":\"v2.0.0\",\"xla_git_rev\":\"v2.0.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.0_3.8_cuda_11.7)\" -t=local_image",
                ]
                id         = "build_xla_docker_image"
                name       = "gcr.io/cloud-builders/docker"
                # (5 unchanged attributes hidden)
            }

            # (6 unchanged blocks hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # module.versioned_builds["r2.0_3.8_cuda_11.8"].module.cloud_build.google_cloudbuild_trigger.trigger will be updated in-place
  ~ resource "google_cloudbuild_trigger" "trigger" {
        id                 = "projects/tpu-pytorch-releases/locations/us-central1/triggers/5ecbf793-5290-41d4-a99c-7055cadd06ca"
        name               = "r2-0-3-8-cuda-11-8"
        tags               = []
        # (10 unchanged attributes hidden)

      ~ build {
            tags          = []
            # (3 unchanged attributes hidden)

          ~ step {
              ~ args       = [
                    "-c",
                  - "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=accelerator=cuda --build-arg=arch=amd64 --build-arg=bundle_libtpu=1 --build-arg=cuda_version=11.8 --build-arg=git_tag=v2.0.0 --build-arg=package_version=2.0 --build-arg=python_version=3.8 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.8\",\"git_tag\":\"v2.0.0\",\"package_version\":\"2.0\",\"python_version\":\"3.8\",\"pytorch_git_rev\":\"v2.0.0\",\"xla_git_rev\":\"v2.0.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.0_3.8_cuda_11.8)\" -t=local_image",
                  + "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=python_version=3.8 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.8\",\"git_tag\":\"v2.0.0\",\"package_version\":\"2.0\",\"python_version\":\"3.8\",\"pytorch_git_rev\":\"v2.0.0\",\"xla_git_rev\":\"v2.0.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.0_3.8_cuda_11.8)\" -t=local_image",
                ]
                id         = "build_xla_docker_image"
                name       = "gcr.io/cloud-builders/docker"
                # (5 unchanged attributes hidden)
            }

            # (6 unchanged blocks hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # module.versioned_builds["r2.0_3.8_tpuvm"].module.cloud_build.google_cloudbuild_trigger.trigger will be updated in-place
  ~ resource "google_cloudbuild_trigger" "trigger" {
        id                 = "projects/tpu-pytorch-releases/locations/us-central1/triggers/200aa1c6-a7a2-4630-8715-aeff337a5da0"
        name               = "r2-0-3-8-tpuvm"
        tags               = []
        # (10 unchanged attributes hidden)

      ~ build {
            tags          = []
            # (3 unchanged attributes hidden)

          ~ step {
              ~ args       = [
                    "-c",
                  - "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=accelerator=tpu --build-arg=arch=amd64 --build-arg=bundle_libtpu=1 --build-arg=cuda_version=11.8 --build-arg=git_tag=v2.0.0 --build-arg=package_version=2.0 --build-arg=python_version=3.8 --build-arg=ansible_vars='{\"accelerator\":\"tpu\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.8\",\"git_tag\":\"v2.0.0\",\"package_version\":\"2.0\",\"python_version\":\"3.8\",\"pytorch_git_rev\":\"v2.0.0\",\"xla_git_rev\":\"v2.0.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.0_3.8_tpuvm)\" -t=local_image",
                  + "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=python_version=3.8 --build-arg=ansible_vars='{\"accelerator\":\"tpu\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.8\",\"git_tag\":\"v2.0.0\",\"package_version\":\"2.0\",\"python_version\":\"3.8\",\"pytorch_git_rev\":\"v2.0.0\",\"xla_git_rev\":\"v2.0.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.0_3.8_tpuvm)\" -t=local_image",
                ]
                id         = "build_xla_docker_image"
                name       = "gcr.io/cloud-builders/docker"
                # (5 unchanged attributes hidden)
            }

            # (6 unchanged blocks hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # module.versioned_builds["r2.1_3.10_cuda_11.8"].module.cloud_build.google_cloudbuild_trigger.trigger will be updated in-place
  ~ resource "google_cloudbuild_trigger" "trigger" {
        id                 = "projects/tpu-pytorch-releases/locations/us-central1/triggers/3a9c9bc6-42f8-42c0-bff3-91c00e55fd51"
        name               = "r2-1-3-10-cuda-11-8"
        tags               = []
        # (10 unchanged attributes hidden)

      ~ build {
            tags          = []
            # (3 unchanged attributes hidden)

          ~ step {
              ~ args       = [
                    "-c",
                  - "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=accelerator=cuda --build-arg=arch=amd64 --build-arg=bundle_libtpu=1 --build-arg=cuda_version=11.8 --build-arg=git_tag=v2.1.0 --build-arg=package_version=2.1 --build-arg=python_version=3.10 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.8\",\"git_tag\":\"v2.1.0\",\"package_version\":\"2.1\",\"python_version\":\"3.10\",\"pytorch_git_rev\":\"v2.1.0\",\"xla_git_rev\":\"v2.1.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.1_3.10_cuda_11.8)\" -t=local_image",
                  + "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=python_version=3.10 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.8\",\"git_tag\":\"v2.1.0\",\"package_version\":\"2.1\",\"python_version\":\"3.10\",\"pytorch_git_rev\":\"v2.1.0-rc2\",\"xla_git_rev\":\"v2.1.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.1_3.10_cuda_11.8)\" -t=local_image",
                ]
                id         = "build_xla_docker_image"
                name       = "gcr.io/cloud-builders/docker"
                # (5 unchanged attributes hidden)
            }

            # (6 unchanged blocks hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # module.versioned_builds["r2.1_3.10_tpuvm"].module.cloud_build.google_cloudbuild_trigger.trigger will be updated in-place
  ~ resource "google_cloudbuild_trigger" "trigger" {
        id                 = "projects/tpu-pytorch-releases/locations/us-central1/triggers/52d653bf-ab2c-4819-90a7-9c0b5545c67c"
        name               = "r2-1-3-10-tpuvm"
        tags               = []
        # (10 unchanged attributes hidden)

      ~ build {
            tags          = []
            # (3 unchanged attributes hidden)

          ~ step {
              ~ args       = [
                    "-c",
                  - "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=accelerator=tpu --build-arg=arch=amd64 --build-arg=bundle_libtpu=0 --build-arg=cuda_version=11.8 --build-arg=git_tag=v2.1.0 --build-arg=package_version=2.1 --build-arg=python_version=3.10 --build-arg=ansible_vars='{\"accelerator\":\"tpu\",\"arch\":\"amd64\",\"bundle_libtpu\":\"0\",\"cuda_version\":\"11.8\",\"git_tag\":\"v2.1.0\",\"package_version\":\"2.1\",\"python_version\":\"3.10\",\"pytorch_git_rev\":\"v2.1.0\",\"xla_git_rev\":\"v2.1.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.1_3.10_tpuvm)\" -t=local_image",
                  + "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=python_version=3.10 --build-arg=ansible_vars='{\"accelerator\":\"tpu\",\"arch\":\"amd64\",\"bundle_libtpu\":\"0\",\"cuda_version\":\"11.8\",\"git_tag\":\"v2.1.0\",\"package_version\":\"2.1\",\"python_version\":\"3.10\",\"pytorch_git_rev\":\"v2.1.0-rc2\",\"xla_git_rev\":\"v2.1.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.1_3.10_tpuvm)\" -t=local_image",
                ]
                id         = "build_xla_docker_image"
                name       = "gcr.io/cloud-builders/docker"
                # (5 unchanged attributes hidden)
            }

            # (6 unchanged blocks hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # module.versioned_builds["r2.1_3.8_cuda_11.8"].module.cloud_build.google_cloudbuild_trigger.trigger will be updated in-place
  ~ resource "google_cloudbuild_trigger" "trigger" {
        id                 = "projects/tpu-pytorch-releases/locations/us-central1/triggers/a5fe77d5-41af-42db-bcf8-d95c4a1617c6"
        name               = "r2-1-3-8-cuda-11-8"
        tags               = []
        # (10 unchanged attributes hidden)

      ~ build {
            tags          = []
            # (3 unchanged attributes hidden)

          ~ step {
              ~ args       = [
                    "-c",
                  - "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=accelerator=cuda --build-arg=arch=amd64 --build-arg=bundle_libtpu=1 --build-arg=cuda_version=11.8 --build-arg=git_tag=v2.1.0 --build-arg=package_version=2.1 --build-arg=python_version=3.8 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.8\",\"git_tag\":\"v2.1.0\",\"package_version\":\"2.1\",\"python_version\":\"3.8\",\"pytorch_git_rev\":\"v2.1.0\",\"xla_git_rev\":\"v2.1.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.1_3.8_cuda_11.8)\" -t=local_image",
                  + "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=python_version=3.8 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.8\",\"git_tag\":\"v2.1.0\",\"package_version\":\"2.1\",\"python_version\":\"3.8\",\"pytorch_git_rev\":\"v2.1.0-rc2\",\"xla_git_rev\":\"v2.1.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.1_3.8_cuda_11.8)\" -t=local_image",
                ]
                id         = "build_xla_docker_image"
                name       = "gcr.io/cloud-builders/docker"
                # (5 unchanged attributes hidden)
            }

            # (6 unchanged blocks hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # module.versioned_builds["r2.1_3.8_cuda_12.0"].module.cloud_build.google_cloudbuild_trigger.trigger will be updated in-place
  ~ resource "google_cloudbuild_trigger" "trigger" {
        id                 = "projects/tpu-pytorch-releases/locations/us-central1/triggers/af81363f-2460-4bac-86eb-e412e070ee14"
        name               = "r2-1-3-8-cuda-12-0"
        tags               = []
        # (10 unchanged attributes hidden)

      ~ build {
            tags          = []
            # (3 unchanged attributes hidden)

          ~ step {
              ~ args       = [
                    "-c",
                  - "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=accelerator=cuda --build-arg=arch=amd64 --build-arg=bundle_libtpu=1 --build-arg=cuda_version=12.0 --build-arg=git_tag=v2.1.0 --build-arg=package_version=2.1 --build-arg=python_version=3.8 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"12.0\",\"git_tag\":\"v2.1.0\",\"package_version\":\"2.1\",\"python_version\":\"3.8\",\"pytorch_git_rev\":\"v2.1.0\",\"xla_git_rev\":\"v2.1.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.1_3.8_cuda_12.0)\" -t=local_image",
                  + "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=python_version=3.8 --build-arg=ansible_vars='{\"accelerator\":\"cuda\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"12.0\",\"git_tag\":\"v2.1.0\",\"package_version\":\"2.1\",\"python_version\":\"3.8\",\"pytorch_git_rev\":\"v2.1.0-rc2\",\"xla_git_rev\":\"v2.1.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.1_3.8_cuda_12.0)\" -t=local_image",
                ]
                id         = "build_xla_docker_image"
                name       = "gcr.io/cloud-builders/docker"
                # (5 unchanged attributes hidden)
            }

            # (6 unchanged blocks hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # module.versioned_builds["r2.1_3.8_tpuvm"].module.cloud_build.google_cloudbuild_trigger.trigger will be updated in-place
  ~ resource "google_cloudbuild_trigger" "trigger" {
        id                 = "projects/tpu-pytorch-releases/locations/us-central1/triggers/beb9db69-7419-48a4-97ee-48ba8d1df319"
        name               = "r2-1-3-8-tpuvm"
        tags               = []
        # (10 unchanged attributes hidden)

      ~ build {
            tags          = []
            # (3 unchanged attributes hidden)

          ~ step {
              ~ args       = [
                    "-c",
                  - "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=accelerator=tpu --build-arg=arch=amd64 --build-arg=bundle_libtpu=1 --build-arg=cuda_version=11.8 --build-arg=git_tag=v2.1.0 --build-arg=package_version=2.1 --build-arg=python_version=3.8 --build-arg=ansible_vars='{\"accelerator\":\"tpu\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.8\",\"git_tag\":\"v2.1.0\",\"package_version\":\"2.1\",\"python_version\":\"3.8\",\"pytorch_git_rev\":\"v2.1.0\",\"xla_git_rev\":\"v2.1.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.1_3.8_tpuvm)\" -t=local_image",
                  + "docker build --progress=plain --network=cloudbuild -f=Dockerfile . --build-arg=python_version=3.8 --build-arg=ansible_vars='{\"accelerator\":\"tpu\",\"arch\":\"amd64\",\"bundle_libtpu\":\"1\",\"cuda_version\":\"11.8\",\"git_tag\":\"v2.1.0\",\"package_version\":\"2.1\",\"python_version\":\"3.8\",\"pytorch_git_rev\":\"v2.1.0-rc2\",\"xla_git_rev\":\"v2.1.0\"}' -t=\"us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/xla:$(echo r2.1_3.8_tpuvm)\" -t=local_image",
                ]
                id         = "build_xla_docker_image"
                name       = "gcr.io/cloud-builders/docker"
                # (5 unchanged attributes hidden)
            }

            # (6 unchanged blocks hidden)
        }

        # (2 unchanged blocks hidden)
    }

Plan: 0 to add, 13 to change, 0 to destroy.

….1 releases

will-cromar

Thanks!

To tag our own builds as rc2 to match upstream, would we update the package_version to 2.1.0rc2?

mateuszlewko · 2023-09-06T15:22:02Z

We can, should I apply that change?

…

On Wed, Sep 6, 2023, 17:13 Will Cromar ***@***.***> wrote: ***@***.**** approved this pull request. Thanks! To tag our own builds as rc2 to match upstream, would we update the package_version to 2.1.0rc2? — Reply to this email directly, view it on GitHub <#5543 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABPEWDPA5KHWNSZNBVGHZVDXZCHH5ANCNFSM6AAAAAA4NCOYPI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

will-cromar · 2023-09-06T15:25:27Z

Yeah, please do

mateuszlewko · 2023-09-06T16:19:00Z

Done. I also re-run 2.1 triggers.

Allow to override PyTorch git revision and set it to v2.1.0-rc2 for 2…

9874dfa

….1 releases

mateuszlewko requested review from JackCaoG, will-cromar and wonjoo-wj September 6, 2023 10:08

mateuszlewko requested review from stgpetrovic and yeounoh as code owners September 6, 2023 10:08

will-cromar approved these changes Sep 6, 2023

View reviewed changes

Change 2.1 package version to 2.1.0rc2

716ec94

mateuszlewko merged commit 3f978df into master Sep 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Allow to override PyTorch git revision and set it to v2.1.0-rc2 for 2.1 releases #5543

Allow to override PyTorch git revision and set it to v2.1.0-rc2 for 2.1 releases #5543

Uh oh!

mateuszlewko commented Sep 6, 2023

Uh oh!

will-cromar left a comment

Uh oh!

mateuszlewko commented Sep 6, 2023 via email

Uh oh!

will-cromar commented Sep 6, 2023

Uh oh!

mateuszlewko commented Sep 6, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Allow to override PyTorch git revision and set it to v2.1.0-rc2 for 2.1 releases #5543

Allow to override PyTorch git revision and set it to v2.1.0-rc2 for 2.1 releases #5543

Uh oh!

Conversation

mateuszlewko commented Sep 6, 2023

Uh oh!

will-cromar left a comment

Choose a reason for hiding this comment

Uh oh!

mateuszlewko commented Sep 6, 2023 via email

Uh oh!

will-cromar commented Sep 6, 2023

Uh oh!

mateuszlewko commented Sep 6, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants