-
Notifications
You must be signed in to change notification settings - Fork 21.4k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Update base for Update on "[2/2] Intel GPU Runtime Upstreaming for Ge…
…nerator" # Motivation According to [[1/2] Intel GPU Runtime Upstreaming for Generator](#118528), as mentioned in [[RFC] Intel GPU Runtime Upstreaming](#114842), the second PR covers the changes under `python frontend`. # Design Currently, it primarily offers geneartor-related APIs, including - `torch.xpu.default_generators` - `torch.xpu.get_rng_state` - `torch.xpu.get_rng_state_all` - `torch.xpu.initial_seed` - `torch.xpu.manual_seed` - `torch.xpu.manual_seed_all` - `torch.xpu.seed` - `torch.xpu.seed_all` - `torch.xpu.set_rng_state` - `torch.xpu.set_rng_state_all` # Additional Context The differences with CUDA: The generator-related frontend python APIs are 1:1 mapping with CUDA. cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]
- Loading branch information
Showing
6,276 changed files
with
4,430 additions
and
9,279 deletions.
The diff you're trying to view is too large. We only load the first 3000 changed files.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
# Configuring the builder. | ||
|
||
## Install prerequisites. | ||
|
||
``` | ||
$ sudo dnf install docker | ||
``` | ||
|
||
## Add services. | ||
|
||
``` | ||
$ sudo cp self-hosted-builder/*.service /etc/systemd/system/ | ||
$ sudo systemctl daemon-reload | ||
``` | ||
|
||
## Download qemu-user-static image | ||
|
||
``` | ||
# sudo docker pull docker.io/iiilinuxibmcom/qemu-user-static:6.1.0-1 | ||
``` | ||
|
||
## Autostart the x86_64 emulation support. | ||
|
||
``` | ||
$ sudo systemctl enable --now qemu-user-static | ||
``` | ||
|
||
## Rebuild the image | ||
|
||
In order to build or update the `iiilinuxibmcom/actions-runner` image, e.g. to get the | ||
latest OS security fixes, use the following commands: | ||
|
||
``` | ||
$ cd self-hosted-builder | ||
$ sudo docker build \ | ||
--build-arg repo=<owner>/<name> \ | ||
--build-arg token=<***> \ | ||
--pull \ | ||
-f actions-runner.Dockerfile \ | ||
-t iiilinuxibmcom/actions-runner \ | ||
. | ||
``` | ||
|
||
If it fails, ensure that selinux doesn't prevent it from working. | ||
In worst case, selinux can be disabled with `setenforce 0`. | ||
|
||
## Autostart the runner. | ||
|
||
``` | ||
$ sudo systemctl enable --now actions-runner@$NAME | ||
``` |
66 changes: 66 additions & 0 deletions
66
.github/scripts/s390x-ci/self-hosted-builder/actions-runner.Dockerfile
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
# Self-Hosted IBM Z Github Actions Runner. | ||
|
||
# Temporary image: amd64 dependencies. | ||
FROM docker.io/amd64/ubuntu:22.04 as ld-prefix | ||
ENV DEBIAN_FRONTEND=noninteractive | ||
RUN apt-get update && apt-get -y install ca-certificates libicu70 libssl3 | ||
|
||
# Main image. | ||
FROM docker.io/s390x/ubuntu:22.04 | ||
|
||
# Packages for pytorch building and testing. | ||
ENV DEBIAN_FRONTEND=noninteractive | ||
RUN apt-get update && apt-get -y install \ | ||
cmake \ | ||
curl \ | ||
gcc \ | ||
git \ | ||
jq \ | ||
libxml2-dev \ | ||
libxslt-dev \ | ||
ninja-build \ | ||
python-is-python3 \ | ||
python3 \ | ||
python3-dev \ | ||
python3-pip \ | ||
pybind11-dev \ | ||
python3-numpy \ | ||
libopenblas-dev \ | ||
liblapack-dev \ | ||
libgloo-dev \ | ||
python3-yaml \ | ||
python3-scipy \ | ||
virtualenv | ||
|
||
# amd64 dependencies. | ||
COPY --from=ld-prefix / /usr/x86_64-linux-gnu/ | ||
RUN ln -fs ../lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 /usr/x86_64-linux-gnu/lib64/ | ||
RUN ln -fs /etc/resolv.conf /usr/x86_64-linux-gnu/etc/ | ||
ENV QEMU_LD_PREFIX=/usr/x86_64-linux-gnu | ||
|
||
# Scripts. | ||
COPY fs/ / | ||
|
||
RUN chmod +x /usr/bin/actions-runner /usr/bin/entrypoint | ||
|
||
# amd64 Github Actions Runner. | ||
RUN useradd -m actions-runner | ||
USER actions-runner | ||
WORKDIR /home/actions-runner | ||
RUN curl -L https://github.com/actions/runner/releases/download/v2.309.0/actions-runner-linux-x64-2.309.0.tar.gz | tar -xz | ||
|
||
# repository | ||
ARG repo | ||
|
||
# repository token | ||
ARG token | ||
|
||
RUN ./config.sh \ | ||
--unattended \ | ||
--url "https://github.com/${repo}" \ | ||
--token "${token}" \ | ||
--no-default-labels \ | ||
--labels self-hosted,linux.s390x | ||
|
||
ENTRYPOINT ["/usr/bin/entrypoint"] | ||
CMD ["/usr/bin/actions-runner"] |
22 changes: 22 additions & 0 deletions
22
.github/scripts/s390x-ci/self-hosted-builder/actions-runner@.service
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
[Unit] | ||
Description=Self-Hosted IBM Z Github Actions Runner | ||
Wants=qemu-user-static | ||
After=qemu-user-static | ||
StartLimitIntervalSec=0 | ||
|
||
[Service] | ||
Type=simple | ||
Restart=always | ||
ExecStartPre=-/usr/bin/docker rm --force actions-runner.%i | ||
ExecStart=/usr/bin/docker run \ | ||
--init \ | ||
--interactive \ | ||
--name=actions-runner.%i \ | ||
--rm \ | ||
iiilinuxibmcom/actions-runner | ||
ExecStop=/bin/sh -c "docker exec actions-runner.%i kill -INT -- -1" | ||
ExecStop=/bin/sh -c "docker wait actions-runner.%i" | ||
ExecStop=/bin/sh -c "docker rm actions-runner.%i" | ||
|
||
[Install] | ||
WantedBy=multi-user.target |
6 changes: 6 additions & 0 deletions
6
.github/scripts/s390x-ci/self-hosted-builder/fs/usr/bin/actions-runner
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
#!/usr/bin/env bash | ||
|
||
set -e -u | ||
|
||
# Run one job. | ||
./run.sh --once |
30 changes: 30 additions & 0 deletions
30
.github/scripts/s390x-ci/self-hosted-builder/fs/usr/bin/entrypoint
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
#!/usr/bin/env bash | ||
|
||
# | ||
# Container entrypoint that waits for all spawned processes. | ||
# | ||
|
||
set -e -u | ||
|
||
# Create a FIFO and start reading from its read end. | ||
tempdir=$(mktemp -d "/tmp/done.XXXXXXXXXX") | ||
trap 'rm -r "$tempdir"' EXIT | ||
done="$tempdir/pipe" | ||
mkfifo "$done" | ||
cat "$done" & waiter=$! | ||
|
||
# Start the workload. Its descendants will inherit the FIFO's write end. | ||
status=0 | ||
if [ "$#" -eq 0 ]; then | ||
bash 9>"$done" || status=$? | ||
else | ||
"$@" 9>"$done" || status=$? | ||
fi | ||
|
||
# When the workload and all of its descendants exit, the FIFO's write end will | ||
# be closed and `cat "$done"` will exit. Wait until it happens. This is needed | ||
# in order to handle SelfUpdater, which the workload may start in background | ||
# before exiting. | ||
wait "$waiter" | ||
|
||
exit "$status" |
11 changes: 11 additions & 0 deletions
11
.github/scripts/s390x-ci/self-hosted-builder/qemu-user-static.service
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
[Unit] | ||
Description=Support for transparent execution of non-native binaries with QEMU user emulation | ||
|
||
[Service] | ||
Type=oneshot | ||
# The source code for iiilinuxibmcom/qemu-user-static is at https://github.com/iii-i/qemu-user-static/tree/v6.1.0-1 | ||
# TODO: replace it with multiarch/qemu-user-static once version >6.1 is available | ||
ExecStart=/usr/bin/docker run --rm --interactive --privileged docker.io/iiilinuxibmcom/qemu-user-static:6.1.0-1 --reset -p yes | ||
|
||
[Install] | ||
WantedBy=multi-user.target |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.