Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: Add GPU builds #193

Merged
merged 3 commits into from May 10, 2020
Merged

ci: Add GPU builds #193

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d241636
ci: Add GPU builds
ddn0 May 7, 2020
6717cbe
Remove unused bliss submodule
ddn0 May 7, 2020
ba57a3e
ci: Checkout submodules
ddn0 May 7, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
122 changes: 108 additions & 14 deletions .circleci/config.yml
View file
Open in desktop
Expand Up @@ -17,10 +17,13 @@ jobs:
- checkout
- run: |
apt-get -q update -y
apt-get -q install -y apt-transport-https ca-certificates gnupg software-properties-common wget
apt-get -q install -y apt-transport-https ca-certificates git gnupg software-properties-common wget
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key 2>/dev/null | apt-key add -
apt-add-repository -y 'deb https://apt.llvm.org/bionic/ llvm-toolchain-bionic-10 main'
apt-get -q update -y
- run: git submodule sync
- run: git submodule update --init
- run: |
apt-get -q install -y \
clang-format-10

Expand All @@ -34,14 +37,17 @@ jobs:
- checkout
- run: |
apt-get -q update -y
apt-get -q install -y apt-transport-https ca-certificates gnupg software-properties-common wget
apt-get -q install -y apt-transport-https ca-certificates git gnupg software-properties-common wget
wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | apt-key add -
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key 2>/dev/null | apt-key add -
apt-add-repository -y 'ppa:ubuntu-toolchain-r/test'
apt-add-repository -y 'deb https://apt.llvm.org/bionic/ llvm-toolchain-bionic-10 main'
apt-add-repository -y 'deb https://apt.kitware.com/ubuntu/ bionic main'
apt-add-repository -y 'ppa:mhier/libboost-latest'
apt-get -q update -y
- run: git submodule sync
- run: git submodule update --init
- run: |
apt-get -q install -y \
clang-10 \
cmake \
Expand Down Expand Up @@ -78,6 +84,10 @@ jobs:
- checkout
- run: |
apt-get -q update -y
apt-get -q install -y git
- run: git submodule sync
- run: git submodule update --init
- run: |
apt-get -q install -y \
cmake \
g++ \
Expand Down Expand Up @@ -108,11 +118,13 @@ jobs:
- checkout
- run: |
apt-get -q update -y
apt-get -q install -y apt-transport-https ca-certificates gnupg software-properties-common wget
apt-get -q install -y apt-transport-https ca-certificates git gnupg software-properties-common wget
wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | apt-key add -
apt-add-repository 'deb https://apt.kitware.com/ubuntu/ bionic main'
apt-get -q update -y

- run: git submodule sync
- run: git submodule update --init
- run: |
apt-get -q install -y \
cmake \
g++ \
Expand All @@ -134,6 +146,62 @@ jobs:
(cd /tmp/build \
&& chown -R runner . \
&& su runner -c "ctest -V --label-regex quick --parallel 2")
"Ubuntu-18.04-cuda10.1-build-only":
docker:
- image: nvidia/cuda:10.1-devel-ubuntu18.04
steps:
- checkout
- run: |
apt-get -q update -y
apt-get -q install -y apt-transport-https ca-certificates git gnupg software-properties-common wget
wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | apt-key add -
apt-add-repository 'deb https://apt.kitware.com/ubuntu/ bionic main'
apt-get -q update -y
- run: git submodule sync
- run: git submodule update --init
- run: |
apt-get -q install -y \
cmake \
g++ \
gcc \
libboost-all-dev \
libeigen3-dev \
libopenmpi-dev \
llvm-7-dev \
openmpi-bin \
ssh

cmake -S . -B /tmp/build -DGALOIS_ENABLE_DIST=ON -DGALOIS_ENABLE_GPU=ON
cmake --build /tmp/build --target input
cmake --build /tmp/build --parallel 2
"Ubuntu-18.04-cuda10.2-build-only":
docker:
- image: nvidia/cuda:10.2-devel-ubuntu18.04
steps:
- checkout
- run: |
apt-get -q update -y
apt-get -q install -y apt-transport-https ca-certificates git gnupg software-properties-common wget
wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | apt-key add -
apt-add-repository 'deb https://apt.kitware.com/ubuntu/ bionic main'
apt-get -q update -y
- run: git submodule sync
- run: git submodule update --init
- run: |
apt-get -q install -y \
cmake \
g++ \
gcc \
libboost-all-dev \
libeigen3-dev \
libopenmpi-dev \
llvm-7-dev \
openmpi-bin \
ssh

cmake -S . -B /tmp/build -DGALOIS_ENABLE_DIST=ON -DGALOIS_ENABLE_GPU=ON
cmake --build /tmp/build --target input
cmake --build /tmp/build --parallel 2
"CentOS-8-gcc":
docker:
- image: centos:8
Expand All @@ -143,6 +211,10 @@ jobs:
# Needs to access PowerTools packages for eigen, so get access to those
yum -y -q install dnf-plugins-core
yum -y -q config-manager --set-enabled PowerTools
yum -y -q install git
- run: git submodule sync
- run: git submodule update --init
- run: |
yum -y -q install \
boost-devel \
cmake \
Expand Down Expand Up @@ -184,6 +256,10 @@ jobs:
# Needs to access PowerTools packages for eigen, so get access to those
yum -y -q install dnf-plugins-core
yum -y -q config-manager --set-enabled PowerTools
yum -y -q install git
- run: git submodule sync
- run: git submodule update --init
- run: |
yum -y -q install \
boost-devel \
eigen3-devel \
Expand All @@ -210,16 +286,20 @@ jobs:
-DCMAKE_CXX_COMPILER=clang++
cmake --build /tmp/build --target input
cmake --build /tmp/build --parallel 2
cd /tmp/build \
(cd /tmp/build \
&& chown -R runner . \
&& su runner -c "ctest -V --label-regex quick --parallel 2"
&& su runner -c "ctest -V --label-regex quick --parallel 2")
"Arch":
docker:
- image: archlinux/base
steps:
- checkout
- run: |
pacman -Sy
pacman -q -S --noconfirm git
- run: git submodule sync
- run: git submodule update --init
- run: |
# NB(ddn): make requires libffi but its package doesn't depend on it.
pacman -q -S --noconfirm \
boost \
Expand All @@ -246,6 +326,10 @@ jobs:
- image: alpine:latest
steps:
- checkout
- run: |
apk add --no-cache --no-progress git
- run: git submodule sync
- run: git submodule update --init
- run: |
apk add --no-cache --no-progress \
boost-dev \
Expand Down Expand Up @@ -275,6 +359,10 @@ jobs:
- image: fedora:latest
steps:
- checkout
- run: |
yum -y -q install git
- run: git submodule sync
- run: git submodule update --init
- run: |
yum -y -q install \
boost-devel \
Expand Down Expand Up @@ -305,6 +393,10 @@ jobs:
- image: fedora:latest
steps:
- checkout
- run: |
yum -y -q install git
- run: git submodule sync
- run: git submodule update --init
- run: |
yum -y -q install \
boost-devel \
Expand All @@ -316,12 +408,12 @@ jobs:
make \
openmpi-devel \
zlib-devel
# Get the "module" function set up before loading MPI.

chmod 755 /root
useradd runner
mkdir -p /tmp/build

# Get the "module" function set up before loading MPI.
. /etc/bashrc
module load mpi
cmake \
Expand All @@ -340,13 +432,15 @@ workflows:
version: 2
build:
jobs:
- "Alpine"
- "Arch"
- "CentOS-8-clang"
- "CentOS-8-gcc"
- "CheckFormat"
- "Sanitize"
- "Debian"
- "Ubuntu-18.04"
- "CentOS-8-gcc"
- "CentOS-8-clang"
- "Arch"
- "Alpine"
- "Fedora-gcc"
- "Fedora-clang"
- "Fedora-gcc"
- "Sanitize"
- "Ubuntu-18.04"
- "Ubuntu-18.04-cuda10.2-build-only"
- "Ubuntu-18.04-cuda10.1-build-only"
3 changes: 0 additions & 3 deletions .gitmodules
View file
Open in desktop
@@ -1,6 +1,3 @@
[submodule "bliss"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this removed? I thought this was required for pangolin.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is already vendored in external directly rather than as a submodule. The submodule itself was just something @chenxuhao made to track bliss upstream but it seems simpler to avoid inserting an intermediary.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's better it's not a submodule but that is not related to this PR. As long as pangolin builds don't break, it's fine to be included in this PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, the reason the commit is here is because we have to enable submodules for GPU builds, so I took the opportunity to clean up our submodule configuration, which has these unused entries.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, makes sense.

path = bliss
url = https://github.com/chenxuhao/bliss.git
[submodule "moderngpu"]
path = external/moderngpu
url = https://github.com/moderngpu/moderngpu.git
Expand Down
5 changes: 1 addition & 4 deletions .travis.yml
View file
Open in desktop
Expand Up @@ -2,11 +2,8 @@ dist: bionic

language: c++

# At the moment, our LCI submodule has an additional submodule within it
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. This comment is outdated now that the LCI submodule is gone.

# that refers to some non-public copy of googletest. To avoid failing when
# cloning Galois, we have to turn off cloning submodules.
git:
submodules: false
submodules: true

matrix:
include:
Expand Down