Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support NVidia HPC SDK (PGI) #2475

Merged
merged 27 commits into from Sep 12, 2020
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
74d3442
Added guards to the includes
Sep 9, 2020
9b53c43
fix: missing include
henryiii Sep 9, 2020
be79610
tests: split up prealoc tests
henryiii Sep 10, 2020
a1cd376
fix: PGI compiler fix
henryiii Sep 10, 2020
20ea47f
fix: PGI void_t only
henryiii Sep 10, 2020
9b24083
fix: try to appease nvcc
henryiii Sep 10, 2020
a75dae0
ci: better ordering for slow tests
henryiii Sep 10, 2020
48687d1
ci: minor improvements to testing
henryiii Sep 10, 2020
f5ebec5
ci: Add NumPy to testing
henryiii Sep 11, 2020
a996fe9
ci: Eigen generates CUDA warnings / PGI errors
henryiii Sep 11, 2020
acb951d
Added CentOS7 back for a moment
Sep 11, 2020
2ac94e3
Fix YAML
henryiii Sep 11, 2020
e181db5
ci: runs-on missing
henryiii Sep 11, 2020
37e717d
centos7 is missing pytest
henryiii Sep 11, 2020
7533160
ci: use C++11 on CentOS 7
henryiii Sep 11, 2020
a9a7e1d
ci: test something else
henryiii Sep 11, 2020
5332d74
Try just adding flags on CentOS 7
henryiii Sep 11, 2020
4469de8
fix: CentOS 7
henryiii Sep 11, 2020
73127c5
refactor: move include to shared location
henryiii Sep 11, 2020
cba6265
Added verbose flag
Sep 11, 2020
7280383
Try to use system cmake3 on CI
Sep 11, 2020
fd62422
Try to use system cmake3 on CI, attempt2
Sep 11, 2020
f1478d6
Try to use system cmake3 on CI, attempt3
Sep 11, 2020
0a99b16
tests: not finding pytest should be a warning, not a fatal error
henryiii Sep 11, 2020
287f74e
tests: cleanup
henryiii Sep 11, 2020
663683f
Weird issue?
henryiii Sep 11, 2020
20c901c
fix: final polish
henryiii Sep 12, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
23 changes: 23 additions & 0 deletions .github/CONTRIBUTING.md
Expand Up @@ -164,6 +164,29 @@ name, pre-commit):
pre-commit install
```

### Build recipes

This builds with the Intel compiler (assuming it is in your path, along with a
recent CMake and Python 3):

```bash
python3 -m venv venv
. venv/bin/activate
pip install pytest
cmake -S . -B build-intel -DCMAKE_CXX_COMPILER=$(which icpc) -DDOWNLOAD_CATCH=ON -DDOWNLOAD_EIGEN=ON -DPYBIND11_WERROR=ON
```

This will test the PGI compilers:

```bash
docker run --rm -it -v $PWD:/pybind11 nvcr.io/hpc/pgi-compilers:ce
apt-get update && apt-get install -y python3-dev python3-pip python3-pytest
wget -qO- "https://cmake.org/files/v3.18/cmake-3.18.1-Linux-x86_64.tar.gz" | tar --strip-components=1 -xz -C /usr/local
cmake -S pybind11/ -B build
cmake --build build
```


[pre-commit]: https://pre-commit.com
[pybind11.readthedocs.org]: http://pybind11.readthedocs.org/en/latest
[issue tracker]: https://github.com/pybind/pybind11/issues
Expand Down
126 changes: 104 additions & 22 deletions .github/workflows/ci.yml
Expand Up @@ -69,7 +69,6 @@ jobs:
python: 3.9-dev
arch: x64


name: "🐍 ${{ matrix.python }} • ${{ matrix.runs-on }} • ${{ matrix.arch }} ${{ matrix.args }}"
runs-on: ${{ matrix.runs-on }}
continue-on-error: ${{ endsWith(matrix.python, 'dev') }}
Expand Down Expand Up @@ -196,6 +195,109 @@ jobs:
- name: Interface test
run: cmake --build build --target test_cmake_build


cuda:
runs-on: ubuntu-latest
name: "🐍 3.8 • CUDA 11 • Ubuntu 20.04"
container: nvidia/cuda:11.0-devel-ubuntu20.04

steps:
- uses: actions/checkout@v2

# tzdata will try to ask for the timezone, so set the DEBIAN_FRONTEND
- name: Install 🐍 3
run: apt-get update && DEBIAN_FRONTEND="noninteractive" apt-get install -y cmake git python3-dev python3-pytest python3-numpy

- name: Configure
run: cmake -S . -B build -DPYBIND11_CUDA_TESTS=ON -DPYBIND11_WERROR=ON -DDOWNLOAD_CATCH=ON

- name: Build
run: cmake --build build -j2 -v

- name: Python tests
run: cmake --build build --target pytest


centos-nvhpc:
runs-on: ubuntu-latest
name: "🐍 3 • CentOS8/PGI • x64"
container: "centos:8"

steps:
- uses: actions/checkout@v2

- name: Add Python 3 and a few requirements
run: yum update -y && yum install -y git python3-devel python3-numpy python3-pytest make environment-modules

- name: Install CMake with pip
run: |
python3 -m pip install --upgrade pip
python3 -m pip install cmake --prefer-binary

- name: Install NVidia HPC SDK
run: yum -y install https://developer.download.nvidia.com/hpc-sdk/nvhpc-20-7-20.7-1.x86_64.rpm https://developer.download.nvidia.com/hpc-sdk/nvhpc-2020-20.7-1.x86_64.rpm

- name: Configure
shell: bash
run: |
source /etc/profile.d/modules.sh
module load /opt/nvidia/hpc_sdk/modulefiles/nvhpc/20.7
cmake -S . -B build -DDOWNLOAD_CATCH=ON -DCMAKE_CXX_STANDARD=14 -DPYTHON_EXECUTABLE=$(python3 -c "import sys; print(sys.executable)")

- name: Build
run: cmake --build build -j 2

- name: Python tests
run: cmake --build build --target pytest

- name: C++ tests
run: cmake --build build --target cpptest

- name: Interface test
run: cmake --build build --target test_cmake_build

centos-nvhpc7:
runs-on: ubuntu-latest
name: "🐍 3 • CentOS7/PGI • x64"
container: "centos:7"

steps:
- uses: actions/checkout@v2

- name: Add Python 3 and a few requirements
run: yum update -y && yum install -y git python3-devel make environment-modules

- name: Install CMake with pip
run: |
python3 -m pip install --upgrade pip
python3 -m pip install pytest cmake --prefer-binary

- name: Install NVidia HPC SDK
run: yum -y install https://developer.download.nvidia.com/hpc-sdk/nvhpc-20-7-20.7-1.x86_64.rpm https://developer.download.nvidia.com/hpc-sdk/nvhpc-2020-20.7-1.x86_64.rpm

- name: Configure
shell: bash
run: |
source /etc/profile.d/modules.sh
module load /opt/nvidia/hpc_sdk/modulefiles/nvhpc/20.7
cmake -S . -B build -DDOWNLOAD_CATCH=ON \
-DCMAKE_CXX_STANDARD=11 \
-DPYTHON_EXECUTABLE=$(python3 -c "import sys; print(sys.executable)") \
-DCMAKE_CXX_FLAGS="-Wc,--pending_instantiations=0" \
-DPYBIND11_TEST_FILTER="test_smart_ptr.cpp;test_virtual_functions.cpp"

- name: Build
run: cmake --build build -j 2

- name: Python tests
run: cmake --build build --target pytest

- name: C++ tests
run: cmake --build build --target cpptest

- name: Interface test
run: cmake --build build --target test_cmake_build

gcc:
runs-on: ubuntu-latest
strategy:
Expand Down Expand Up @@ -243,6 +345,7 @@ jobs:
- name: Interface test
run: cmake --build build --target test_cmake_build


centos:
runs-on: ubuntu-latest
strategy:
Expand Down Expand Up @@ -289,27 +392,6 @@ jobs:
- name: Interface test
run: cmake --build build --target test_cmake_build

cuda:
runs-on: ubuntu-latest
name: "🐍 3.8 • CUDA 11 • Ubuntu 20.04"
container: nvidia/cuda:11.0-devel-ubuntu20.04

steps:
- uses: actions/checkout@v2

# tzdata will try to ask for the timezone, so set the DEBIAN_FRONTEND
- name: Install 🐍 3
run: apt-get update && DEBIAN_FRONTEND="noninteractive" apt-get install -y cmake python3-dev python3-pytest

- name: Configure
run: cmake -S . -B build -DPYBIND11_CUDA_TESTS=ON -DPYBIND11_WERROR=ON -DDOWNLOAD_CATCH=ON

- name: Build
run: cmake --build build -j2 -v

- name: Python tests
run: cmake --build build --target pytest


install-classic:
name: "🐍 3.5 • Debian • x86 • Install"
Expand Down
1 change: 1 addition & 0 deletions README.md
Expand Up @@ -99,6 +99,7 @@ In addition to the core functionality, pybind11 provides some extra goodies:
v2.0 and a [workaround][intel-15-workaround])
5. Cygwin/GCC (tested on 2.5.1)
6. NVCC (CUDA 11 tested)
7. NVIDIA PGI (20.7 tested)

## About

Expand Down
9 changes: 9 additions & 0 deletions include/pybind11/detail/common.h
Expand Up @@ -154,6 +154,7 @@
#include <vector>
#include <string>
#include <stdexcept>
#include <exception>
#include <unordered_set>
#include <unordered_map>
#include <memory>
Expand Down Expand Up @@ -501,8 +502,16 @@ template <bool... Bs> using select_indices = typename select_indices_impl<index_
template <bool B> using bool_constant = std::integral_constant<bool, B>;
template <typename T> struct negation : bool_constant<!T::value> { };

// PGI cannot detect operator delete with the "compatible" void_t impl, so
// using the new one (C++14 defect, so generally works on newer compilers, even
// if not in C++17 mode)
#if defined(__PGIC__)
template<typename... > using void_t = void;
#else
template <typename...> struct void_t_impl { using type = void; };
template <typename... Ts> using void_t = typename void_t_impl<Ts...>::type;
#endif


/// Compile-time all/any/none of that check the boolean value of all template types
#if defined(__cpp_fold_expressions) && !(defined(_MSC_VER) && (_MSC_VER < 1916))
Expand Down
48 changes: 38 additions & 10 deletions tests/CMakeLists.txt
Expand Up @@ -16,6 +16,29 @@ else()
cmake_policy(VERSION 3.18)
endif()

# Only needed for CMake < 3.5 support
include(CMakeParseArguments)

# Filter out items; print an optional message if any items filtered
#
# Usage:
# pybind11_filter_tests(LISTNAME file1.cpp file2.cpp ... MESSAGE "")
#
macro(PYBIND11_FILTER_TESTS LISTNAME)
henryiii marked this conversation as resolved.
Show resolved Hide resolved
cmake_parse_arguments(ARG "" "MESSAGE" "" ${ARGN})
set(PYBIND11_FILTER_TESTS_FOUND OFF)
foreach(filename IN LISTS ARG_UNPARSED_ARGUMENTS)
list(FIND ${LISTNAME} ${filename} _FILE_FOUND)
if(_FILE_FOUND GREATER -1)
list(REMOVE_AT ${LISTNAME} ${_FILE_FOUND})
set(PYBIND11_FILTER_TESTS_FOUND ON)
endif()
endforeach()
if(PYBIND11_FILTER_TESTS_FOUND AND ARG_MESSAGE)
message(STATUS "${ARG_MESSAGE}")
endif()
endmacro()

# New Python support
if(DEFINED Python_EXECUTABLE)
set(PYTHON_EXECUTABLE "${Python_EXECUTABLE}")
Expand All @@ -34,6 +57,9 @@ option(PYBIND11_CUDA_TESTS "Enable building CUDA tests (requires CMake 3.12+)" O
set(PYBIND11_TEST_OVERRIDE
""
CACHE STRING "Tests from ;-separated list of *.cpp files will be built instead of all tests")
set(PYBIND11_TEST_FILTER
""
CACHE STRING "Tests from ;-separated list of *.cpp files will be removed from all tests")

if(CMAKE_CURRENT_SOURCE_DIR STREQUAL CMAKE_SOURCE_DIR)
# We're being loaded directly, i.e. not via add_subdirectory, so make this
Expand Down Expand Up @@ -106,21 +132,23 @@ if(PYBIND11_TEST_OVERRIDE)
set(PYBIND11_TEST_FILES ${PYBIND11_TEST_OVERRIDE})
endif()

# Skip test_async for Python < 3.5
list(FIND PYBIND11_TEST_FILES test_async.cpp PYBIND11_TEST_FILES_ASYNC_I)
if((PYBIND11_TEST_FILES_ASYNC_I GREATER -1) AND (PYTHON_VERSION VERSION_LESS 3.5))
message(STATUS "Skipping test_async because Python version ${PYTHON_VERSION} < 3.5")
list(REMOVE_AT PYBIND11_TEST_FILES ${PYBIND11_TEST_FILES_ASYNC_I})
# You can also filter tests:
if(PYBIND11_TEST_FILTER)
pybind11_filter_tests(PYBIND11_TEST_FILES ${PYBIND11_TEST_FILTER})
endif()

if(PYTHON_VERSION VERSION_LESS 3.5)
pybind11_filter_tests(PYBIND11_TEST_FILES test_async.cpp MESSAGE
"Skipping test_async on Python 2")
endif()

# Skip tests for CUDA check:
# /pybind11/tests/test_constants_and_functions.cpp(125):
# error: incompatible exception specifications
list(FIND PYBIND11_TEST_FILES test_constants_and_functions.cpp PYBIND11_TEST_FILES_CAF_I)
if((PYBIND11_TEST_FILES_CAF_I GREATER -1) AND PYBIND11_CUDA_TESTS)
message(
STATUS "Skipping test_constants_and_functions due to incompatible exception specifications")
list(REMOVE_AT PYBIND11_TEST_FILES ${PYBIND11_TEST_FILES_CAF_I})
if(PYBIND11_CUDA_TESTS)
pybind11_filter_tests(
PYBIND11_TEST_FILES test_constants_and_functions.cpp MESSAGE
"Skipping test_constants_and_functions due to incompatible exception specifications")
endif()

string(REPLACE ".cpp" ".py" PYBIND11_PYTEST_FILES "${PYBIND11_TEST_FILES}")
Expand Down
8 changes: 5 additions & 3 deletions tests/test_factory_constructors.cpp
Expand Up @@ -11,6 +11,7 @@
#include "pybind11_tests.h"
#include "constructor_stats.h"
#include <cmath>
#include <new>

// Classes for testing python construction via C++ factory function:
// Not publicly constructible, copyable, or movable:
Expand Down Expand Up @@ -298,11 +299,12 @@ TEST_SUBMODULE(factory_constructors, m) {
static void *operator new(size_t, void *p) { py::print("noisy placement new"); return p; }
static void operator delete(void *p, size_t) { py::print("noisy delete"); ::operator delete(p); }
static void operator delete(void *, void *) { py::print("noisy placement delete"); }
#if defined(_MSC_VER) && _MSC_VER < 1910
# if defined(_MSC_VER) && _MSC_VER < 1910
// MSVC 2015 bug: the above "noisy delete" isn't invoked (fixed in MSVC 2017)
static void operator delete(void *p) { py::print("noisy delete"); ::operator delete(p); }
#endif
static void operator delete(void *p) { py::print("noisy delete"); ::operator delete(p); }
# endif
};

py::class_<NoisyAlloc>(m, "NoisyAlloc")
// Since these overloads have the same number of arguments, the dispatcher will try each of
// them until the arguments convert. Thus we can get a pre-allocation here when passing a
Expand Down
17 changes: 15 additions & 2 deletions tests/test_factory_constructors.py
Expand Up @@ -336,10 +336,10 @@ def strip_comments(s):
return re.sub(r'\s+#.*', '', s)


def test_reallocations(capture, msg):
def test_reallocation_a(capture, msg):
"""When the constructor is overloaded, previous overloads can require a preallocated value.
This test makes sure that such preallocated values only happen when they might be necessary,
and that they are deallocated properly"""
and that they are deallocated properly."""

pytest.gc_collect()

Expand All @@ -353,6 +353,9 @@ def test_reallocations(capture, msg):
~NoisyAlloc()
noisy delete
"""


def test_reallocation_b(capture, msg):
with capture:
create_and_destroy(1.5)
assert msg(capture) == strip_comments("""
Expand All @@ -365,6 +368,8 @@ def test_reallocations(capture, msg):
noisy delete # operator delete
""")


def test_reallocation_c(capture, msg):
with capture:
create_and_destroy(2, 3)
assert msg(capture) == strip_comments("""
Expand All @@ -375,6 +380,8 @@ def test_reallocations(capture, msg):
noisy delete # operator delete
""")


def test_reallocation_d(capture, msg):
with capture:
create_and_destroy(2.5, 3)
assert msg(capture) == strip_comments("""
Expand All @@ -386,6 +393,8 @@ def test_reallocations(capture, msg):
noisy delete # operator delete
""")


def test_reallocation_e(capture, msg):
with capture:
create_and_destroy(3.5, 4.5)
assert msg(capture) == strip_comments("""
Expand All @@ -397,6 +406,8 @@ def test_reallocations(capture, msg):
noisy delete # operator delete
""")


def test_reallocation_f(capture, msg):
with capture:
create_and_destroy(4, 0.5)
assert msg(capture) == strip_comments("""
Expand All @@ -409,6 +420,8 @@ def test_reallocations(capture, msg):
noisy delete # operator delete
""")


def test_reallocation_g(capture, msg):
with capture:
create_and_destroy(5, "hi")
assert msg(capture) == strip_comments("""
Expand Down
5 changes: 3 additions & 2 deletions tests/test_smart_ptr.py
@@ -1,7 +1,8 @@
# -*- coding: utf-8 -*-
import pytest
from pybind11_tests import smart_ptr as m
from pybind11_tests import ConstructorStats

m = pytest.importorskip("pybind11_tests.smart_ptr")
from pybind11_tests import ConstructorStats # noqa: E402


def test_smart_ptr(capture):
Expand Down