NVCC + C++17 #1103

ax3l · 2021-09-15T00:28:22Z

Work-around a build issue with NVCC in C++17 builds.

include/openPMD/backend/Attributable.hpp(437):
error #289: no instance of constructor "openPMD::Attribute::Attribute" matches the argument list
            argument types are: (std::__cxx11::string)
          detected during instantiation of "__nv_bool openPMD::AttributableInterface::setAttribute(const std::__cxx11::string &, T) [with T=std::__cxx11::string]"

from

inline bool
AttributableInterface::setAttribute( std::string const & key, char const value[] )
{
    return this->setAttribute(key, std::string(value));
}

Seen with WarpX and:

CUDA 11.0.2 + GCC 8.3.0 (x86_64, Cori GPU@NERSC)
CUDA 11.0.2 + GCC 7.5.0 (x86_64, GitHub Actions with Ubuntu 18.04 C++17, CMake 3.17+ ECP-WarpX/WarpX#2300)

Not seen with WarpX and:

CUDA 11.0.3 (NVCC 11.0.221) + GCC 9.3.0 (ppc64le, Summit@OLCF)

Work-around a build issue with NVCC in C++17 builds. ``` include/openPMD/backend/Attributable.hpp(437): error openPMD#289: no instance of constructor "openPMD::Attribute::Attribute" matches the argument list argument types are: (std::__cxx11::string) detected during instantiation of "__nv_bool openPMD::AttributableInterface::setAttribute(const std::__cxx11::string &, T) [with T=std::__cxx11::string]" ``` from ``` inline bool AttributableInterface::setAttribute( std::string const & key, char const value[] ) { return this->setAttribute(key, std::string(value)); } ``` Seen with: - NVCC 11.0.2 + GCC 8.3.0 - NVCC 11.0.2 + GCC 7.5.0

ax3l · 2021-09-15T00:33:12Z

include/openPMD/backend/Attributable.hpp

+        Attribute::resource val(value);
+        it->second = Attribute(std::move(val));


Just FYI, this triggers the same issue:

Suggested change

Attribute::resource val(value);

it->second = Attribute(std::move(val));

it->second = Attribute(std::move(value));

Could you test if this one works:

Attribute::resource val(std::move(value)); it->second = Attribute(std::move(vale));

or even:

it->second = Attribute(Attribute::resource(std::move(value)));

That way would at least be without copying

Also, let's put a comment there to make it clear that this is purposefully strange

Good idea. I have no access to Cori due to maintenance today and will test it tomorrow.

Tested it now, the explicit construction of the resource in place works as a work-around, too :)
Added also a comment :)

* NVCC + C++17 Work-around a build issue with NVCC in C++17 builds. ``` include/openPMD/backend/Attributable.hpp(437): error openPMD#289: no instance of constructor "openPMD::Attribute::Attribute" matches the argument list argument types are: (std::__cxx11::string) detected during instantiation of "__nv_bool openPMD::AttributableInterface::setAttribute(const std::__cxx11::string &, T) [with T=std::__cxx11::string]" ``` from ``` inline bool AttributableInterface::setAttribute( std::string const & key, char const value[] ) { return this->setAttribute(key, std::string(value)); } ``` Seen with: - NVCC 11.0.2 + GCC 8.3.0 - NVCC 11.0.2 + GCC 7.5.0 * NVCC 11.0.2 C++17 work-around: Add Comment

* Read: time/dt also in long double (#1096) * Python: time/dt round-trip Test writing and reading time and dt on an iteration via properties. * Fix: Iteration read of long double time Support reading of `dt` and `time` attributes if they are of type `long double`. (openPMD standard: all `floatX` supported) * Executables: CXX_STANDARD/EXTENSIONS (#1102) Set `CXX_EXTENSIONS OFF` and `CXX_STANDARD_REQUIRED ON` for created executables. This mitigates issues with NVCC 11.0 and C++17 builds seen as added `-std=gnu++17` flags that lead to ``` nvcc fatal : Value 'gnu++17' is not defined for option 'std' ``` when using `nvcc` as CXX compiler directly. * Doc: More Locations -DPython_EXECUTABLE (#1104) Mention the `-DPython_EXECUTABLE` twice more in build examples. * NVCC + C++17 (#1103) * NVCC + C++17 Work-around a build issue with NVCC in C++17 builds. ``` include/openPMD/backend/Attributable.hpp(437): error #289: no instance of constructor "openPMD::Attribute::Attribute" matches the argument list argument types are: (std::__cxx11::string) detected during instantiation of "__nv_bool openPMD::AttributableInterface::setAttribute(const std::__cxx11::string &, T) [with T=std::__cxx11::string]" ``` from ``` inline bool AttributableInterface::setAttribute( std::string const & key, char const value[] ) { return this->setAttribute(key, std::string(value)); } ``` Seen with: - NVCC 11.0.2 + GCC 8.3.0 - NVCC 11.0.2 + GCC 7.5.0 * NVCC 11.0.2 C++17 work-around: Add Comment * Lazy parsing: Make findable in docs and use in openpmd-ls (#1111) * Use deferred iteration parsing in openpmd-ls * Make lazy/deferred parsing searchable * Add a way to search for usesteps key * HDF5: Document HDF5_USE_FILE_LOCKING (#1106) Document a HDF5 read work-around that we currently need on OLCF Jupyter (https://jupyter.olcf.ornl.gov), due to a mounting issue of GPFS in the Jupyter serice (OLCFHELP-3685). From the HDF5 1.10.1 Release Notes: ``` Other New Features and Enhancements =================================== Library ------- - Added a mechanism for disabling the SWMR file locking scheme. The file locking calls used in HDF5 1.10.0 (including patch1) will fail when the underlying file system does not support file locking or where locks have been disabled. To disable all file locking operations, an environment variable named HDF5_USE_FILE_LOCKING can be set to the five-character string 'FALSE'. This does not fundamentally change HDF5 library operation (aside from initial file open/create, SWMR is lock-free), but users will have to be more careful about opening files to avoid problematic access patterns (i.e.: multiple writers) that the file locking was designed to prevent. Additionally, the error message that is emitted when file lock operations set errno to ENOSYS (typical when file locking has been disabled) has been updated to describe the problem and potential resolution better. (DER, 2016/10/26, HDFFV-9918) ``` This also exists as a compilation option for HDF5 in CMake, where it defaults to ``TRUE`` by default, which is also what distributions/ package managers ship. Disabling from Bash: ```bash export HDF5_USE_FILE_LOCKING=FALSE ``` Disabling from Python: ```py import os os.environ['HDF5_USE_FILE_LOCKING'] = "FALSE" ``` * Avoid object slicing when deriving from Series class (#1107) * Make Series class final * Use private constructor to avoid object slicing * Doc: OMPI_MCA_io Control (#1114) Document OpenMPI MPI-I/O backend control. We have documented this long in #446. * openPMD.hpp: Include auxiliary StringManip (#1124) Include this, handy functions. * CXX Std: Remember <variant> Impl. (#1128) We use `<variant>` or `<mpark/variant.hpp>` in our public API interface for datatypes, depending on the C++ standard. This pull request makes sure that the same implementation is used in downstream code, even if the C++ standard is switched. This avoids ABI issues when, e.g., using a C++14 built openPMD-api in a C++17 downstream code. * Spack: No More `load -r` (#1125) The `-r` argument was removed from `spack load` and is now implied. * Fix AppVeyor: Python Executable (#1127) * GH Action: Add MSVC & ClangCL on Win * Fix AppVeyor: Python Executable * Avoid mismatching system Python and Conda Python * Conda: Fix Numpy * CMake: Skip Pipe Test Written in a too special way, we cannot assume SH is always present * Test 8b (Bench Read Parallel): Support Variable encoding, Fix Bugs (#1131) * added support to read variable encoding, plus fixed some bugs * fixed style * Update examples/8b_benchmark_read_parallel.cpp remove commented out code Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update examples/8b_benchmark_read_parallel.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update examples/8b_benchmark_read_parallel.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update examples/8b_benchmark_read_parallel.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update examples/8b_benchmark_read_parallel.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update examples/8b_benchmark_read_parallel.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update examples/8b_benchmark_read_parallel.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update examples/8b_benchmark_read_parallel.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update examples/8b_benchmark_read_parallel.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update examples/8b_benchmark_read_parallel.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update examples/8b_benchmark_read_parallel.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * removed commented line * updated 8b env option Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * HDF5 I/O optimizations (#1129) * Include HDF5 optimization options * Fix code style check * Fix validations and include checks * Fix style check * Remove unecessary strict check * Update documentation with HDF5 tuning options * Update contributions * Fix Guards for H5Pset_all_coll_metadata* * MPI Guard: H5Pset_all_coll_metadata* * Remove duplicated variable Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Include known issues section for HDF5 (#1132) * Update known issues with HDF5 and collective metadata operations * Fix rst link and tiny typo * Add targeted bugfix releases. Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Include check for paged allocation (#1133) * Include check for paged allocation * Update ParallelHDF5IOHandler.cpp * libfabric 1.6+: Document SST Work-Arounds (#1134) * libfabric 1.6+: Document SST Work-Arounds Document work-arounds for libfabric 1.6+ on Cray systems when using data staging / streaming with ADIOS2 SST. Co-authored-by: Franz Pöschel <franz.poeschel@gmail.com> * Fix: Read Inconsistent Zero Pads (#1118) * [Draft] Fix: Read Inconsistent Zero Pads Some codes mess up the zero-padding in `fileBased` encoding, e.g., when specifying padding to 5 digits but creating >100'000 output steps. Files like those cannot yet be parsed and fell back to no padding, which fails to open the file: ``` openpmd_00000.h5 openpmd_02000.h5 openpmd_101000.h5 openpmd_01000.h5 openpmd_100000.h5 openpmd_104000.h5 ``` Error: ``` RuntimeError: [HDF5] Failed to open HDF5 file diags/diag1/openpmd_0.h5 ``` * Revert previous changes except for test Parse iteration numbers that are longer than their padding Read inconsistent zero padding * Overflow Padding: Read Test * Warn if the prefix does end in a digit * Fix: Don't let oversize numbers accidentally bump the padding * Update test * Issue warnings on misleading patterns also when writing * Minor Style Update Co-authored-by: Franz Pöschel <franz.poeschel@gmail.com> * Release: 0.14.3 Co-authored-by: Franz Pöschel <franz.poeschel@gmail.com> Co-authored-by: guj <guj@users.noreply.github.com> Co-authored-by: Jean Luca Bez <jeanlucabez@gmail.com> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov>

The `Attribute` constructors with implicit variant conversion sometimes do not work in ICC 19.1.2. ``` openPMD-api/src/RecordComponent.cpp(226): error: no instance of constructor "openPMD::Attribute::Attribute" matches the argument list argument types are: (openPMD::Extent) Attribute a(getExtent()); ^ openPMD-api/include/openPMD/backend/Attribute.hpp(50): note: this candidate was rejected because arguments do not match class Attribute : ^ openPMD-api/include/openPMD/backend/Attribute.hpp(50): note: this candidate was rejected because arguments do not match class Attribute : ^ openPMD-api/include/openPMD/backend/Attribute.hpp(79): note: this candidate was rejected because arguments do not match Attribute(resource r) : Variant(std::move(r)) ^ ``` Same work-around as for NVCC in openPMD#1103

The `Attribute` constructors with implicit variant conversion sometimes do not work in ICC 19.1.2. ``` openPMD-api/src/RecordComponent.cpp(226): error: no instance of constructor "openPMD::Attribute::Attribute" matches the argument list argument types are: (openPMD::Extent) Attribute a(getExtent()); ^ openPMD-api/include/openPMD/backend/Attribute.hpp(50): note: this candidate was rejected because arguments do not match class Attribute : ^ openPMD-api/include/openPMD/backend/Attribute.hpp(50): note: this candidate was rejected because arguments do not match class Attribute : ^ openPMD-api/include/openPMD/backend/Attribute.hpp(79): note: this candidate was rejected because arguments do not match Attribute(resource r) : Variant(std::move(r)) ^ ``` Same work-around as for NVCC in #1103

The `Attribute` constructors with implicit variant conversion sometimes do not work in ICC 19.1.2. ``` openPMD-api/src/RecordComponent.cpp(226): error: no instance of constructor "openPMD::Attribute::Attribute" matches the argument list argument types are: (openPMD::Extent) Attribute a(getExtent()); ^ openPMD-api/include/openPMD/backend/Attribute.hpp(50): note: this candidate was rejected because arguments do not match class Attribute : ^ openPMD-api/include/openPMD/backend/Attribute.hpp(50): note: this candidate was rejected because arguments do not match class Attribute : ^ openPMD-api/include/openPMD/backend/Attribute.hpp(79): note: this candidate was rejected because arguments do not match Attribute(resource r) : Variant(std::move(r)) ^ ``` Same work-around as for NVCC in openPMD#1103

* Add `cmake/` to MANIFEST.in Forgot to add in #1128, which is then missing in Python sdist files for pip from-source builds. * Add failing test * Read gridSpacing also from long doubles * Python: 3.10 Add support for Python 3.10. * CI: Python 3.10 on Alpine Linux musllinux uses the musl libc on an Alpine stack, especially for docker-ish containers. Let's add coverage while also covering CPython 3.10 * Fix: Python Iteration Repr Typo Fix a typo in the `__repr__` of `Iteration` in Python. * Remove invalid records from our data structure entirely Until now, the half-parsed records were not deleted. Our sanity checks in `Series::~Series()` caught up on that and threw errors. * Fix doxygen? No idea why this PR triggers that error * Don't apply compression operators multipe times Happened so far in variable-based iteration encoding * ICC 19.1.2: CXX17 Work-Arounds (Variant) The `Attribute` constructors with implicit variant conversion sometimes do not work in ICC 19.1.2. ``` openPMD-api/src/RecordComponent.cpp(226): error: no instance of constructor "openPMD::Attribute::Attribute" matches the argument list argument types are: (openPMD::Extent) Attribute a(getExtent()); ^ openPMD-api/include/openPMD/backend/Attribute.hpp(50): note: this candidate was rejected because arguments do not match class Attribute : ^ openPMD-api/include/openPMD/backend/Attribute.hpp(50): note: this candidate was rejected because arguments do not match class Attribute : ^ openPMD-api/include/openPMD/backend/Attribute.hpp(79): note: this candidate was rejected because arguments do not match Attribute(resource r) : Variant(std::move(r)) ^ ``` Same work-around as for NVCC in #1103 * CMake 3.22+: Policy CMP0127 Fix a warning with CMake 3.22+. We use simple syntax in cmake_dependent_option, so we are compatible with the extended syntax in CMake 3.22+: https://cmake.org/cmake/help/v3.22/policy/CMP0127.html * Docs: Minor Warnings Fix minor warnings in the docs: - double reference (make 2nd ref annonymous) - misaligned reference * Docs: Fix HDF5 JSON File Fix misnamed filename and misnamed prefix in text. * Docs: Update Citation & Add BibTeX Add `.bib` blocks for easier citation of standard and API. * Docs: Fix CLI Highlighting Fix a highlighting issue for `openPMD-ls` in the docs. Explicitly use `python3` for module calls. * CMake: Warning Flags First in CXXFLAGS If we append them, then we overwrite flags like `-Wno-...` from environment variables. * ADIOS2 fixes: Incompatibilites w/ `master` Referes to post-`2.7.1` `master` branch of ADIOS2. Catch new ADIOS2 error type: string getCompressionOperator Dirtily catch ADIOS2 exception in automatically_deactivate_span Cleanly destruct a Series that errored during flushing Problem was that IO tasks that do not return cleanly but throw an exception must be removed from the IO queue, otherwise the task will be tried again during destruction. Usually, this means that anything will happen. SerialIOTest: Remove Comments (resolved) Print warning when removing task from IO queue std::string and fix warning * Automatically deactivate span-based API when operators are present * Add .pre-commit-config.yaml * Release: 0.14.4 Co-authored-by: Franz Pöschel <franz.poeschel@gmail.com>

ax3l added the workaround label Sep 15, 2021

ax3l added this to the 0.14.3 milestone Sep 15, 2021

ax3l requested a review from franzpoeschel September 15, 2021 00:28

ax3l assigned franzpoeschel Sep 15, 2021

ax3l commented Sep 15, 2021

View reviewed changes

NVCC 11.0.2 C++17 work-around: Add Comment

284512c

ax3l force-pushed the fix-nvccCXX17 branch from d9cba87 to 284512c Compare September 16, 2021 16:45

franzpoeschel approved these changes Sep 22, 2021

View reviewed changes

franzpoeschel merged commit 81fe373 into openPMD:dev Sep 22, 2021

ax3l deleted the fix-nvccCXX17 branch September 22, 2021 18:26

ax3l mentioned this pull request Dec 8, 2021

ICC 19.1.2: CXX17 Work-Arounds (Variant) #1157

Merged

ax3l added bug affects latest release labels Dec 8, 2021

This was referenced Jan 27, 2022

Fix Attribute constructor error on NVCC #1184

Merged

C++17: Remove Earlier EDG Attribute Work-Arounds #1185

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NVCC + C++17 #1103

NVCC + C++17 #1103

ax3l commented Sep 15, 2021 •

edited

Loading

ax3l Sep 15, 2021 •

edited

Loading

franzpoeschel Sep 15, 2021 •

edited

Loading

ax3l Sep 15, 2021

ax3l Sep 16, 2021

		Attribute::resource val(value);
		it->second = Attribute(std::move(val));

	Attribute::resource val(value);
	it->second = Attribute(std::move(val));
	it->second = Attribute(std::move(value));

NVCC + C++17 #1103

NVCC + C++17 #1103

Conversation

ax3l commented Sep 15, 2021 • edited Loading

ax3l Sep 15, 2021 • edited Loading

Choose a reason for hiding this comment

franzpoeschel Sep 15, 2021 • edited Loading

Choose a reason for hiding this comment

ax3l Sep 15, 2021

Choose a reason for hiding this comment

ax3l Sep 16, 2021

Choose a reason for hiding this comment

ax3l commented Sep 15, 2021 •

edited

Loading

ax3l Sep 15, 2021 •

edited

Loading

franzpoeschel Sep 15, 2021 •

edited

Loading