Skip to content

Conversation

rparolin
Copy link
Collaborator

@rparolin rparolin commented Sep 4, 2025

Problem Statement

Running unit tests for cuda_bindings via python -m pytest tests/ was manifesting the following error on WSL.

[:57](https://nvidia.slack.com/archives/D09BRBYV2JC/p1757008678349519)
____________________________________________________________ ERROR collecting test_cuda.py _____________________________________________________________
ImportError while importing test module '/home/rparolin/cuda-python/cuda_bindings/tests/test_cuda.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../miniconda3/envs/v1/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/test_cuda.py:11: in <module>
    import cuda.bindings.driver as cuda
E   ImportError: /home/rparolin/cuda-python/cuda_bindings/cuda/bindings/driver.cpython-312-x86_64-linux-gnu.so: undefined symbol: _ZSt21__glibcxx_assert_failPKciS0_S0_

Removing name mangling shows this is the internal assert handler called by the stdlibc++ standard library controlled by the preprocessor macro _GLIBCXX_ASSERTIONS.

std::__glibcxx_assert_fail(char const*, int, char const*, char const*)

Looking at the resulting gcc compiler command line used to build the c-extensions you can see we are passing mismatched preprocessor toggles; namely, NDEBUG and _GLIBCXX_ASSERTIONS. NDEBUG disables debug functionality including asserts and then we at the same enable stdlib asserts.

This PR ensures we only enable stdlib asserts when compiling in debug.

@copy-pr-bot
Copy link
Contributor

copy-pr-bot bot commented Sep 4, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@rparolin rparolin requested a review from leofang September 4, 2025 21:25
@rparolin rparolin changed the title Fixing cuda_bindings local build manifesting with GCC-13 on Linux (WSL) Fixing cuda_bindings local build errors manifesting with GCC-13 on Linux (WSL) Sep 4, 2025
@rparolin rparolin marked this pull request as ready for review September 4, 2025 21:50
@rparolin rparolin marked this pull request as draft September 4, 2025 21:50
@rparolin
Copy link
Collaborator Author

rparolin commented Sep 4, 2025

/ok to test 6205570

@rparolin
Copy link
Collaborator Author

rparolin commented Sep 4, 2025

pre-commit.ci autofix

@rparolin rparolin marked this pull request as ready for review September 4, 2025 21:58
@github-actions

This comment has been minimized.

@leofang
Copy link
Member

leofang commented Sep 4, 2025

/ok to test c56fab1

@leofang leofang added bug Something isn't working P1 Medium priority - Should do cuda.bindings Everything related to the cuda.bindings module to-be-backported Trigger the bot to raise a backport PR upon merge labels Sep 4, 2025
@leofang leofang enabled auto-merge (squash) September 4, 2025 22:57
@leofang leofang merged commit 559f749 into main Sep 4, 2025
53 checks passed
@leofang leofang deleted the rparolin/cuda_bindings_local_build_fix branch September 4, 2025 23:40
github-actions bot pushed a commit that referenced this pull request Sep 4, 2025
…Linux (WSL) (#946)

* Fixing cuda_bindings local build manifesting with GCC-13 on Linux (WSL)

* [pre-commit.ci] auto code formatting

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
(cherry picked from commit 559f749)
@github-actions
Copy link

github-actions bot commented Sep 4, 2025

@github-actions
Copy link

github-actions bot commented Sep 4, 2025

Doc Preview CI
Preview removed because the pull request was closed or merged.

@rwgk
Copy link
Collaborator

rwgk commented Sep 5, 2025

Just sharing an observation:

Running unit tests for cuda_bindings via python -m pytest tests/ was manifesting the following error on WSL.

Interesting. I'm using "GCC-13 on Linux (WSL)", too, but never saw that error. An obvious difference at first glance is that I don't use conda very much when running the cuda_bindings unit tests.

leofang pushed a commit that referenced this pull request Sep 5, 2025
…Linux (WSL) (#946) (#947)

* Fixing cuda_bindings local build manifesting with GCC-13 on Linux (WSL)

* [pre-commit.ci] auto code formatting

---------


(cherry picked from commit 559f749)

Co-authored-by: Rob Parolin <rparolin@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@rparolin
Copy link
Collaborator Author

rparolin commented Sep 5, 2025

Just sharing an observation:

Running unit tests for cuda_bindings via python -m pytest tests/ was manifesting the following error on WSL.

Interesting. I'm using "GCC-13 on Linux (WSL)", too, but never saw that error. An obvious difference at first glance is that I don't use conda very much when running the cuda_bindings unit tests.

We may want to do an audit of the binaries publish via pypi and conda. I suspect we unintentionally enabled debug symbols somewhere.

mdboom pushed a commit to mdboom/cuda-python that referenced this pull request Sep 10, 2025
…Linux (WSL) (NVIDIA#946)

* Fixing cuda_bindings local build manifesting with GCC-13 on Linux (WSL)

* [pre-commit.ci] auto code formatting

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@cpcloud cpcloud mentioned this pull request Oct 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working cuda.bindings Everything related to the cuda.bindings module P1 Medium priority - Should do to-be-backported Trigger the bot to raise a backport PR upon merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants