Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running multiple SDFGs with the same name using CUDA fails with DuplicateDLLError #10

Closed
definelicht opened this issue Nov 6, 2019 · 1 comment

Comments

@definelicht
Copy link
Contributor

Problem

When compiling and running multiple SDFGs with the same name from the same Python executable, the SDFG is usually deleted in between (I assume by garbage collection?), which unloads the dynamic libraries.

However, when CUDA is involved, it seems that the SDFGs are not deleted even when they are no longer references by Python, which results in the loaded library to stick around. In practice, this can result in DuplicateDLLError when trying to load a new SDFG using the same name.

Expected behavior

SDFGs should be cleaned up when they are no longer referenced, and it should be possible to run multiple SDFGs with the same name from the same Python executable without explicitly deleting them.

To reproduce

Run test tests/library/blas_dot.py on the library_nodes branch, but change the initialization of dace.SDFG to always have the same name, then run the test including the cuBLAS runs.

Workaround

Name each SDFG differently, or explicitly call del my_sdfg between executions.

@tbennun
Copy link
Collaborator

tbennun commented Feb 6, 2020

Fixed in #114

@tbennun tbennun closed this as completed Feb 6, 2020
tbennun added a commit that referenced this issue Feb 6, 2020
Reload DLLs on failure (fixes Issue #10)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants