Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check if reentrant version of CFITSIO is used #5239

Merged
merged 4 commits into from
Jan 9, 2024

Conversation

szkarpinski
Copy link
Collaborator

@szkarpinski szkarpinski commented Dec 12, 2023

Category:

Bug fix (non-breaking change which fixes an issue)

Description:

CFITSIO is by default compiled in non-reentrant mode, resulting in segfaults when two or more pipelines are active. NVIDIA/DALI_deps#96 enables multithreading and this PR adds a runtime check and a test for that.

Additional information:

Affected modules and functionalities:

Key points relevant for the review:

Tests:

  • Existing tests apply
  • New tests added
    • Python tests
    • GTests
    • Benchmark
    • Other
  • N/A

Checklist

Documentation

  • Existing documentation applies
  • Documentation updated
    • Docstring
    • Doxygen
    • RST
    • Jupyter
    • Other
  • N/A

DALI team only

Requirements

  • Implements new requirements
  • Affects existing requirements
  • N/A

REQ IDs: N/A

JIRA TASK: DALI-3746

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [11485028]: BUILD STARTED

@NVIDIA NVIDIA deleted a comment from dali-automaton Dec 14, 2023
@NVIDIA NVIDIA deleted a comment from dali-automaton Dec 14, 2023
@NVIDIA NVIDIA deleted a comment from dali-automaton Dec 14, 2023
@NVIDIA NVIDIA deleted a comment from dali-automaton Dec 14, 2023
@NVIDIA NVIDIA deleted a comment from dali-automaton Dec 14, 2023
@NVIDIA NVIDIA deleted a comment from dali-automaton Dec 14, 2023
@NVIDIA NVIDIA deleted a comment from dali-automaton Dec 14, 2023
@dali-automaton
Copy link
Collaborator

CI MESSAGE: [11485028]: BUILD PASSED

@szkarpinski szkarpinski marked this pull request as ready for review December 14, 2023 12:53
Comment on lines 61 to 65

DALI_ENFORCE(fits_is_reentrant(),
"Loaded instance of cfitsio library does not support multithreading. "
"Please recompile cfitsio in reentrant mode (--enable-reentrant) "
"or use cfitsio delivered in DALI_deps");
Copy link
Contributor

@mzient mzient Dec 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't we just warn the user and guard the code with a mutex, like this?

std::unique_lock lock(mtx, std::defer_lock);
if (!fits_is_reentrant)
    lock.lock();

...unless we statically link it and ship it with the .whl.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea! Thank you!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added locking to FITS_CALL

@mzient mzient assigned awolant and unassigned klecki Dec 15, 2023
@@ -1 +1 @@
345c534814ffcfd40c2df3fe3d4a00c0d911b94c
FIXME
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert to what's in main - the relevant DALI_deps PR has already been merged.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
@szkarpinski
Copy link
Collaborator Author

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [11957948]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [11957948]: BUILD PASSED

@szkarpinski szkarpinski merged commit 8f4959f into NVIDIA:main Jan 9, 2024
7 checks passed
mzient pushed a commit to mzient/DALI that referenced this pull request Jan 9, 2024
Check if reentrant version of CFITSIO is used. If not, emit a warning and use exclusive lock when calling CFITSIO functions.

---------

Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
mzient pushed a commit to mzient/DALI that referenced this pull request Jan 9, 2024
Check if reentrant version of CFITSIO is used. If not, emit a warning and use exclusive lock when calling CFITSIO functions.

---------

Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants