Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sure that the relase of python memory from the DALI tensor happens inside GIL #5101

Merged
merged 1 commit into from
Oct 18, 2023

Conversation

JanuszL
Copy link
Contributor

@JanuszL JanuszL commented Oct 17, 2023

  • no-copy mode of the DALI tensor constructor keeps an internal reference
    to the python object. When the Tensor is released it needs to happen
    inside GIL as it touches the reference count. This PR makes sure
    that the GIL is acquired.

Category:

Bug fix (non-breaking change which fixes an issue)

Description:

  • no-copy mode of the DALI tensor constructor keeps an internal reference
    to the python object. When the Tensor is released it needs to happen
    inside GIL as it touches the reference count. This PR makes sure
    that the GIL is acquired.

Relegates to #5072

Additional information:

Affected modules and functionalities:

Key points relevant for the review:

Tests:

  • Existing tests apply
  • New tests added
    • Python tests
      • test_backend_impl.test_create_tensor_and_make_it_release_memory
    • GTests
    • Benchmark
    • Other
  • N/A

Checklist

Documentation

  • Existing documentation applies
  • Documentation updated
    • Docstring
    • Doxygen
    • RST
    • Jupyter
    • Other
  • N/A

DALI team only

Requirements

  • Implements new requirements
  • Affects existing requirements
  • N/A

REQ IDs: N/A

JIRA TASK: DALI-3668

@JanuszL
Copy link
Contributor Author

JanuszL commented Oct 17, 2023

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [10306640]: BUILD STARTED

arr = np.random.rand(3, 5, 6)
tensor = TensorCPU(arr, "NHWC")
assert_array_equal(arr, np.array(tensor))
arr = None

Check notice

Code scanning / CodeQL

Unused local variable Note

Variable arr is not used.
tensor = TensorCPU(arr, "NHWC")
assert_array_equal(arr, np.array(tensor))
arr = None
tensor = None

Check notice

Code scanning / CodeQL

Unused local variable Note

Variable tensor is not used.
…ns inside GIL

- no-copy mode of the DALI tensor constructor keeps an internal reference
  to the python object. When the Tensor is released it needs to happen
  inside GIL as it touches the reference count. This PR makes sure
  that the GIL is acquired.

Signed-off-by: Janusz Lisiecki <jlisiecki@nvidia.com>
@dali-automaton
Copy link
Collaborator

CI MESSAGE: [10306640]: BUILD PASSED

@JanuszL JanuszL merged commit 15a4956 into NVIDIA:main Oct 18, 2023
4 checks passed
@JanuszL JanuszL deleted the gil_lock branch October 18, 2023 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants