Make DALI array_interface memory writable

[JanuszL]

• marking DALI memory shared by array_interface read-only prevents it
  from sharing directly with Torch and possibly other libraries.
  This PR makes it writable.
• memory can be shared anyway but it requires an intermediate step
  through cupy which makes the memory writable

Category:

Other*/Breaking change(I hope not)

Description:

• marking DALI memory shared by array_interface read-only prevents it
  from sharing directly with Torch and possibly other libraries.
  This PR makes it writable.
• memory can be shared anyway but it requires an intermediate step
  through cupy which makes the memory writable

Additional information:

Affected modules and functionalities:

• tensor cuda_array_interface and array_interface

Key points relevant for the review:

• NA

Tests:

• Existing tests apply
  • adjusted test_array_interface_tensor_cpu and test_cuda_array_interface_tensor_gpu
• New tests added
  • Python tests
  • GTests
  • Benchmark
  • Other
• N/A

Checklist

Documentation

• Existing documentation applies
• Documentation updated
  • Docstring
  • Doxygen
  • RST
  • Jupyter
  • Other
• N/A

DALI team only

Requirements

• Implements new requirements
• Affects existing requirements
• N/A

REQ IDs: N/A

JIRA TASK: N/A

[JanuszL]

!build

[JanuszL]

The latest torch complains value cannot be converted to type uint8 without overflow.

[JanuszL]

The latest torch complains value cannot be converted to type uint8 without overflow.

[dali-automaton]

CI MESSAGE: [8039230]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [8039230]: BUILD PASSED

[mzient]

Shouldn't it be "writable" in the title?

[JanuszL]

Shouldn't it be "writable" in the title?

Fixed

[klecki]

Just a question: when converting form one Tensor type to another, like when we do TensorList -> Tensor, we typically pack the shared_ptr to the memory somewhere. Here, I don't think there is even Python object reference to the original tensor. Wouldn't it crash in an iteration or two in most cases?

[JanuszL]

Just a question: when converting form one Tensor type to another, like when we do TensorList -> Tensor, we typically pack the shared_ptr to the memory somewhere. Here, I don't think there is even Python object reference to the original tensor. Wouldn't it crash in an iteration or two in most cases?

I don't think the pipeline exposes a non-continuous tensor as the output. In the general case, it may not work, in most cases, we just expose the pointer to the underlying, continuous memory, so the tensor itself doesn't have to live long.
The problem you describes is rather about the conversion from TensorList to Tensor itself, and possible problems of the array_interface are just a derivate of it.

[klecki]

The problem I'm referring to is specifically how long the allocation is valid. But thinking about it, the array interface is just a property of the object, and accessing it doesn't impact the lifetime of the original object. So the user of array interface must take lifetimes into account and keep the original object alive as long as they are using the raw pointer that they obtain. I think we should be fine, until someone keeps this pointer for too long thinking it contents won't change. I don't know what Pytorch may do with such memory.

[JanuszL]

The problem I'm referring to is specifically how long the allocation is valid. But thinking about it, the array interface is just a property of the object, and accessing it doesn't impact the lifetime of the original object. So the user of array interface must take lifetimes into account and keep the original object alive as long as they are using the raw pointer that they obtain. I think we should be fine, until someone keeps this pointer for too long thinking it contents won't change. I don't know what Pytorch may do with such memory.

Yes, that is what I had in mind. I think PyTorch just wraps the memory up and does nothing with it. After all, it was possible already but with cupy in the middle.

[klecki]

Also, reading the docs of array_interface:

A reference to the object exposing the array interface must be stored by the new object if the memory area is to be secured.

I think it solves my issues.
BTW @JanuszL, maybe you want to follow up with CUDA array interface v3? https://numba.readthedocs.io/en/stable/cuda/cuda_array_interface.html

[JanuszL]

I guess that we want to use the new interface at some point.