Pinned memory allocation returns odd size

[jakirkham]

It looks like CuPy allocates more bytes than expected when calling cupy.cuda.alloc_pinned_memory. Any ideas why that might be?

In [1]: import numpy 
   ...: import cupy                                                             

In [2]: a = cupy.arange(1_000_000)                                              

In [3]: b = numpy.asarray(cupy.cuda.alloc_pinned_memory(a.nbytes)).view(a.dtype)
   ...:                                                                         

In [4]: b.nbytes                                                                
Out[4]: 8388608

In [5]: a.nbytes                                                                
Out[5]: 8000000

• Conditions (you can just paste the output of python -c 'import cupy; cupy.show_config()')
  • CuPy version: 7.6.0
  • OS/Platform: Ubuntu 18.04.4 LTS
  • CUDA version: 10.2
  • cuDNN/NCCL version (if applicable): 7.6.5/2.5.7.1

CuPy Version          : 7.6.0
CUDA Root             : /datasets/jkirkham/miniconda/envs/rapids15dev
CUDA Build Version    : 10020
CUDA Driver Version   : 10020
CUDA Runtime Version  : 10020
cuBLAS Version        : 10202
cuFFT Version         : 10102
cuRAND Version        : 10102
cuSOLVER Version      : (10, 3, 0)
cuSPARSE Version      : 10301
NVRTC Version         : (10, 2)
cuDNN Build Version   : 7605
cuDNN Version         : 7605
NCCL Build Version    : 2406
NCCL Runtime Version  : 2507
CUB Version           : None
cuTENSOR Version      : None

[leofang]

I think this was the reason:

cupy/cupy/cuda/pinned_memory.pyx

Lines 296 to 298 in 725c350

	# Round up the memory size to fit memory alignment of cudaHostAlloc
	unit = self._allocation_unit_size
	size = internal.clp2(((size + unit - 1) // unit) * unit)

[jakirkham]

So what's the best way to use this? Just trim the array viewing the allocation afterwards? Or is there a better way for one to be allocating pinned memory?

[leofang]

Hi John, sorry I dropped the ball. This is usually what I do

>>> mem = cupy.cuda.alloc_pinned_memory(a.nbytes)
>>> b = numpy.frombuffer(mem, a.dtype, a.size)
>>> b.nbytes
8000000

Perhaps the core devs can recommend better approaches?

[leofang]

Another way:

>>> mem = cupy.cuda.alloc_pinned_memory(a.nbytes)
>>> c = numpy.ndarray((a.size,), dtype=a.dtype, buffer=mem)
>>> c.nbytes
8000000

[kmaehashi]

We were not aware of this use case of pinned memory allocator. When do you need this?

[leofang]

@kmaehashi It is very useful to back NumPy arrays by pinned memory. I use this very often when knowing in advance there'll be frequent device-host transfer following.

[jakirkham]

Thanks Leo! That makes sense 🙂

Yeah this is the same thing we were looking at. Having a NumPy array is a bit easier to work with.

[kmaehashi]

Related to #4080, I'm wondering if it's better to provide this (equivalent to Leo's snippet) as an API under cupyx.
#3625 (comment)

PyCUDA has an API for this feature:
https://documen.tician.de/pycuda/driver.html#pagelocked-allocation

[jakirkham]

That would be very helpful 🙂

[leofang]

if it's better to provide this (equivalent to Leo's snippet) as an API under cupyx

I sent #4870 (still WIP) to address this need.