Segmentation Fault When Using DecodeSingleFrame()

[GoodarzMehr]

Describe the bug
I am trying to use VPF to encode and decode NumPy uint8 arrays. However, when I run my code to first encode a sample ndarray and then decode it, I get a Segmentation fault when using DecodeSingleFrame().

To Reproduce
Here's my code:

import PyNvCodec as nvc
import numpy as np

gpuID = 0

nvEnc = nvc.PyNvEncoder({'bitrate': '30K', 'fps': '10', 'codec': 'hevc', 's': '256x256'}, gpuID)
nvDec = nvc.PyNvDecoder(256, 256, nvc.PixelFormat.NV12, nvc.CudaVideoCodec.HEVC, gpuID)

rawFrame = np.random.randint(0, 255, size = (256, 256, 3), dtype = np.uint8)

print('Raw frame size is ' + str(rawFrame.size) + ' bytes.')

encodedFrame = np.ndarray(shape = (0), dtype = np.uint8)

count, success = 0, False

while success is not True and count < 10:
    success = nvEnc.EncodeSingleFrame(rawFrame, encodedFrame, sync = False)
    count += 1

print('Encoded frame size is ' + str(encodedFrame.size) + ' bytes.')

success = nvDec.DecodeSingleFrame(encodedFrame)

Expected behavior
No Segmentation fault.

Screenshots
Here's the gdb output:

(gdb) run EncoderDecoder.py 
Starting program: /usr/bin/python3 EncoderDecoder.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffe37ff640 (LWP 9761)]
[New Thread 0x7fffe2ffe640 (LWP 9762)]
[New Thread 0x7fffe07fd640 (LWP 9763)]
[New Thread 0x7fffdbffc640 (LWP 9764)]
[New Thread 0x7fffdb7fb640 (LWP 9765)]
[New Thread 0x7fffd8ffa640 (LWP 9766)]
[New Thread 0x7fffd67f9640 (LWP 9767)]
[New Thread 0x7fffd3ff8640 (LWP 9768)]
[New Thread 0x7fffd17f7640 (LWP 9769)]
[New Thread 0x7fffceff6640 (LWP 9770)]
[New Thread 0x7fffcc7f5640 (LWP 9771)]
[New Thread 0x7fffc9ff4640 (LWP 9772)]
[New Thread 0x7fffc77f3640 (LWP 9773)]
[New Thread 0x7fffc4ff2640 (LWP 9774)]
[New Thread 0x7fffc27f1640 (LWP 9775)]
[New Thread 0x7fffbfff0640 (LWP 9776)]
[New Thread 0x7fffbd7ef640 (LWP 9777)]
[New Thread 0x7fffbafee640 (LWP 9778)]
[New Thread 0x7fffb87ed640 (LWP 9779)]
[New Thread 0x7fffb5fec640 (LWP 9780)]
[New Thread 0x7fffb37eb640 (LWP 9781)]
[New Thread 0x7fffb0fea640 (LWP 9782)]
[New Thread 0x7fffb07e9640 (LWP 9783)]
[New Thread 0x7fffa39ed640 (LWP 9787)]
[New Thread 0x7fffa31ec640 (LWP 9788)]
[New Thread 0x7fffa2607640 (LWP 9789)]
Raw frame size is 196608 bytes.
Encoded frame size is 32918 bytes.

Thread 1 "python3" received signal SIGSEGV, Segmentation fault.
0x00007ffff6a49ec8 in VPF::Task::Execute() () from /home/goodi/.local/lib/python3.10/site-packages/PyNvCodec/_PyNvCodec.cpython-310-x86_64-linux-gnu.so
(gdb) bt
#0  0x00007ffff6a49ec8 in VPF::Task::Execute() () at /home/goodi/.local/lib/python3.10/site-packages/PyNvCodec/_PyNvCodec.cpython-310-x86_64-linux-gnu.so
#1  0x00007ffff6a1564a in  () at /home/goodi/.local/lib/python3.10/site-packages/PyNvCodec/_PyNvCodec.cpython-310-x86_64-linux-gnu.so
#2  0x00007ffff6a15cc6 in  () at /home/goodi/.local/lib/python3.10/site-packages/PyNvCodec/_PyNvCodec.cpython-310-x86_64-linux-gnu.so
#3  0x00007ffff6a1dcea in  () at /home/goodi/.local/lib/python3.10/site-packages/PyNvCodec/_PyNvCodec.cpython-310-x86_64-linux-gnu.so
#4  0x00007ffff69f4d30 in  () at /home/goodi/.local/lib/python3.10/site-packages/PyNvCodec/_PyNvCodec.cpython-310-x86_64-linux-gnu.so
#5  0x00005555556b099e in  ()
#6  0x00005555556a74ab in _PyObject_MakeTpCall ()
#7  0x00005555556bef0b in  ()
#8  0x000055555569f462 in _PyEval_EvalFrameDefault ()
#9  0x0000555555695ed6 in  ()
#10 0x000055555578c366 in PyEval_EvalCode ()
#11 0x00005555557b9108 in  ()
#12 0x00005555557b1f5b in  ()
#13 0x00005555557b8e55 in  ()
#14 0x00005555557b8338 in _PyRun_SimpleFileObject ()
#15 0x00005555557b8033 in _PyRun_AnyFileObject ()
#16 0x00005555557a92de in Py_RunMain ()
#17 0x000055555577f32d in Py_BytesMain ()
#18 0x00007ffff7c29d90 in __libc_start_call_main (main=main@entry=0x55555577f2f0, argc=argc@entry=2, argv=argv@entry=0x7fffffffe108) at ../sysdeps/nptl/libc_start_call_main.h:58
#19 0x00007ffff7c29e40 in __libc_start_main_impl (main=0x55555577f2f0, argc=2, argv=0x7fffffffe108, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe0f8) at ../csu/libc-start.c:392
#20 0x000055555577f225 in _start ()

Desktop (please complete the following information):

• OS: Ubuntu 22.04
• Nvidia driver version: 530.30.02
• CUDA version: 12.1
• Video Codec SDK version: I have no idea where or how to find this.
• Python version: 3.10.6
• FFmpeg version: 4.4.2

[theHamsta]

Can you check out #457. I believe you are using the wrong function overload. I agree that the API is confusing. It also was for me.

[GoodarzMehr]

Thank you! Using DecodeFrameFromPacket() resolves the Segmentation fault issue. However, I'm not sure how to interpret the decoded frame. For example, when inputting a 256x256 uint8 array, while the input frame size is 65536B, the decoded frame size is 98304B. The first two-thirds of the decoded frame seems to be the original data (though I'm not sure in what order) and the last third some random stuff. I imagine this being related to the codec expecting a 12-bit input instead of 8, so I wonder what to do next. I tried changing the decoder PixelFormat to Y, but it then only returns an empty frame.

[theHamsta]

Hi! Probably the dimensions of the Numpy arrays need to be adapted to account for the size of NV12 (height, width, 2), but it returned also mostly garbage for me. Will look into this next week.

[GoodarzMehr]

I think I figured out what's going on. The second one-third of the decoded frame needs to be interleaved in the first one-third. This is the code that worked for me, I'm putting it here for anyone interested as I couldn't find any examples of this use case online.

import time
import numpy as np
import PyNvCodec as nvc
import matplotlib.pyplot as plt

gpuID = 0
w = 1024
h = 256
size = str(w) + 'x' + str(h)

nvEnc = nvc.PyNvEncoder({'bitrate': '30K', 'fps': '10', 'codec': 'hevc', 's': size}, gpuID)
nvDec = nvc.PyNvDecoder(w, h, nvc.PixelFormat.NV12, nvc.CudaVideoCodec.HEVC, gpuID)

rawFrame = np.zeros((int(3 * h / 2), w), dtype=np.uint8)

for i in range(h):
    for j in range(w):
        rawFrame[i][j] = (i * i + j * j) % 256

print('Raw frame size is ' + str(rawFrame.size) + ' bytes.')

encodedFrame = np.ndarray(shape=(0), dtype=np.uint8)
decodedFrame = np.ndarray(shape=(0), dtype=np.uint8)

count, success = 0, False

start = time.time()

while success is not True and count < 10:
    success = nvEnc.EncodeSingleFrame(rawFrame, encodedFrame, sync=False)
    count += 1

finish = time.time()

print('Encoded frame size is ' + str(encodedFrame.size) + ' bytes. Encoding took ' + str(finish - start)
      + ' seconds.')

count, success = 0, False

start = time.time()

while success is not True and count < 10:
    success = nvDec.DecodeFrameFromPacket(decodedFrame, encodedFrame)
    count += 1

finish = time.time()

print('Decoded frame size is ' + str(decodedFrame.size) + ' bytes. Decoding took ' + str(finish - start)
      + ' seconds.')

decodedFrame = decodedFrame.reshape(-1, w)

decodedFrameCorrected = np.zeros((h, w), dtype=np.uint8)
decodedFrameCorrected[0::2, :] = decodedFrame[0:int(h / 2), :]
decodedFrameCorrected[1::2, :] = decodedFrame[int(h / 2):h, :]

fig, ax = plt.subplots(1, 2, figsize=(12, 6), dpi = 300)

ax[0].imshow(rawFrame[:h, :], cmap='gray')
ax[0].set_title('Raw Frame')

ax[1].imshow(decodedFrameCorrected, cmap='gray')
ax[1].set_title('Decoded Frame')

plt.show()

The output of the code is something like this:

Raw frame size is 393216 bytes.
Encoded frame size is 27120 bytes. Encoding took 0.009379148483276367 seconds.
Decoded frame size is 393216 bytes. Decoding took 0.008161544799804688 seconds.