process crushes without error info

[AceCLee]

Hello Rigaya!

When I encoded some videos, it crushed without error info.

nvencc64 --input input.mkv --output test.mkv

normal cmd output:

NVENC / CUDA NVENC API 9.1, CUDA 10.2, schedule mode: auto
Input Buffers CUDA, 20 frames
Input Info avcuvid: H.265/HEVC, 1920x1080, 24000/1001 fps
Vpp Filters cspconv(p010 -> nv12)
Output Info H.264/AVC high @ Level auto
1920x1080p 1:1 23.976fps (24000/1001fps)
avwriter: h264 => matroska
Encoder Preset default
Rate Control CQP I:20 P:23 B:25
Lookahead off
GOP length 240 frames
B frames 3 frames [ref mode: disabled]
Ref frames 3 frames, MultiRef L0:auto L1:auto
AQ off
Others mv:auto cabac deblock adapt-transform:auto bdirect:auto

encoded 0 frames, 0.00 fps, 0.00 kbps, 0.00 MB
encode time 0:00:01, CPULoad: 7.7%

track level log output:

GPUAutoSelect: Success.
avhw reader selected.
InitInput: input selected : 9.
avcuvid/avsw: opened file "input.mkv".
avcuvid/avsw: got stream information.
avcuvid/avsw: found video stream, stream idx 0
avcuvid/avsw: use video stream #0 for input, codec hevc, stream time_base 1/1000, codec_timebase 1001/24000.
avcuvid/avsw: can be decoded by cuvid.
avcuvid: set log_level_offset for HEVC codec reader.
avcuvid: start predecode.
avcuvid: GetHeader: 111 bytes.
avcuvid: initialized hevc codec context for parser: time_base: 1001/24000, pkt_timebase: 1/1000.
avcuvid: fps decoder invalid: false
avcuvid: read 128 packets.
avcuvid: checking 111 frame samples.
avcuvid: stream timebase 1001/24000
avcuvid: decoder fps 24000/1001
avcuvid: duration histgram of 2 frames
avcuvid: 42 [ 79 frames]
avcuvid: 41 [ 32 frames]
avcuvid: read 256 packets.
avcuvid: checking 239 frame samples.
avcuvid: stream timebase 1001/24000
avcuvid: decoder fps 24000/1001
avcuvid: duration histgram of 2 frames
avcuvid: 42 [169 frames]
avcuvid: 41 [ 70 frames]
avcuvid: fps mul: 1001
avcuvid: raw avgDuration: 41.707113
avcuvid: estimatedAvgFps: 24001/1001
avcuvid: use decoder fps...
avcuvid: final AvgFps (raw64): 24001/1001
avcuvid: final AvgFps (gcd): 24000/1001
avcuvid: final AvgFps (round): 24000/1001
avcuvid: avcuvid: H.265/HEVC, 1920x1080, 24000/1001 fps
avcuvid: sar 0:0, shift 6
vfr mode automatically enabled with timebase 1001/96000
InitInput: Success.
InitCuda: device #0.

Checking Environment Info...
NVEncC (x64) 4.62 (r1345) by rigaya, Feb 1 2020 13:04:23 (VC 1916/Win/avx2)
OS Version Windows 10 x64 (18363)
CPU Intel Core i7-8750H @ 2.20GHz [TB: 3.89GHz] (6C/12T)
GPU #0: GeForce RTX 2060 (441.66)
NVENC / CUDA NVENC API 9.1, CUDA 10.2, schedule mode: auto
cuInit: Success.
cuDeviceGetCount: Success.
cuDeviceGet: ID:0.
cuDeviceComputeCapability: Success: 7.5.
NVENC capabilities: OK.
using cuda schedule mode: auto.
cuCtxCreate: Success.
cuCtxPopCurrent: Success.
cuvidInit: Success.
cuvidCtxLockCreate: Success.
InitCuda: Success.
nvEncodeAPICreateInstance(APIVer=0x71020009): Success.
NvEncOpenEncodeSessionEx(device_type=NV_ENC_DEVICE_TYPE_CUDA): Success.
InitNVEncInstance: Success.
createDeviceFeatureList: Success.
cuvid: nvcuvid.dll available
cuvid: created frame queue
cuvid: created video parser
cuvid: DecodePacket: success
InitDecoder: Success.
cspconv: cspconv(p010 -> nv12)
InitFilters: Success.
SetInputParam: Success.
m_pEncodeAPI->nvEncInitializeEncoder: Success.
CreateEncoder: Success.
AllocateIOBuffers: Success.
InitChapters: Success.
Eanble NVML Monitoring
Performace Monitor: none
Performace Plot : none
InitPerfMonitor: Success.
Output: Using avformat writer.
avout: output filename: "test.mkv"
avout: allocated internal buffer 8 MB.
avout: set external output buffer 8 MB.
avout: output video stream fps: 24000/1001
avout: opened video avcodec
avout: output video stream timebase: 1/1000
avout: bDtsUnavailable: off
avout: Initialized video output.
avout: avwriter: h264 => matroska
avout: starting output thread...
Output: Initialized avformat writer.
InitOutput: Success.
NVEncC (x64) 4.62 (r1345) by rigaya, Feb 1 2020 13:04:23 (VC 1916/Win/avx2)
OS Version Windows 10 x64 (18363)
CPU Intel Core i7-8750H @ 2.20GHz [TB: 3.89GHz] (6C/12T)
GPU #0: GeForce RTX 2060 (1920 cores, 1335 MHz)[PCIe3x16][441.66]
NVENC / CUDA NVENC API 9.1, CUDA 10.2, schedule mode: auto
Input Buffers CUDA, 20 frames
Input Info avcuvid: H.265/HEVC, 1920x1080, 24000/1001 fps
Vpp Filters cspconv(p010 -> nv12)
Output Info H.264/AVC high @ Level auto
1920x1080p 1:1 23.976fps (24000/1001fps)
avwriter: h264 => matroska
Encoder Preset default
Rate Control CQP I:20 P:23 B:25
Lookahead off
GOP length 240 frames
B frames 3 frames [ref mode: disabled]
Ref frames 3 frames, MultiRef L0:auto L1:auto
AQ off
Others mv:auto cabac deblock adapt-transform:auto bdirect:auto
Started Encode thread
Set packet 0
Set packet 1
Set packet 2
Set packet 3
Set packet 4
Set packet 5
Set packet 6
Set packet 7
Set packet 8
Set packet 9
Set packet 10
Set packet 11
Set packet 12
Set packet 13
Set packet 14
Set packet 15
Set packet 16
Set packet 17
Set packet 18
Set packet 19
Set packet 20
... ...
Set packet 34791
Set packet 34792
Set packet 34793
Set packet 34794
Set packet 34795
Set packet 34796
Set packet 34797
Set packet 34798
Set packet 34799
Set packet 34800
Set packet 34801
Set packet 34802
Set packet 34803
Set packet 34804
Set packet 34805
Set packet 34806
Set packet 34807
Flushing Decoder
Flushed Decoder
Flushed Encoder
avout: Closing...
avout: closed output thread...
avout: closed queues...
avout: Closed avformat context.
avout: Closed File Pointer.
avout: Closed format.
avout: Closed video.
avout: Closed.
avcuvid: Closing...
avcuvid: Closed Stream Packet Buffer.
avcuvid: Closed caption handler.
avcuvid: Closing file pointer...
avcuvid: Closing pb->buffer...
avcuvid: Closed pb->buffer.
avcuvid: Closing avio context...
avcuvid: Closed avio context.
avcuvid: Closed file pointer.
avcuvid: Closing avformat context...
avcuvid: Closed avformat context.
avcuvid: Free formatOptions...
avcuvid: Freed formatOptions.
avcuvid: Closed format.
avcuvid: Close parser...
avcuvid: Closed parser.
avcuvid: Close codecCtx for parser...
avcuvid: Closed codecCtx for parser.
avcuvid: Free extra data...
avcuvid: Freed extra data.
avcuvid: Closed video.
avcuvid: Cleared frame pos list.
avcuvid: Closed.

encoded 0 frames, 0.00 fps, 0.00 kbps, 0.00 MB
encode time 0:00:07, CPULoad: 5.0%
avcuvid: Closing...
avcuvid: Closed Stream Packet Buffer.
avcuvid: Closed caption handler.
avcuvid: Closed format.
avcuvid: Closed video.
avcuvid: Cleared frame pos list.
avcuvid: Closed.
avcuvid: Closing...
avcuvid: Close...
avout: Closing...
avout: closed queues...
avout: Closed format.
avout: Closed video.
avout: Closed.
nvEncDestroyEncoder: success.
cuvid: Closing decoder...
cuvid: cuvidDestroyVideoParser: Fin.
cuvid: Closed decoder.
cuvidCtxLockDestroy...
cuvidCtxLockDestroy: Fin.
Closing EncodeStatus...
Closed EncodeStatus.
cuCtxDestroy...
cuCtxDestroy: Fin.
Closing perf monitor...
perf monitor: Closing thread...
perf monitor: Closed thread.
perf monitor: Closing perf counter...
perf monitor: Closed perf counter.
Closing logger...

when I extracted video stream from mkv file, I could encode the video stream file with NVEnc successfully.

[rigaya]

Thanks you for the log, it seems like no frame is coming out of the hw decoder, but it's difficult to figure out without the actual input file...

Does it work fine with software decoder (--avsw)?

[AceCLee]

Yes, it works fine with software decoder.

[AceCLee]

The size of this video is a little large. If NVEnc can output more debug info, maybe it is easier to solve the problem?

[rigaya]

I've added some more debug/trace logs in NVEnc 4.63, would you run with "--log-level trace --log output.log" and provide the log file (output.log)? Might be able to get a hint from there.

[AceCLee]

cmd param:
nvencc64 --input input.mkv --output test.mkv --log-level trace --log output.log
trace level log:
output.zip

[rigaya]

Thank you for the log.

When using the hardware decoder, it go through the following steps.

input → demuxer(libavformat) → cuvidParser → cuvidDecoder → vpp filters → encoder → muxer → output

Checking the log, I can see that the video bitstream is sent to the cuvidParser but not coming out, and not sent to the decoder. It will be difficult to know what is going wrong in cuvidParser, as it is part of cuvid.

Currently, I have not came up of a solution to go into this problem further.

[AceCLee]

Is it possible to switch to software decoder when cuvidParser does not output the video stream?

[rigaya]

I'm finding it difficult to implement changing the decoder after starting the transcoding process, but NVEnc 5.00 shall be able to detect cuvidParser error.

[AceCLee]

Yes, NVEnc 5.00 raises an error, but it does not seem to terminate.

log:

Set packet #993, size 308, pts 41458 (41.458)
[hevc @ 000001fe6da89100] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
Set packet #994, size 185, pts 41375 (41.375)
[hevc @ 000001fe6da89100] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
Set packet #995, size 212, pts 41416 (41.416)
[hevc @ 000001fe6da89100] nal_unit_type: 0(TRAIL_N), nuh_layer_id: 0, temporal_id: 0
Set packet #996, size 207, pts 41500 (41.5)
[hevc @ 000001fe6da89100] nal_unit_type: 0(TRAIL_N), nuh_layer_id: 0, temporal_id: 0
Set packet #997, size 239, pts 41541 (41.541)
[hevc @ 000001fe6da89100] nal_unit_type: 0(TRAIL_N), nuh_layer_id: 0, temporal_id: 0
Set packet #998, size 7293, pts 41917 (41.917)
[hevc @ 000001fe6da89100] nal_unit_type: 0(TRAIL_N), nuh_layer_id: 0, temporal_id: 0
Set packet #999, size 415, pts 41750 (41.75)
cuvid: cuvid failing to parse/decode video stream.
Error in DecodePacket: 999 (CUDA_ERROR_UNKNOWN).

And process will keep running without log output. Maybe process should return some non-zero exit code and terminate?

[rigaya]

Thank you for testing, the updated NVEnc 5.01 should terminate after error detection.

[patpatqu]

I am using 5.01 to encode a HDR movie MKV file and it "crushes without error info" as well. Then I use FFMPEG to do the same thing and some error messages appeared. But the FFMPEG encoding continued and the file came out fine. The error messages from FFMPEG are:-

 [hevc @ 000001b05bb12fc0] PPS id out of range: 15
 [AVBSFContext @ 000001b05c1a3a40] No start code is found.

[rigaya]

Might be related to #237, so NVEnc 5.04 might help.

[AceCLee]

Might be related to #237, so NVEnc 5.04 might help.

Thanks for your efforts, but NVEnc 5.04 raises cuvid error as before.

And version info of nvencc64 cmd output is "NVEncC (x64) 5.03 (r1557) by rigaya, Jun 11 2020 15:24:18 (VC 1916/Win/avx2)".

[rigaya]

And version info of nvencc64 cmd output is "NVEncC (x64) 5.03 (r1557) by rigaya, Jun 11 2020 15:24:18 (VC 1916/Win/avx2)".

About version number, I have re-uploaded updating the version number to 5.04.