Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Some NVENC presets are unusable on Tesla M40 #3136

Closed
zhaofengli opened this issue May 25, 2021 · 2 comments
Closed

Some NVENC presets are unusable on Tesla M40 #3136

zhaofengli opened this issue May 25, 2021 · 2 comments
Labels
bug Something isn't working encoding

Comments

@zhaofengli
Copy link

On Tesla M40, the P7 and P6 presets cause CUDA initialization failure (error 8). I was able to work around it with this patch to get Xpra to choose the next preset, but perhaps there is a better way to solve this.

Full logs 
2021-05-25 02:21:47,988 init_context(1250, 444, 'BGRX', ('ARGB', 'BGRA', 'BGRX', 'GBRP', 'GBRP10', 'RGB', 'XRGB', 'YUV420P', 'YUV422P', 'YUV444P', 'YUV444P10', 'r210'), 'h264', 98, 86, (1, 1), {b'eos': True, b'flush': True, b'video_scaling': True, b'video_b_frames': (b'h264',), b'video_max_size': (4096, 4096), b'max-soft-expired': 5, b'send-timestamps': False, b'pillow.version': b'8.2.0', b'jpeg.version': 1, b'webp.version': (1, 1, 0), b'swscale.version': (5, 9, 100), b'cython.version': b'4.2', b'avcodec2.version': (58, 134, 100), b'vpx.version': b'1.10.0', b'min-quality': 30, b'speed': 0, b'min-speed': 30, b'rgb_zlib': True, b'rgb_lz4': True, b'rgb_lzo': False, b'rgb_brotli': False, b'rgb_none': True, b'rgb_formats': (b'RGB', b'RGBX', b'RGBA'), b'full_csc_modes': {b'h264': (b'ARGB', b'BGRA', b'BGRX', b'GBRP', b'RGB', b'XRGB', b'YUV420P', b'YUV422P', b'YUV444P'), b'vp8': (b'YUV420P',), b'h265': (b'BGRX', b'GBRP', b'RGB', b'XRGB', b'YUV420P', b'YUV422P', b'YUV444P'), b'mpeg4': (b'YUV420P',), b'mpeg1': (b'YUV420P',), b'mpeg2': (b'YUV420P',), b'vp9': (b'YUV420P', b'YUV444P'), b'webp': (b'BGRX', b'BGRA')}, b'h264.YUV420P.profile': b'high10', b'h264.fast-decode': False, b'transparency': True})
2021-05-25 02:21:47,988 get_target_pixel_format(98)=BGRX for encoding=h264, scaling=(1, 1), NATIVE_RGB=True, YUV444_CODEC_SUPPORT={'h264': True, 'h265': False}, YUV420_ENABLED=True, YUV444_ENABLED=True, YUV444_THRESHOLD=85, LOSSLESS_ENABLED=True, src_format=BGRX, dst_formats=ARGB, BGRA, BGRX, GBRP, GBRP10, RGB, XRGB, YUV420P, YUV422P, YUV444P, YUV444P10, r210
2021-05-25 02:21:47,989 using lossy h264 compression at 98% quality with pixel format BGRX
2021-05-25 02:21:48,109 CUDA 10.2.0 / PyCUDA 2020.1, found 1 device:
2021-05-25 02:21:48,109   + Tesla M40 @ 0000:42:00.0 (memory: 83% free, compute: 5.2)
2021-05-25 02:21:48,151 selected device Tesla M40 @ 0000:42:00.0
2021-05-25 02:21:48,152 init_cuda() pixel format=BGRX, device_id=0
2021-05-25 02:21:48,152 init_cuda cuda_device= (Tesla M40 @ 0000:42:00.0)
2021-05-25 02:21:48,272 init_cuda cuda_context=
2021-05-25 02:21:48,272 init_cuda cuda info={'driver': {'version': (10, 2, 0), 'driver_version': 11020}}
2021-05-25 02:21:48,272 init_cuda pycuda info={'version': {'': (2020, 1), 'text': '2020.1'}}
2021-05-25 02:21:48,273 CUDA Output Buffer=0x3378fa0000, pitch=5120
2021-05-25 02:21:48,275 inputBuffer=[0 0 0 ... 0 0 0] (size=2293760)
2021-05-25 02:21:48,275 open_encode_session() cuda_context=, cuda_context_ptr=0x7fc69c2da5c0
2021-05-25 02:21:48,275 open_encode_session() functionList=0x7fc69c416340
2021-05-25 02:21:48,278 success, encoder context=0x7fc69c417380 (1 context in use)
2021-05-25 02:21:48,278 found 2 encoders:
2021-05-25 02:21:48,278 [0] H264
2021-05-25 02:21:48,278 [1] HEVC
2021-05-25 02:21:48,278 codecs=H264, HEVC
2021-05-25 02:21:48,279 c_parseguid(6BC82762-4E63-4CA4-AA85-1E50F321F6BF)={'Data1': 1808279394, 'Data2': 20067, 'Data3': 19620, 'Data4': b'\xaa\x85\x1eP\xf3!\xf6\xbf@\x95!b\xc8\x7f'}
2021-05-25 02:21:48,279 found 17 presets:
2021-05-25 02:21:48,279 * default              P frame interval=1, gop length=30
2021-05-25 02:21:48,279 * low-latency          P frame interval=1, gop length=infinite
2021-05-25 02:21:48,279 * hp                   P frame interval=1, gop length=30
2021-05-25 02:21:48,279 * hq                   P frame interval=1, gop length=30
2021-05-25 02:21:48,279 * bd                   P frame interval=3, gop length=30
2021-05-25 02:21:48,279 * low-latency-hq       P frame interval=1, gop length=infinite
2021-05-25 02:21:48,279 * low-latency-hp       P frame interval=1, gop length=infinite
2021-05-25 02:21:48,280 * streaming            P frame interval=1, gop length=30
2021-05-25 02:21:48,280 * lossless             P frame interval=1, gop length=30
2021-05-25 02:21:48,280 * lossless-hp          P frame interval=1, gop length=30
2021-05-25 02:21:48,280 tuning=ultra-low-latency (3)
2021-05-25 02:21:48,280 * P1                   P frame interval=1, gop length=infinite
2021-05-25 02:21:48,280 tuning=ultra-low-latency (3)
2021-05-25 02:21:48,280 * P2                   P frame interval=1, gop length=infinite
2021-05-25 02:21:48,280 tuning=ultra-low-latency (3)
2021-05-25 02:21:48,280 * P3                   P frame interval=1, gop length=infinite
2021-05-25 02:21:48,280 tuning=ultra-low-latency (3)
2021-05-25 02:21:48,280 * P4                   P frame interval=1, gop length=infinite
2021-05-25 02:21:48,281 tuning=ultra-low-latency (3)
2021-05-25 02:21:48,281 * P5                   P frame interval=1, gop length=infinite
2021-05-25 02:21:48,281 tuning=ultra-low-latency (3)
2021-05-25 02:21:48,281 * P6                   P frame interval=1, gop length=infinite
2021-05-25 02:21:48,281 tuning=ultra-low-latency (3)
2021-05-25 02:21:48,281 * P7                   P frame interval=1, gop length=infinite
2021-05-25 02:21:48,281 presets for 6BC82762-4E63-4CA4-AA85-1E50F321F6BF: default, low-latency, hp, hq, bd, low-latency-hq, low-latency-hp, streaming, lossless, lossless-hp, P1, P2, P3, P4, P5, P6, P7 (pixel format=BGRX)
2021-05-25 02:21:48,281 preset          default: speed=   50, quality=   50 (lossless=False - want lossless=False)
2021-05-25 02:21:48,281 preset      low-latency: speed=   80, quality=   20 (lossless=False - want lossless=False)
2021-05-25 02:21:48,281 preset               hp: speed=   60, quality=   40 (lossless=False - want lossless=False)
2021-05-25 02:21:48,282 preset               hq: speed=   50, quality=   70 (lossless=False - want lossless=False)
2021-05-25 02:21:48,282 preset               bd: speed=   40, quality=   80 (lossless=False - want lossless=False)
2021-05-25 02:21:48,282 preset   low-latency-hq: speed=   70, quality=   60 (lossless=False - want lossless=False)
2021-05-25 02:21:48,282 preset   low-latency-hp: speed=  100, quality=    0 (lossless=False - want lossless=False)
2021-05-25 02:21:48,282 preset        streaming: speed=-1000, quality=-1000 (lossless=False - want lossless=False)
2021-05-25 02:21:48,282 preset         lossless: speed=    0, quality=  100 (lossless=True - want lossless=False)
2021-05-25 02:21:48,282 preset      lossless-hp: speed=   30, quality=  100 (lossless=True - want lossless=False)
2021-05-25 02:21:48,282 preset               P1: speed=   50, quality=   10 (lossless=False - want lossless=False)
2021-05-25 02:21:48,282 preset               P2: speed=   50, quality=   25 (lossless=False - want lossless=False)
2021-05-25 02:21:48,282 preset               P3: speed=   50, quality=   40 (lossless=False - want lossless=False)
2021-05-25 02:21:48,282 preset               P4: speed=   50, quality=   55 (lossless=False - want lossless=False)
2021-05-25 02:21:48,282 preset               P5: speed=   50, quality=   70 (lossless=False - want lossless=False)
2021-05-25 02:21:48,282 preset               P6: speed=   50, quality=   85 (lossless=False - want lossless=False)
2021-05-25 02:21:48,283 preset               P7: speed=   50, quality=  100 (lossless=False - want lossless=False)
2021-05-25 02:21:48,283 get_preset(H264) speed=86, quality=98, lossless=False, pixel_format=BGRX, options={216: [('default', 'B2DFB705-4EBD-4C49-9B5F-24A777D3E587')], 246: [('low-latency', '49DF21C5-6DFA-4FEB-9787-6ACC9EFFB726'), ('P3', '36850110-3A07-441F-94D5-3670631F91F6')], 226: [('hp', '60E4C59F-E846-4484-A56D-CD45BE9FDDF6')], 156: [('hq', '34DBA71D-A77B-4B8F-9C3E-B6D5DA24C012'), ('P5', '21C6E6B4-297A-4CBA-998F-B6CBDE72ADE3')], 146: [('bd', '82E3E450-BDBB-4E40-989C-82A90DF9EF32'), ('low-latency-hq', 'C5F733B9-EA97-4CF9-BEC2-BF78A74FD105')], 322: [('low-latency-hp', '67082A44-4BAD-48FA-98EA-93056D150A58')], 336: [('P1', 'FC0A8D3E-45F8-4CF8-80C7-298871590EBF')], 291: [('P2', 'F581CFB8-88D6-4381-93F0-DF13F9C27DAB')], 201: [('P4', '90A7B826-DF06-4862-B9D2-CD6D73A08681')], 111: [('P6', '8E75C279-6299-4AB6-8302-0B215A335CF5')], 78: [('P7', '84848C12-6F71-4C13-931B-53E283F57974')]}
2021-05-25 02:21:48,283 using preset 'P7' for speed=86, quality=98, lossless=0, pixel_format=BGRX
2021-05-25 02:21:48,283 c_parseguid(84848C12-6F71-4C13-931B-53E283F57974)={'Data1': 2223279122, 'Data2': 28529, 'Data3': 19475, 'Data4': b'\x93\x1bS\xe2\x83\xf5yt\xe0\x8b!b\xc8\x7f'}
2021-05-25 02:21:48,283 init_params(H264) using preset=P7
2021-05-25 02:21:48,283 6 profiles:
2021-05-25 02:21:48,283 * 0727BCAA-78C4-4C83-8C2F-EF3DFF267C6A : baseline
2021-05-25 02:21:48,283 * 60B5C1D4-67FE-4790-94D5-C4726D7B6E6D : main
2021-05-25 02:21:48,283 * E7CBC309-4F7A-4B89-AF2A-D537C92BE310 : high
2021-05-25 02:21:48,284 * 40847BF5-33F7-4601-9084-E8FE3C1DB8B7 : stereo
2021-05-25 02:21:48,284 * 7AC663CB-A598-4960-B844-339B261A7D52 : high-444
2021-05-25 02:21:48,284 * BFD6F8E7-233C-4341-8B3E-4818523803F4 : auto
2021-05-25 02:21:48,284 using profile=auto
2021-05-25 02:21:48,284 9 input format types:
2021-05-25 02:21:48,284 * 0x1
2021-05-25 02:21:48,284  + 0x1 : NV12_PL
2021-05-25 02:21:48,284 * 0x10
2021-05-25 02:21:48,284  + 0x10 : YV12_PL
2021-05-25 02:21:48,284 * 0x100
2021-05-25 02:21:48,284  + 0x100 : IYUV_PL
2021-05-25 02:21:48,284 * 0x1000
2021-05-25 02:21:48,285  + 0x1000 : YUV444_PL
2021-05-25 02:21:48,285 * 0x1000000
2021-05-25 02:21:48,285  + 0x1000000 : ARGB
2021-05-25 02:21:48,285 * 0x10000000
2021-05-25 02:21:48,285  + 0x10000000 : ABGR
2021-05-25 02:21:48,285 * 0x4000000
2021-05-25 02:21:48,285  + 0x4000000 : AYUV
2021-05-25 02:21:48,285 * 0x2000000
2021-05-25 02:21:48,285  + 0x2000000 : ARGB10
2021-05-25 02:21:48,285 * 0x20000000
2021-05-25 02:21:48,285  + 0x20000000 : ABGR10
2021-05-25 02:21:48,285 tuning=ultra-low-latency (3)
2021-05-25 02:21:48,286 constQP: 2
2021-05-25 02:21:48,286 chromaFormatIDC(BGRX)=3
2021-05-25 02:21:48,292 init_cuda failed
Traceback (most recent call last):
  File "xpra/codecs/nvenc/encoder.pyx", line 1705, in xpra.codecs.nvenc.encoder.Encoder.init_device
  File "xpra/codecs/nvenc/encoder.pyx", line 1886, in xpra.codecs.nvenc.encoder.Encoder.init_nvenc
  File "xpra/codecs/nvenc/encoder.pyx", line 1900, in xpra.codecs.nvenc.encoder.Encoder.init_encoder
  File "xpra/codecs/nvenc/encoder.pyx", line 1487, in xpra.codecs.nvenc.encoder.raiseNVENC
xpra.codecs.nvenc.encoder.NVENCException: %s - returned %i: This indicates that one or more of the parameter passed to the API call is invalid.
2021-05-25 02:21:48,293 threaded_init_device({b'eos': True, b'flush': True, b'video_scaling': True, b'video_b_frames': (b'h264',), b'video_max_size': (4096, 4096), b'max-soft-expired': 5, b'send-timestamps': False, b'pillow.version': b'8.2.0', b'jpeg.version': 1, b'webp.version': (1, 1, 0), b'swscale.version': (5, 9, 100), b'cython.version': b'4.2', b'avcodec2.version': (58, 134, 100), b'vpx.version': b'1.10.0', b'min-quality': 30, b'speed': 0, b'min-speed': 30, b'rgb_zlib': True, b'rgb_lz4': True, b'rgb_lzo': False, b'rgb_brotli': False, b'rgb_none': True, b'rgb_formats': (b'RGB', b'RGBX', b'RGBA'), b'full_csc_modes': {b'h264': (b'ARGB', b'BGRA', b'BGRX', b'GBRP', b'RGB', b'XRGB', b'YUV420P', b'YUV422P', b'YUV444P'), b'vp8': (b'YUV420P',), b'h265': (b'BGRX', b'GBRP', b'RGB', b'XRGB', b'YUV420P', b'YUV422P', b'YUV444P'), b'mpeg4': (b'YUV420P',), b'mpeg1': (b'YUV420P',), b'mpeg2': (b'YUV420P',), b'vp9': (b'YUV420P', b'YUV444P'), b'webp': (b'BGRX', b'BGRA')}, b'h264.YUV420P.profile': b'high10', b'h264.fast-decode': False, b'transparency': True})
Traceback (most recent call last):
  File "xpra/codecs/nvenc/encoder.pyx", line 1677, in xpra.codecs.nvenc.encoder.Encoder.threaded_init_device
  File "xpra/codecs/nvenc/encoder.pyx", line 1711, in xpra.codecs.nvenc.encoder.Encoder.init_device
  File "xpra/codecs/nvenc/encoder.pyx", line 1705, in xpra.codecs.nvenc.encoder.Encoder.init_device
  File "xpra/codecs/nvenc/encoder.pyx", line 1886, in xpra.codecs.nvenc.encoder.Encoder.init_nvenc
  File "xpra/codecs/nvenc/encoder.pyx", line 1900, in xpra.codecs.nvenc.encoder.Encoder.init_encoder
  File "xpra/codecs/nvenc/encoder.pyx", line 1487, in xpra.codecs.nvenc.encoder.raiseNVENC
xpra.codecs.nvenc.encoder.NVENCException: %s - returned %i: This indicates that one or more of the parameter passed to the API call is invalid.
2021-05-25 02:21:48,293 Warning: failed to initialize NVENC device
2021-05-25 02:21:48,293  error 8:
2021-05-25 02:21:48,293  'This indicates that one or more of the parameter passed to the API call is invalid.'
2021-05-25 02:21:48,293 clean() cuda_context=, encoder context=0x7fc69c417380
2021-05-25 02:21:48,294 cuda_clean() freeing CUDA host buffer [0 0 0 ... 0 0 0]
2021-05-25 02:21:48,295 cuda_clean() freeing CUDA output buffer 0x3378fa0000
2021-05-25 02:21:48,295 cuda_clean() destroying encoder 0x7fc69c417380
2021-05-25 02:21:48,295 cuda_clean() (still 0 context in use)
2021-05-25 02:21:48,296 clean() done

Xpra 4.2
OS: NixOS 21.05
Driver Version: 460.73.01
CUDA Version: 11.2
@zhaofengli zhaofengli mentioned this issue May 25, 2021
Merged
9 tasks
totaam added a commit that referenced this issue May 25, 2021
@totaam
#3136 keep track of presets that fail with NV_ENC_ERR_INVALID_PTR
50a5c9f
@totaam
Copy link
Collaborator

totaam commented May 25, 2021

@zhaofengli Thanks!
How about the slightly modified version I have just applied?

It only records a "bad preset" if the error code is NV_ENC_ERR_INVALID_PARAM.
This is based your log sample which contains This indicates that one or more of the parameter passed to the API call is invalid.

@totaam totaam added bug Something isn't working encoding labels May 25, 2021
@zhaofengli
Copy link
Author

Cool, tested the modified patch and it works! Thanks for your work on Xpra!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working encoding
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@totaam @zhaofengli