Getting error in window_video_source after out of memory error

[totaam]

Issue migrated from trac ticket # 736

component: android | priority: major | resolution: fixed

2014-11-19 21:07:45: nickc created the issue

---

Seeing the following two stacktraces while testing #733:

2014-11-19 12:59:45,328 setup_pipeline failed for (66, (1, 1), None, 0, 0, None, 'BGRX', (1, 1), 600, 400, codec_spec(nvenc:nvenc4))
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/xpra/server/window_video_source.py", line 1095, in setup_pipeline
    if self.setup_pipeline_option(width, height, src_format, *option):
  File "/usr/lib64/python2.7/site-packages/xpra/server/window_video_source.py", line 1156, in setup_pipeline_option
    ve.init_context(enc_width, enc_height, enc_in_format, dst_formats, encoder_spec.encoding, quality, speed, encoder_scaling, self.encoding_options)
  File "encoder.pyx", line 1254, in xpra.codecs.nvenc4.encoder.Encoder.init_context (xpra/codecs/nvenc4/encoder.c:8739)
NVENCException: opening session - returned 10: This indicates that the API call failed because it was unable to allocate enough memory to perform the requested operation.
2014-11-19 12:59:45,853 error on window 88
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/xpra/server/source.py", line 362, in recalculate_delays
    ws.reconfigure()
  File "/usr/lib64/python2.7/site-packages/xpra/server/window_source.py", line 746, in reconfigure
    self.update_encoding_options(force_reload)
  File "/usr/lib64/python2.7/site-packages/xpra/server/window_video_source.py", line 630, in update_encoding_options
    self.check_pipeline_score(force_reload)
  File "/usr/lib64/python2.7/site-packages/xpra/server/window_video_source.py", line 670, in check_pipeline_score
    if ve is None or csce is None or ve.is_closed() or csce.is_closed():
AttributeError: 'xpra.codecs.csc_swscale.colorspace_converter.Color' object has no attribute 'is_closed'

[totaam]

2014-11-19 23:51:38: antoine changed owner from antoine to nickc

[totaam]

2014-11-19 23:51:38: antoine commented

---

I believe this was caused by r8108: although the calls to is_closed were there before, we never actually hit that codepath because we never kept the same csc module whilst changing the encoder. That's why this only occurs now with nvenc enabled as per #733.

I fixed a similar bug in r8121 for the csc cython module, csc opencl already had it, r8125 fixes this for csc swscale.
Please close if this bug is gone.

[totaam]

2014-12-03 20:28:57: antoine changed status from new to closed

[totaam]

2014-12-03 20:28:57: antoine changed resolution from ** to fixed

[totaam]

2014-12-03 20:28:57: antoine commented

---

Backports for v0.14.x were in 8128, I assume this is fixed so closing. Feel free to re-open if I've missed anything.