html5 client visual corruption #2200

totaam · 2019-03-10T05:48:11Z

Issue migrated from trac ticket # 2200

component: html5 | priority: blocker | resolution: fixed

2019-03-10 05:48:11: antoine created the issue

Seen with the win32 shadow server as tested for #2124.
Looks like some screen updates go missing. Likely to be scroll or video updates.

Links that may be related:

scrolling improvements #1426#comment:8 - html5 visual corruption

web client does not update regions... #2053: web client does not update regions...

#2060: html5 h264 broadway decoder picture dimension is rounded up

nvenc produces invalid stream #2051: nvenc produces invalid stream - unlikely, reproducible without nvenc

html5 scroll paint error #1992: html5 scroll paint error

Scroll paint bug in HTML5 Client #1637: Scroll paint bug in HTML5 Client

html5 client should honour paint flush order #1432: html5 client should honour paint flush order

The text was updated successfully, but these errors were encountered:

totaam · 2019-03-10T05:49:12Z

2019-03-10 05:49:12: antoine changed status from new to assigned

totaam · 2019-03-10T05:49:12Z

2019-03-10 05:49:12: antoine edited the issue description

totaam · 2019-03-10T08:12:25Z

2019-03-10 08:12:25: antoine commented

It seems that this only occurs reliably with the python3 version of the server.

with a python2 server, the server -d compress with html5 client "paint" debug enabled looks like this:

2019-03-10 09:03:35,967 Handshake complete; enabling connection
2019-03-10 09:03:35,967  automatic picture encoding enabled, also available:
2019-03-10 09:03:35,967   jpeg, png, rgb32, webp, h264
2019-03-10 09:03:36,107 HTML5 Linux Chrome client version 2.5
2019-03-10 09:03:36,123 shadow server: setting default keymap translation
2019-03-10 09:03:36,123  client root window size is 2039x1415
2019-03-10 09:03:36,170 client got hello: server version 2.5 accepted our connection
2019-03-10 09:03:36,170 client connection_progress( Session started ,   ,  100 )
2019-03-10 09:03:36,185 client startup complete
2019-03-10 09:03:36,185 client connection-established
2019-03-10 09:03:36,279 compress:  16.0ms for 1648x1060 pixels at    0,0    for wid=1     using      h264 with ratio   4.7%  ( 6823KB to   319KB), sequence     1, client_options={u'type': u'IDR', u'frame': 0, u'pts': 0L, 'csc': u'YUV420P', 'scaled_size': (824, 530)}
2019-03-10 09:03:36,295 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:03:36,295 client draw do_paint( 326914  bytes of   h264  data  1648 x 1060  at  0 , 0 ) focused= true
2019-03-10 09:03:36,342 client broadway decoder initialized
2019-03-10 09:03:36,717 client draw broadway picture decoded:  1810432 bytes, size  832 x 544, paint location:  0,0 with infos= undefined
2019-03-10 09:03:36,717 client draw request_redraw for Window(1)
2019-03-10 09:03:36,717 client draw decode time for  h264  sequence  1 :  423 , flush= 0
2019-03-10 09:03:36,717 client draw animation frame: 1 windows to paint
2019-03-10 09:03:36,732 compress:   0.0ms for 1648x40   pixels at    0,1020 for wid=1     using      webp with ratio   8.5%  (  257KB to    21KB), sequence     3, client_options={u'quality': 100, u'rgb_format': 'BGRX'}
2019-03-10 09:03:36,748 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:03:36,748 client draw do_paint( 22346  bytes of   webp  data  1648 x 40  at  0 , 1020 ) focused= true
2019-03-10 09:03:36,763 client draw request_redraw for Window(1)
2019-03-10 09:03:36,763 client draw decode time for  webp  sequence  2 :  4 , flush= 0
2019-03-10 09:03:36,763 client draw may_paint_now() paint pending= 0 , paint queue length= 0
2019-03-10 09:03:36,763 client draw animation frame: 1 windows to paint
2019-03-10 09:03:37,170 Error: cannot connect to named pipe '\\.\pipe\Xpra\Main'
2019-03-10 09:03:37,498 compress:  16.0ms for 1648x1060 pixels at    0,0    for wid=1     using      h264 with ratio   0.1%  ( 6823KB to     9KB), sequence     3, client_options={u'type': u'P', u'frame': 1, u'pts': 1219L, 'csc': u'YUV420P', 'scaled_size': (824, 530)}
2019-03-10 09:03:37,513 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:03:37,513 client draw do_paint( 9384  bytes of   h264  data  1648 x 1060  at  0 , 0 ) focused= true
2019-03-10 09:03:37,529 client draw broadway picture decoded:  1810432 bytes, size  832 x 544, paint location:  0,0 with infos= undefined
2019-03-10 09:03:37,529 client draw request_redraw for Window(1)
2019-03-10 09:03:37,529 client draw decode time for  h264  sequence  3 :  16 , flush= 0
2019-03-10 09:03:37,529 client draw animation frame: 1 windows to paint
2019-03-10 09:03:38,045 compress:   0.0ms for 1648x14   pixels at    0,989  for wid=1     using      webp with ratio   1.0%  (   90KB to     0KB), sequence     5, client_options={u'quality': 100, u'rgb_format': 'BGRX'}
2019-03-10 09:03:38,107 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:03:38,107 client draw do_paint( 944  bytes of   webp  data  1648 x 14  at  0 , 989 ) focused= true
2019-03-10 09:03:38,107 client draw request_redraw for Window(1)
2019-03-10 09:03:38,107 client draw decode time for  webp  sequence  4 :  2 , flush= 0
2019-03-10 09:03:38,107 client draw may_paint_now() paint pending= 0 , paint queue length= 0
2019-03-10 09:03:38,123 client draw animation frame: 1 windows to paint
2019-03-10 09:03:38,545 compress:   0.0ms for 1648x14   pixels at    0,989  for wid=1     using      webp with ratio   1.0%  (   90KB to     0KB), sequence     6, client_options={u'quality': 100, u'rgb_format': 'BGRX'}

Same with python3 server:

2019-03-10 09:06:59,562 Handshake complete; enabling connection
2019-03-10 09:06:59,562  automatic picture encoding enabled, also available:
2019-03-10 09:06:59,562   jpeg, png, rgb32, webp, h264
2019-03-10 09:06:59,578 HTML5 Linux Chrome client version 2.5
2019-03-10 09:06:59,593 shadow server: setting default keymap translation
2019-03-10 09:06:59,593  client root window size is 1730x1415
2019-03-10 09:06:59,641 client got hello: server version 2.5 accepted our connection
2019-03-10 09:06:59,641 client connection_progress( Session started ,   ,  100 )
2019-03-10 09:06:59,657 client startup complete
2019-03-10 09:06:59,657 client connection-established
2019-03-10 09:06:59,735 compress:  15.0ms for 1648x1060 pixels at    0,0    for wid=1     using      h264 with ratio   2.6%  ( 6823KB to   179KB), sequence     1, client_options={'frame': 0, 'pts': 0, 'type': 'IDR', 'csc': 'YUV420P', 'scaled_size': (824, 530)}
2019-03-10 09:06:59,750 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:06:59,750 client draw do_paint( 183530  bytes of   h264  data  1648 x 1060  at  0 , 0 ) focused= true
2019-03-10 09:06:59,782 client broadway decoder initialized
2019-03-10 09:07:00,000 client draw request_redraw for Window(1)
2019-03-10 09:07:00,016 client draw decode time for  h264  sequence  1 :  264 , flush= 0
2019-03-10 09:07:00,016 client draw animation frame: 1 windows to paint
2019-03-10 09:07:00,047 compress:   0.0ms for 1648x14   pixels at    0,989  for wid=1     using     rgb32 with ratio   1.5%  (   90KB to     1KB), sequence     3, client_options={'rgb_format': 'BGRX', 'lz4': 1, 'flush': 1}
2019-03-10 09:07:00,047 compress:   0.0ms for 1648x40   pixels at    0,1020 for wid=1     using      webp with ratio   8.4%  (  257KB to    21KB), sequence     4, client_options={'rgb_format': 'BGRX', 'quality': 100}
2019-03-10 09:07:00,047 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:07:00,047 client draw do_paint( 1415  bytes of   rgb32  data  1648 x 14  at  0 , 989 ) focused= true
2019-03-10 09:07:00,063 client draw got  92288 to paint with stride 6592
2019-03-10 09:07:00,063 client draw decode time for  rgb32  sequence  2 :  8 , flush= 1
2019-03-10 09:07:00,063 client draw may_paint_now() paint pending= 0 , paint queue length= 0
2019-03-10 09:07:00,063 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:07:00,063 client draw do_paint( 22268  bytes of   webp  data  1648 x 40  at  0 , 1020 ) focused= true
2019-03-10 09:07:00,079 client draw request_redraw for Window(1)
2019-03-10 09:07:00,079 client draw decode time for  webp  sequence  3 :  5 , flush= 0
2019-03-10 09:07:00,079 client draw may_paint_now() paint pending= 0 , paint queue length= 0
2019-03-10 09:07:00,094 client draw animation frame: 1 windows to paint
2019-03-10 09:07:00,407 compress:   0.0ms for 1648x14   pixels at    0,989  for wid=1     using     rgb32 with ratio   1.5%  (   90KB to     1KB), sequence     5, client_options={'rgb_format': 'BGRX', 'lz4': 1, 'flush': 1}
2019-03-10 09:07:00,407 compress:   0.0ms for 1648x9    pixels at    0,1028 for wid=1     using     rgb32 with ratio  18.1%  (   57KB to    10KB), sequence     6, client_options={'rgb_format': 'BGRX', 'lz4': 1}
2019-03-10 09:07:00,422 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:07:00,422 client draw do_paint( 1424  bytes of   rgb32  data  1648 x 14  at  0 , 989 ) focused= true
2019-03-10 09:07:00,422 client draw got  92288 to paint with stride 6592
2019-03-10 09:07:00,422 client draw decode time for  rgb32  sequence  4 :  7 , flush= 1
2019-03-10 09:07:00,422 client draw may_paint_now() paint pending= 0 , paint queue length= 0
2019-03-10 09:07:00,422 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:07:00,422 client draw do_paint( 10718  bytes of   rgb32  data  1648 x 9  at  0 , 1028 ) focused= true
2019-03-10 09:07:00,422 client draw got  59328 to paint with stride 6592
2019-03-10 09:07:00,422 client draw request_redraw for Window(1)
2019-03-10 09:07:00,422 client draw decode time for  rgb32  sequence  5 :  2 , flush= 0
2019-03-10 09:07:00,422 client draw may_paint_now() paint pending= 0 , paint queue length= 0
2019-03-10 09:07:00,422 client draw animation frame: 1 windows to paint
2019-03-10 09:07:00,704 compress:   0.0ms for 1648x40   pixels at    0,1020 for wid=1     using      webp with ratio   8.9%  (  257KB to    22KB), sequence     7, client_options={'rgb_format': 'BGRX', 'quality': 100}
2019-03-10 09:07:00,719 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:07:00,719 client draw do_paint( 23550  bytes of   webp  data  1648 x 40  at  0 , 1020 ) focused= true
2019-03-10 09:07:00,719 client draw request_redraw for Window(1)
2019-03-10 09:07:00,719 client draw decode time for  webp  sequence  6 :  5 , flush= 0
2019-03-10 09:07:00,719 client draw may_paint_now() paint pending= 0 , paint queue length= 0
2019-03-10 09:07:00,735 client draw animation frame: 1 windows to paint
2019-03-10 09:07:00,766 compress:   0.0ms for 1648x40   pixels at    0,1020 for wid=1     using      webp with ratio   8.9%  (  257KB to    22KB), sequence     8, client_options={'rgb_format': 'BGRX', 'quality': 100}
2019-03-10 09:07:00,782 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:07:00,782 client draw do_paint( 23546  bytes of   webp  data  1648 x 40  at  0 , 1020 ) focused= true
2019-03-10 09:07:00,782 client draw request_redraw for Window(1)
2019-03-10 09:07:00,782 client draw decode time for  webp  sequence  7 :  3 , flush= 0
2019-03-10 09:07:00,782 client draw may_paint_now() paint pending= 0 , paint queue length= 0
2019-03-10 09:07:00,782 client draw animation frame: 1 windows to paint
2019-03-10 09:07:00,954 compress:   0.0ms for 1648x14   pixels at    0,989  for wid=1     using     rgb32 with ratio   1.5%  (   90KB to     1KB), sequence     9, client_options={'rgb_format': 'BGRX', 'lz4': 1}
2019-03-10 09:07:00,969 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:07:00,969 client draw do_paint( 1415  bytes of   rgb32  data  1648 x 14  at  0 , 989 ) focused= true
2019-03-10 09:07:00,969 client draw got  92288 to paint with stride 6592
2019-03-10 09:07:00,969 client draw request_redraw for Window(1)
2019-03-10 09:07:00,969 client draw decode time for  rgb32  sequence  8 :  1 , flush= 0
2019-03-10 09:07:00,969 client draw may_paint_now() paint pending= 0 , paint queue length= 0
2019-03-10 09:07:00,969 client draw animation frame: 1 windows to paint

So the python3 server never triggers broadway picture decoded.

totaam · 2019-03-10T09:43:55Z

2019-03-10 09:43:55: antoine changed status from assigned to closed

totaam · 2019-03-10T09:43:55Z

2019-03-10 09:43:55: antoine set resolution to fixed

totaam · 2019-03-10T09:43:55Z

2019-03-10 09:43:55: antoine commented

Using XPRA_SAVE_VIDEO_STREAMS=1 xpra shadow ..., then using ffmpeg to extract an image from the first frame:

ffmpeg -i window-1-0.1-x264.h264 -frames:v 1 -f image2 frame.png

I can see that the frames in that stream are just fine.
And ffprobe shows:

$ ffprobe window-1-0.1-x264.h264 
ffprobe version 4.0.3 Copyright (c) 2007-2018 the FFmpeg developers
  built with gcc 8 (GCC)
  libavutil      56. 14.100 / 56. 14.100
  libavcodec     58. 18.100 / 58. 18.100
  libavformat    58. 12.100 / 58. 12.100
  libavdevice    58.  3.100 / 58.  5.100
  libavfilter     7. 16.100 /  7. 16.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  1.100 /  5.  1.100
  libswresample   3.  1.100 /  3.  1.100
  libpostproc    55.  1.100 / 55.  1.100
[h264 @ 0x56024fdac4c0] decoding for stream 0 failed
Input #0, h264, from 'window-1-0.1-x264.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p(progressive), 824x530, 25 tbr, 1200k tbn, 50 tbc

Then with the "same" stream from a python2 server:

$ ffprobe window-1-0.1-x264.h264 
ffprobe version 4.0.3 Copyright (c) 2007-2018 the FFmpeg developers
  built with gcc 8 (GCC)
  libavutil      56. 14.100 / 56. 14.100
  libavcodec     58. 18.100 / 58. 18.100
  libavformat    58. 12.100 / 58. 12.100
  libavdevice    58.  3.100 / 58.  5.100
  libavfilter     7. 16.100 /  7. 16.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  1.100 /  5.  1.100
  libswresample   3.  1.100 /  3.  1.100
  libpostproc    55.  1.100 / 55.  1.100
[h264 @ 0x55db912104c0] decoding for stream 0 failed
Input #0, h264, from 'window-1-0.1-x264.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (Constrained Baseline), yuv420p(progressive), 824x530, 25 tbr, 1200k tbn, 50 tbc

The big difference is in the profile:

with python3 and -d x264:

using default profile=high
x264: get_tune() TUNE=None, fast_decode=1, content_type=desktop
x264 context=0x53cf9e0, YUV420P  824x530  quality=100, speed=80, content_type=desktop
x264 params: {'me': {'type': 'DIA', 'me-range': 16, 'mv-range': -1, 'mv-range-thread': -1,
    'subpel_refine': 2, 'weighted-pred': 0}, 'rc': {'rc-method': 'CRF', 'qp_constant': -1,
    'qp_min': 0, 'qp_max': 2147483647, 'qp_step': 4, 'bitrate': 0, 'vbv_max_bitrate': 0,
    'vbv_buffer_size': 0, 'mb-tree': False, 'lookahead': -1}, 'vfr-input': False,
    'bframe-adaptive': 'FAST', 'open-gop': True, 'bluray-compat': False, 'cabac': False,
    'deblocking-filter': False, 'intra-refresh': False, 'interlaced': False,
    'constrained_intra': False, 'threads': 'auto', 'sliced-threads': True}
x264 encode YUV420P frame     0 as  IDR slice with 4 nals, tune=b'zerolatency', total  221909 bytes, keyframe=True , delayed=0

with python2

x264: get_tune() TUNE=None, fast_decode=1, content_type=desktop
x264 context=0x4573df0, YUV420P  824x530  quality=100, speed=80, content_type=desktop
x264 params: {u'vfr-input': False, u'intra-refresh': False, u'deblocking-filter': False,
   u'cabac': False, u'threads': u'auto', u'bframe-adaptive': u'FAST', u'constrained_intra' : False,
   u'open-gop': False, u'interlaced': False, u'me': {u'mv-range-thread': -1, u'weighted-pred': 0,
   u'subpel_refine': 2, u'me-range': 16, u'mv-range': -1, u'type': u'DIA'}, u'sliced-threads': True,
   u'bluray-compat': False, u'rc': {u'rc-method': u'CRF', u'vbv_buffer_size': 0, u'lookahead': -1,
   u'qp_min': 0, u'qp_step': 4, u'vbv_max_bitrate': 0, u'qp_max': 2147483647, u'mb-tree': False,
   u'bitrate': 0, u'qp_constant': -1}}
x264 encode YUV420P frame     0 as  IDR slice with 4 nals, tune=zerolatency, total  214879 bytes, keyframe=True , delayed=0

That's because the broadway decoder cannot handle the other profiles.

And so it is just the painful strings vs bytes from python3 (which is also a complete waste of CPU cycles):

we have to use a typedict function to lookup the key without caring if it's a string or bytes
we have to convert the result to a string so that it passes validation.
With python2, we don't need to worry about any of this nonsense.

Fixed in r22037.

totaam closed this as completed Mar 10, 2019

totaam added the v2.4.x label Jan 22, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

html5 client visual corruption #2200

html5 client visual corruption #2200

totaam commented Mar 10, 2019

totaam commented Mar 10, 2019

totaam commented Mar 10, 2019

totaam commented Mar 10, 2019

totaam commented Mar 10, 2019

totaam commented Mar 10, 2019

totaam commented Mar 10, 2019

html5 client visual corruption #2200

html5 client visual corruption #2200

Comments

totaam commented Mar 10, 2019

2019-03-10 05:48:11: antoine created the issue

totaam commented Mar 10, 2019

2019-03-10 05:49:12: antoine changed status from new to assigned

totaam commented Mar 10, 2019

2019-03-10 05:49:12: antoine edited the issue description

totaam commented Mar 10, 2019

2019-03-10 08:12:25: antoine commented

totaam commented Mar 10, 2019

2019-03-10 09:43:55: antoine changed status from assigned to closed

totaam commented Mar 10, 2019

2019-03-10 09:43:55: antoine set resolution to fixed

totaam commented Mar 10, 2019

2019-03-10 09:43:55: antoine commented