low-latency-playback-over-gaps__t2.html fails in all 8 TVs in London plugfest

[jpiesing]

In the London plugfest, low-latency-playback-over-gaps__t2.html failed the following 2 observation in 5 or 6 of 8 TV sets and failed to execute to completion in two more.

• OF] Every video frame S[k,s] shall be rendered and the video frames shall be rendered in increasing presentation time order.
• [OF] Video: The presented sample shall match the one reported by the currentTime value within the tolerance of +/-(2/framerate + 20ms)

[jpiesing]

Not resuming playback after the gap should be reported as failure rather than TIMEOUT.

[FritzHeiden]

Not resuming playback after the gap should be reported as failure rather than TIMEOUT.

How should we detect this error? As this is unwanted behavior rather than an actual error that is thrown by the application, we would have to manually throw the error so it appears correctly in the results. Here is what comes to my mind: When play position is close enough to the beginning of the gap and the playback remains in waiting state for a certain time, throw an error.

What do you think?

[jpiesing]

The failure of currentTime is fixed by cta-wave/device-playback-task-force#123.

The failure of 'Every video frame' has the following error on TV 2.

FAIL: First frame found is 7, expected to start from 1. First frame number tolerance is 0. Mid frame number tolerance is 10. Frames out of order 368, 251. Last frame detected before gap 120 is within the tolerance of 'stall_tolerance_margin'=7.5 frames of expected frame 125. Total of missing frames is 6.

@yanj-github What is "Frames out of order"? How should that be interpreted?

[yanj-github]

@jpiesing It means frame 251 is detected after frame 368, I would go back to recording and navigate to frame 368 and goes frame by frame. If you have debug logs or the recording that can share with me I can have a look.

[jpiesing]

@louaybassbouss @FritzHeiden Please see my email about sharing recordings from London with Resillion. Both TVs 2 and 4 have this issue.

[louaybassbouss]

@louaybassbouss @FritzHeiden Please see my email about sharing recordings from London with Resillion. Both TVs 2 and 4 have this issue.

Upload in progress. Will send details to @yanj-github.

[jpiesing]

I've labelled this as "Blocker" on the basis that we want to include this test in the release.
If this turns out to be more complex than expected then we can drop the test from the release and remove "Blocker".

[yanj-github]

cta-wave/device-playback-task-force#125
I need help on this please.

[jpiesing]

cta-wave/device-playback-task-force#126 also applies here

[yanj-github]

cta-wave/device-playback-task-force#125

The failure of 'Every video frame' has the following error on TV 2.

FAIL: First frame found is 7, expected to start from 1. First frame number tolerance is 0. Mid frame number tolerance is 10. Frames out of order 368, 251. Last frame detected before gap 120 is within the tolerance of 'stall_tolerance_margin'=7.5 frames of expected frame 125. Total of missing frames is 6.

@yanj-github What is "Frames out of order"? How should that be interpreted?

Frame out of order caused by duplication:
What I observed from recording is that the playback started from beginning and it stopped at frame 120, then it jumped to frame 251 and played till 368 then jumped back again to frame 251 then played till the end. The duration from frame 251 till 368 is repeated.

Will this be test runner or device issue? The test contains duplicated duration. Frame 251 and frame 368, from the example.