ThroughputRule wrong calculation and duplicate loads of same fragment

[sebastien4]

Environment

• The MPD passes the DASH-IF Conformance Tool on http://dashif.org/conformance.html
• The stream has correct Access-Control-Allow-Origin headers (CORS)
• There are no network errors such as 404s in the browser console when trying to play the stream
• The issue observed is not mentioned on https://github.com/Dash-Industry-Forum/dash.js/wiki/FAQ
• The issue occurs in the latest reference client on http://dashif.org/reference/players/javascript/ and not just on my page
• Link to playable MPD file: Only possible in private
• Dash.js version: 2.2.0
• Browser name/version: Chrome 52
• OS name/version: Mac OSX 10.11.5

As discussed with @LloydW93 on slack, we are experiencing rebuffering issues with slow connections around 2.5Mbps, while 2 highest bitrates available are 1.6 and 3Mbps.
Attached is a typical ThroughputRule output from the logs (rebuffering.txt).
Calculation is totally wrong sometimes.

Also I can see duplicate loads of the same fragment in Chrome network tab (see duplicate.png).
It happens on live streams, did not check on vod streams for now.

[dsparacio]

The mismeasurements in the throughput rule are side effect due to the duplicated loads, second from cache. The duplicated loads are from this line

``js
var isEqualMedia = function (req1, req2) {
return !isNaN(req1.index) && (req1.index === req2.index) && (req1.startTime === req2.startTime) && (req1.adaptationIndex === req2.adaptationIndex);
};

**req1.index === req2.index** .   

Seems the real issue is in the Index handler with  this specific media we are duplicating index values for two different times.   

[dsparacio]

Also we should ignore cached chunks in average throughput measurement in the abr rules.... Maybe @LloydW93 has some code for this?

[dsparacio]

index 894 startTime 1469745816
index 894 startTime 1469745818
Index handler is not assigning correct index for time.

[dsparacio]

@sebastien4 If you would like a quick fix let's try this.
Can you make this simple modification and rebuild dash.js
L85 FragmentModel.js - replace with:

return !isNaN(req1.index) && (req1.startTime === req2.startTime) && (req1.adaptationIndex === req2.adaptationIndex);

In addition to it fixing the dublicate loads, I would like to verify that the ABR rules normalize. You should not see traces like this
Default Average throughput 433656 kbps
dash.all.min.js:2 [31259] ThroughputRule requesting switch to index: 3 type: video Priority: Default Average throughput 2315 kbps
dash.all.min.js:2 [32474] ThroughputRule requesting switch to index: 4 type: video Priority: Default Average throughput 662132 kbps

When I say quick fix, there may be a better fix in the DashHandler but we are looking at major changes there so that work will resolve this issue altogether. I will make sure it does.

[sebastien4]

@dsparacio Indeed, no more duplicate loads of chunks, and Throughput rule seems to work fine now. Let's see this in production now..

Thank you.

[dsparacio]

Fixed with PR #1523

[sebastien4]

@AkamaiDASH Still complaints from our users about rebufferings, whatever their available bandwidth. Not sure the quick fix solved all rebuffering issues but at least it solved duplicate loads.

There's sth going wrong with ABR algorithm and our live streams (liveDelay and stableBufferTime set to 30)..

[dsparacio]

@sebastien4 I need more info are you still seeing estimations that are way off. Could be the cache issue @LloydW93 spoke of. I will look to add a simple block of code that will ignore frags that are cache from skewing the average throughput.

[sebastien4]

No, estimations are correct. There must be sth wrong in my MPD settings..

[dsparacio]

Closing this issue. Tracking knownThroughput issues in new issue #1530