Investigate HTTP Status Code 0

[wilhelmberg]

my current hunch

Based on Unity being single threaded and flaws with our current implementation (yet to be pinpointed):

• too many requests are fired (2 for each tile) not an issue, see below Investigate HTTP Status Code 0 #332 (comment)
• lot of requests block each other leading to timeouts

evidence

Tried with Unity@2017.3.0b5 and Unity@2017.3.0b6.

Steps to reproduce:

• branch ErrorCallback
• Globe example
• zoom 6 (remove zoom level safeguard in GlobeTileProvider.cs
• add HandleTileErrorExample.cs to Map game object
• clear cache
• Play

Results:

• Unity seems to be completely frozen and unresponsive for ~35 seconds (on my machine)
• Not even a single tile makes it onto the globe:
  
• Nor do they make it into the cache:
  
• But a whole bunch (several MB) gets downloaded:
  
• All tiles seem to get requested twice:
  
• Only after several hundreds of timeouts tiles also get Status Code 0:
  

[wilhelmberg]

further observations

• Disabling cache doesn't change anything (for bad or worse). I think it's save to say caching is not part of the problem
• Increasing the timeout parameter to eg 1000 seconds yields a lot less failed tiles but also another error (Cannot connect to destination host).
  
• With increased timeout and several runs (till all tiles are in the cache) the globe finally displays as expected:
  
  but memory usage is very high:
  
• exporting to Windows Standalone x64 player yields comparable results:
  • application seems to frozen
  • with default timeout (10s) no tiles are displayed
  • with timeout 1000s and empty cache memory usage also climbs to ~8GB but then the player sometimes crashes with:

    Read from location 00000110 caused an access violation.
    

  • after several runs (till all tiles are in the cache) globe looks like expected. Again with ~8GB memory usage.
• using Experimental .NET 4.6 Scripting Runtime yields comparable results

[wilhelmberg]

Analyzing web requests from outside of Unity (with Fiddler) shows that tiles don't get requested twice, seems it's only their error that shows up twice in the console.

Example session profiling web requests :

• at zoom 5
• should be 1,024 tiles
• Fiddler shows that 938 requests were sent and received
• leaves a difference of 86 tiles (1,024-938) that failed
• console shows 172 error messages which is double the number of failed tiles (86*2)

[wilhelmberg]

1 exception vs 2 exceptions per tile:

1 exception

These seem to be in an intermediate state and ran into the timeout when a response was already received but not processed.
Thus response does have a valid status code.
All of those that I've inspected had 200/OK and had at least some data attached.
Some of that data was the full tile and some not.

2 exceptions (with additional Status Code 0)

These seem to be the ones that ran into the timeout before even being fired or shortly thereafter.
Thus Status Code 0: no response from server included.

UnityWebRequest.timeout / UnityWebRequest.SendWebRequest()

I've also switched to the new UnityWebRequest.timeout and UnityWebRequest.SendWebRequest() but that didn't change anything.
fb994f0

[david-rhodes]

Possible duplicate exceptions as a results of #335?

[wilhelmberg]

Problem occurs when too many tile requests are fired at once.

Going to solve this via an queue wrapper around tile requests that throttles request according to the rate limits returned by API headers.

[jekelija]

Did this actually solve your problem? We are running into a similar issue in our code, but when i try to reproduce it using a test set that just fires off 200 requests at once, we do not get the error, so we assumed that it didnt have to do with many requests firing at once?