Query Exceptionless Fix

[bchong95]

Query Exceptionless Fix

Description

Bug

Queries were returning partial / inconsistent results when encountering 429s. The reason for this is that we were buffering exceptions and not setting the state correctly which ended up nulling out the continuation token. This ended up making the query iterator think that there was no more results instead of executing another roundtrip and getting back the 429.

Fix

For parallel (non order by) the fix was to just refactor the way we enumerate documents. The problem with parallel is that abstracted pages from the query pipeline which meant that parallel would always receive the next page of results when it got the end of one page. If that next page was a 429 we would buffer the 429 and send the results we have so far (but with a null continuation token). The solution was to return a continuation token at the end of each page and not buffer the 429. When the user comes back for the next page they will get the 429. ORDER BY was a trickier, since you will need to grab multiple pages in order to serve the sort order. For this I bit the bullet and returned the 429 and threw away any partial work. In the future we can add more buffering mechanisms to try and return the partial results.

Tests

None of the existing tests caught this situation. This is mainly because there are so many combinations that are hard to generate through unit tests. The fix was to wire through some test settings that will let the individual item producers know if we want to simulate edge cases like 429s and empty pages. From there the integration test hit all the failure cases.

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

[sboshra]

[j82w]

Please create an issue to track removing the TestInjection class with mocking.

[kirankumarkolli]

#1168