Add configurable Error handling for HTTP requests

[swederik]

For the NCI Imaging Data Commons project, users will be able to access data without logging in, but a data usage limit will be put in place to prevent users from abusing the system to directly download studies.

Once the user reaches the data usage limit, the HTTP requests to the proxy will begin returning HTTP 429 Too Many Requests. The exact behaviour we want once this has been reached is currently unknown, but it will likely be something like showing a modal or redirecting the user to the IDC homepage or a form to request more resources.

In general, though, we should be able to configure error handling at the HTTP level for all of the requests that OHIF is making to the PACS. Although this is not in scope in this ticket, for other resources we can resurrect #1434 later on.

This means we will likely need to add error handling in calls to Cornerstone's loadAndCacheImage, and possibly at the level of cornerstoneWADOImageLoader and dicomweb-client.

I can imagine this would be useful for other cases, e.g. how to handle users suddenly receiving 401 responses from PACS.

[salimkanoun]

Hi,
I think the problem I just reported is linked to this one :
#1606

Probably there is a way to hit two birds with one stone,

Best regards,

Salim

[JamesAPetts]

This might be useful, error handlers can be added to cornerstone events fired by loadAndCacheImage here in cornerstoneTools.

https://github.com/cornerstonejs/cornerstoneTools/blob/master/src/stateManagement/loadHandlerManager.js

[wlongabaugh]

@swederik Per your question on IDC Slack, it looks good to go as an IDC:Priority, but I don't believe I have permissions here to modify labels or status. Thanks.

[wlongabaugh]

@JamesAPetts I don't seem to have a way to distinguish between the 429 status returned by the server and other HTTP errors; the error.status value that shows up in the handler is always equal to 0, presumably what is returned by XMLHttpRequest. I have triple-checked the server response and I am pretty sure it is sending back 429. I have only recently been able to go back and work on this. Note the idc-sandbox-002 proxy is set to trigger a 429 after about only 16MB of daily downloads.

[JamesAPetts]

Hey @wlongabaugh, Looking here under dicomweb-client, this response code really is what is being given by the XMLHttpRequest: https://github.com/dcmjs-org/dicomweb-client/blob/ee20303c5dc8d40955685e0e5b0021bb8ead27e2/src/api.js#L142.

XMLHttpRequest is native web api, and about as much control as we have on the client.

A zero error code can come from:

• Before the request completes, the value of status is 0.
• Browsers also report a status of 0 in case of XMLHttpRequest errors.

As this section is wrapped in a request.readyState === 4 if block , it cannot be the first option, as state 4 is ready, this must mean its hitting an XMLHttpRequest error.

I can confirm the 429 is being sent if I hit the requested series data directly in the browser
I just had a check and locally I am hitting a cors error when the 429 is being sent back, which the browser will always cast to a status code of 0. We don't get cors errors when our quota is not exceeded however.

https://idc-testing.appspot.com/projects/chc-tcia/locations/us-central1/datasets/tcga-stad/dicomStores/tcga-stad/study/1.3.6.1.4.1.14519.5.2.1.6354.4025.201007175423148952381133335609

When you return the 429 are you attaching the same Access-Control-Allow-Headers as you do when a 200 is returned?

Kind regards,
James

[fedorov]

Thanks for the quick response @JamesAPetts, I appreciate it!

[wlongabaugh]

Thanks for the analysis; it is very helpful. I don't think I am attaching the headers, and will go do that.

[wlongabaugh]

I have added the CORS headers to the 429 response, and a check indicates that it is getting through now. The error handler for the viewer in idc-dev is now set up to report quota exceeded only when a 429 arrives. Still need to test it with a low daily limit set up to confirm the issue can be closed. Thanks.

[fedorov]

@wlongabaugh are we all set with this issue - can it be closed?

[wlongabaugh]

I wanted to check that the page actually triggers when the quota is hit, and was waiting to test that, but wasn't getting the chance to put a low quota in dev. But I guess you got the (crappy original) quota page yesterday already (correct?) with the actual 4GB limit in place, so yeah it does work as designed. You can close.