Cache calls in S3 and return the last result if there is a 500- API

[jwchumley]

Try to get more resiliency in the API (for temporary elastic search outage for example) to minimize impact on users.

Use Flask Hook after requests. And a hook for custom 500 error handling. After successful request, save response to S3. If 500 error return the last successful request and send alarm.

• Background research on how hooks in Flask work.
• Read up on Decorators.
• After that check back with 18F on how Hooks would work here using S3.
• Make hook and save to S3
• Retrieve

(Save error handling for next sprint)

[LindsayYoung]

After request docs
http://flask.pocoo.org/docs/0.12/api/#flask.Flask.after_request

Example exception handler from DataAct
https://github.com/fedspendingtransparency/data-act-broker-backend/blob/development/dataactbroker/app.py#L83

[pkfec]

documenting the steps on how to cache the web requests and upload web to s3:

• Extend the after request to print the results of the request. (remove later)
• use json.dumps if the request is not already in json
• save that file locally- make a tmp folder (remove later)
• replace save function with a boto3 function that you can write as webservices/util
• you will want all the values to be environment variables (they should already exist so look at the existing code )
• make this into a function you call instead of writing to local disk
• name the file with a folder prefix cached-calls/
• s3.upload_file("example.txt", BUCKET_NAME, "example-cn.txt")
• try that out locally with the dev s3 bucket, look up the vars and see if you can post.
• name files something deterministic (should be an example in downloads)
• when that is successful add it to the files that don't get deleted.
• that is webservices/tasks/download.py in the clear_bucket() function
• we will want to think about how long to keep the files- "expiration date in AWS"**

[pkfec]

Completed the upload section of cache calls (all web requests) and upload to s3.
I have submitted a PR for @LindsayYoung to review. Started working on retrieve the uploaded cache calls for below error codes:

• 500 Internal Server Error
• 502 Bad Gateway
• 503 Service Unavailable
• 504 Gateway Timeout

[AmyKort]

What's the status of this issue?

[AmyKort]

We've completed work described in this ticket. Work on this issue picks up at #2913