FR: Provide a "timeout" option

[mvalota-goreply]

I'm using firebase-admin v2.3.0 in a Google AppEngine app (Standard Environment, Python 2.7).
Unfortunately I'm getting a DeadlineExceededError once in a while when accessing my Firebase DB.

On Appengine the request timeout is 5 seconds, but it is possible to increase it up to 60 seconds, as described here:
https://cloud.google.com/appengine/articles/deadlineexceedederrors

I already tried to follow the official solution proposed by Google for Requests, but it doesn't seem to work:
https://cloud.google.com/appengine/docs/standard/python/issue-requests

I've seen that the google-auth library provides a timeout option in her Requests transport:
https://google-auth.readthedocs.io/en/latest/reference/google.auth.transport.requests.html#module-google.auth.transport.requests

It would be great to have a "timeout" option on Firebase App initialization, in order to avoid the DeadlineExceededError on AppEngine.

have a nice day!
Mauro

[hiranya911]

@mvalota Can you share the exact stacktrace/error message you're seeing? The 5 second timeout is for internal RPC calls made by App Engine. URL Fetch calls have a 60 second timeout by default:

Making requests to external URLs using URLFetch can also produce DeadlineExceededErrors
if the target website is having performance issues or normally takes more than 60 seconds
to reply. The logged stack trace of the DeadlineExceededErrors should contain calls to the
URLFetch libraries in these cases.

I would like to know what is actually triggering the deadline exceeded error.

[mvalota-goreply]

Here's the stacktrace:

An error occurred during a request. (/base/data/home/apps/s~project-name/20170901t114805.403796533333348168/main.py:94)
Traceback (most recent call last):
  File "/base/data/home/apps/s~project-name/20170901t114805.403796533333348168/lib/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/base/data/home/apps/s~project-name/20170901t114805.403796533333348168/lib/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/base/data/home/apps/s~project-name/20170901t114805.403796533333348168/lib/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/base/data/home/apps/s~project-name/20170901t114805.403796533333348168/lib/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/base/data/home/apps/s~project-name/20170901t114805.403796533333348168/lib/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/base/data/home/apps/s~project-name/20170901t114805.403796533333348168/modules/directory_module.py", line 98, in get_groups
    groups_array = firebase_module.get_groups()
  File "/base/data/home/apps/s~project-name/20170901t114805.403796533333348168/modules/firebase_module.py", line 396, in get_groups
    result_dict = ref.get()
  File "/base/data/home/apps/s~project-name/20170901t114805.403796533333348168/lib/firebase_admin/db.py", line 137, in get
    return self._client.request('get', self._add_suffix())
  File "/base/data/home/apps/s~project-name/20170901t114805.403796533333348168/lib/firebase_admin/db.py", line 681, in request
    resp = self._do_request(method, urlpath, **kwargs)
  File "/base/data/home/apps/s~project-name/20170901t114805.403796533333348168/lib/firebase_admin/db.py", line 716, in _do_request
    resp = self._session.request(method, self._url + urlpath, **kwargs)
  File "/base/data/home/apps/s~project-name/20170901t114805.403796533333348168/lib/google/auth/transport/requests.py", line 179, in request
    method, url, data=data, headers=request_headers, **kwargs)
  File "/base/data/home/apps/s~project-name/20170901t114805.403796533333348168/lib/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/base/data/home/apps/s~project-name/20170901t114805.403796533333348168/lib/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/base/data/home/apps/s~project-name/20170901t114805.403796533333348168/lib/requests/adapters.py", line 423, in send
    timeout=timeout
  File "/base/data/home/apps/s~project-name/20170901t114805.403796533333348168/lib/requests_toolbelt/adapters/appengine.py", line 170, in urlopen
    **response_kw)
  File "/base/data/home/apps/s~project-name/20170901t114805.403796533333348168/lib/requests/packages/urllib3/contrib/appengine.py", line 152, in urlopen
    raise TimeoutError(self, e)
TimeoutError: (<requests.packages.urllib3.contrib.appengine.AppEngineManager object at 0xfe3560f0>, DeadlineExceededError('Deadline exceeded while waiting for HTTP response from URL: https://project-name.firebaseio.com/groups.json',))

I received the last expected log at 12:32:56.773, while the error log was at 12:33:01.770 (~ 5 seconds later)

The lines that are causing this error are:
ref = db.reference('groups')
result_dict = ref.get()

But I had the same error on other requests to firebase db (get, update..)

[hiranya911]

Thank you @mvalota. If I push a fix to a separate branch, can you try it out and provide some feedback?

[mvalota-goreply]

Sure, I can try it. Thank you @hiranya911

[hiranya911]

@mvalota I have a fix in progress at #72. You can checkout the hkj-http-timeout branch to give it a try. I suppose you can copy the code from that branch into you App Engine app, and take it for a spin.

[mvalota-goreply]

Hi @hiranya911,
I just deployed a version of my App with your fix in order to test it.

I tried to read/write to Firebase, and it seems to work; but since the problem is not occurring regularly (I had a few timeouts today, while the last errors were returned on 05-06-09/09), I'll keep it checked in order to find any errors.

[mvalota-goreply]

Hi @hiranya911,
I released your fix one week ago, and since that release my application never returned a DeadlineExceededError occurrence.

[hiranya911]

@mvalota That's great news. I will get the fix merged into a release soon.

[hiranya911]

This was released today: https://firebase.google.com/support/release-notes/admin/python#2.4.0