Skip to content

Implementing a Database API for Python Admin SDK #31

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 29 commits into from
Jun 1, 2017
Merged

Conversation

hiranya911
Copy link
Contributor

This PR implements a database API for the Admin SDK, based on the Firebase Database REST API. Related to #28

Summary of changes:

  • Added a get_token() method to App class. This can be used in the database module (or any other module) to obtain an OAuth2 token.
  • Implemented the db module with the single method get_reference(), which returns a db.Reference.
  • Implemented the types db.Reference and db.Query, which enable running queries and updates against the Firebase database.
  • Implemented the _services collection in the App class. Each service module (e.g. auth, db) get registered with the app as a service. This enables gracefully terminating services when an App is deleted.
  • Refactored the service initialization logic into a separate utils module, where it can be reused easily.
  • Implemented unit tests for all the new and changed functionality.
  • Implemented an integration test suite under the new integration/ directory. We can continue to flesh it out in the future.

What's supported?

  • Basic CRUD operations on database nodes.
  • Navigation up and down the database tree.
  • Order-by, filter and range queries.

What's not supported and other limitations?

  • No support for realtime event listeners.
  • No support for transactions.
  • No support for result sorting when running order-by-priority queries.

@hiranya911
Copy link
Contributor Author

The API of this module is still under review, but I think we can go ahead and start reviewing the implementation. I can update the PR if and when any changes are required as a result of the API review process.

Copy link

@wilhuff wilhuff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically LGTM with some nits.

def get_token(self):
"""Returns an OAuth2 bearer token.

This method may return a cached token. But it handles cache invalidation, and therefore
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You indentation here is inconsistent with the rest of the file (shift left by three spaces).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed indentation.

skewed_expiry = self._token.expiry - datetime.timedelta(seconds=_CLOCK_SKEW_SECONDS)
return _clock() < skewed_expiry

def _get_service(self, name, initializer):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's a service, and why would you want to perform this kind of registration?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A service is something we expose from the SDK (e.g. auth, database). By attaching each service to an App instance, we can gracefully terminate/stop them when the App is deleted. Here's the equivalent implementation from Java SDK: https://github.com/firebase/firebase-admin-java/blob/master/src/main/java/com/google/firebase/internal/FirebaseService.java

I've added docstrings to further explain this in the source.

with self._lock:
for service in self._services:
if hasattr(service, 'close'):
service.close()
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible for this to match non-method attributes and then fail with a TypeError when calling them. Depending on whether or not these are user-supplied services you may want to check if close is callable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like callable() is not available in Python 3.0 and 3.1. They dropped it and added it back in 3.2: https://docs.python.org/3/library/functions.html#callable

So I implemented this extra check by looking for the __call__ property on the function itself.

Also found a bug in the logic while testing it. I've fixed it and updated the tests to check for that.

# See the License for the specific language governing permissions and
# limitations under the License.

"""Firebase Database module.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In other repos when referring to the product it's "Firebase Realtime Database" or "Realtime Database". "Firebase Database" is not a thing and is a term we avoid because of potential confusion.

Note that when referring to a user's project-specific storage area "database" is fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


"""Firebase Database module.

This module contains functions and classes that facilitate interacting with the Firebase database.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/Firebase database/Firebase Realtime Database/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

return _Client('https://{0}'.format(parsed.netloc), _OAuth(app), requests.Session())

def request(self, method, urlpath, **kwargs):
return self._do_request(method, urlpath, **kwargs).json()
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments about what these do would be helpful, especially for what are valid values of method and what kinds of keyword arguments are taken here.

"""
cert_path = _get_cert_path(request)
with open(cert_path) as cert:
project_id = json.load(cert).get('project_id')
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really how this works? Isn't project_id the Google Cloud Project ID? Or are those the same as the realtime database name?

Aren't there are group of realtime databases that have different URLs than their project IDs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems the project_id field in the service account key file we download from Firebase console is same as the realtime database name of the project. We have been using this convention in other 2 Admin SDKs too for testing. I'll try to get it confirmed from the database server team.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are legacy projects that came over from pre-cloud Firebase where these values do not match. In new projects they match.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. I think what we have here is good enough for integration tests. It seems it should work fine for any project we create today or in the future.

assert ref.get_value() == value
assert ref.get_priority() == 2

def test_update_children(self, testref):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might consider a test for update_children that updates an existing value, preserving attributes not in the value you pass.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

tests/test_db.py Outdated
firebase_admin.delete_app(app)
with pytest.raises(ValueError):
db.get_reference()

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

two newlines between top-level things

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

tests/test_db.py Outdated
assert isinstance(ordered, list)
assert ordered == expected

@pytest.mark.parametrize('value', [None, False, True, 0, 1, "foo"])
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

single quotes, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@wilhuff wilhuff assigned hiranya911 and unassigned wilhuff Jun 1, 2017
@hiranya911 hiranya911 assigned wilhuff and unassigned hiranya911 Jun 1, 2017
@hiranya911
Copy link
Contributor Author

Made the suggested changes. Sending back for a quick sanity check.

Copy link

@wilhuff wilhuff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wilhuff wilhuff assigned hiranya911 and unassigned wilhuff Jun 1, 2017
@hiranya911 hiranya911 merged commit 2246418 into master Jun 1, 2017
@hiranya911 hiranya911 deleted the hkj-simple-db branch June 1, 2017 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants