New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cloud Firestore Connector #1660

Closed
jannone opened this Issue Jan 17, 2018 · 2 comments

Comments

Projects
None yet
4 participants
@jannone
Copy link

jannone commented Jan 17, 2018

For feature requests, please fill in the next sections:

What feature are you missing?

A Cloud Firestore connector (from Google Firebase)

How could this feature look like in detail? Tradeoffs?

I guess much the same way as other NoSQL connectors that are being planned

@nakleiderer

This comment has been minimized.

Copy link

nakleiderer commented Mar 7, 2018

Here's a collection of considerations to know about when making this connector. Most of this is fact, some of this opinion, and I think the two are discernable in this comment. Take my opinions with a grain of salt.

I highly recommend reading the Firestore documentation to learn more about the product, but I've highlighted some important topics.

Types

Firestore GraphQL Notes
Array List
Boolean Boolean
Bytes String Custom Scalar
Date and time String Custom Scalar
Floating-point number Float
Geographical point Custom Type Ex: type GeoPoint { lat: string, long: string }
Integer Int
Map Custom Type
Null Null
Reference Custom Type
String String

Structure Data

The Firestore documentation goes into detail regarding the pros and cons of different data structures. I think this entire section is worth a read. My thought on this is to model everything, including relationships, as a root-level collection; however, the con to that is:

Getting data that is naturally hierarchical might become increasingly complex as your database grows.

I think the nature of GraphQL and this connector will help abstract away that concern.

Denormalization

Another concern is denormalizing data for performance and cost purposes. An advanced data model may want to aggregate data on the edge of a relationship (i.e. A like count). Firestore does not support aggregate queries, so it would be awesome if this binding managed the complexity of aggregating data in a denormalized way.

Query Data

A few quirks to note:

  • Firestore requires custom indexes to perform some queries:

You can also chain multiple where() methods to create more specific queries (logical AND). However, to combine the equality operator (==) with a range comparison (<, <=, >, or >=), make sure to create a custom index.

  • There are limitations to compound queries that will require some additional checking in Prisma's Where searches:

You can only perform range comparisons (<, <=, >, >=) on a single field

  • <, <=, >, and >= are the only supported search operators. There is no native contains, starts with, ends with, or any other string search operator in Firestore. Some of this behavior could potentially be emulated with range search or generated code in the binding, but that could get messy. External services like Algolia are what Firestore currently recommends for advanced searching.

  • Some, every, and none operators on lists or relations are not natively supported in Firestore.

Indexes

Cloud Firestore requires an index for every query, to ensure the best performance. All document fields are automatically indexed, so queries that only use equality clauses don't need additional indexes. If you attempt a compound query with a range clause that doesn't map to an existing index, you receive an error.

Given this, it's probably important to either expose an index directive on the data model, implicitly add indexes based on fields and types in the data model, or index all the things queryable by Prisma.

The indexes should be output in an indexes.json file with the following format:

{
    "indexes": [
        {
            // The name of the collection
            "collectionId": "restaurants",
            // The location of the indexed field, and the sort order
            "fields": [
                { "fieldPath": "city", "mode": "ASCENDING" },
                { "fieldPath": "avgRating", "mode": "DESCENDING" }
            ]
        },
    ]
}

Optimizations

A batched write can contain up to 500 operations and batching operations together reduces connection overhead resulting in faster data migration.

A additional thoughts

  • I think the binding generated from this should be able to directly use the Firestore SDK and not require a Prisma cluster to stand in the middle. The cluster seems overkill to place in front of what is already a highly available system. Perhaps there is a path for those who wish to collect tracing metrics in the Prisma dashboard. Maybe this could be determined by a config option (active vs. passthrough)
  • The Firestore admin SDK ignores security rules and does not currently support auth impersonation. This means that any querying done by a server will have full access to everything. People using this binding will likely not be able to use the Firestore Rules system and should look at enforcement in another part of the stack.
@stale

This comment has been minimized.

Copy link

stale bot commented Jan 9, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 10 days if no further activity occurs. Thank you for your contributions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment