Skip to content

Latest commit

 

History

History
95 lines (69 loc) · 4.83 KB

SPIKE.md

File metadata and controls

95 lines (69 loc) · 4.83 KB
Raw

Spike: Robust handling of ObjectID type for MongoDB

Ref: loopbackio/loopback-next#3456

Table of contents

The Problem

ObjectID is a native MongoDB datatype, which is used as the datatype of its primary key (id) field; other fields can also be of ObjectID type.

In the current version of Juggler, the interpretation of ObjectID field is a sub-optimal experience supported by a complex determination process in an already complex codebase.

Examples of bad experiences:

  1. Fields need not be set to mongodb: {dataType: 'ObjectID'} to be interpreted as ObjectID.
  2. Unless strictObjectIDCoercion is set to true, any string that looks like an ObjectID will automatically get converted to ObjectID and you won't be able to find the object if you tried searching the database by that string, because it is not a string any more.
  3. To add to the confusion, this behavior can be configured at model and property levels.

Complexity is sometimes unavoidable, but a less complex state is always the desired state. It creates better experience for the consumers, and makes development and maintenance a more pleasant experience for the creators.

Proposed Solution

"Do not interpret any property not set with mongodb: {dataType: 'ObjectID'} as an ObjectID. Use string values for ObjectID in public APIs and manage the coercion processes for the user."

That sums up the proposed solution. Users will no longer have to deal with ObjectID anywhere in their experience with LoopBack/Juggler except only in one location - the model file.

Approaches

1. Top level ObjectID

A top level dataType: 'ObjectID' was suggested, but it was discarded for the following reason:

  1. If a top level ObjectID is supported for MongoDB, then database-specific types like MULTIPOLYGON, VARCHAR, NCLOB etc may also be required to be supported. Considering the number of database-specific types and the number of databases we support, we will end up with a really huge list.
  2. Inflexible when switching connectors. Cannot do this:
{
  type: 'number',
  jsonSchema: {
    format: 'int32',
  },
  mysql: {
    dataType: 'bit',
  },
  postgresql: {
    dataType: 'bit',
    length: 1,
  }
}

2. mongodb: {dataType: 'objectID'}

Setting a property to mongodb: {dataType: 'ObjectID'}, as it is already being done, is the better approach for marking the property as an ObjectID.

Developer Notes

  • The tests in test/new-objectid.test.js is a demonstration of the behavior of the mongodb: {dataType: 'ObjectID'} property.
  • The state of the result object is determined by both loopback-connector-mongodb/lib/mongodb.js and loopback-datasource-juggler/lib/dao.js, so changes are required in both files.
  • Changes in loopback-connector-mongodb/lib/mongodb.js:
    • All helper functions and properties like typeIsObjectId(), strictObjectIDCoercion etc., related to previous behavior of ObjectId should be removed or refactored.
    • Parts of the existing codebase does a good job of coercing string to ObjectID when querying the server.
    • Work is mostly required in coercing ObjectID back to string in database result. This is not a simple task. The object that's returned to the user is not generated by a standard pipeline and there objects can behave very differently depending on the query and undergo further modification in loopback-datasource-juggler/lib/dao.js.
  • Changes in loopback-datasource-juggler/lib/dao.js:
    • Coercion has to be applied in DAO for some queries.
    • Depending on the query, there may be different restrictions and rules.

In the time assigned for this spike I could find out the behavior of all the query methods. Also, I didn't have enough time to explore in deep how LB4 handles mongodb: {dataType: 'ObjectID'}. I experimented with a simple app with the changes in loopback-connector-mongodb/lib/mongodb.js and loopback-datasource-juggler/lib/dao.js but was hit with Type 'MongoDataSource' has no properties in common with type 'LifeCycleObserver'.; I wonder why it would happen.

Implementation proposal

  • Database-specific connectors should provide their coercion function to DAO.
  • DAO should apply the coercion function on the result object and return it to the user without changing the Model's actual properties.
  • Discover foreign key type without users needing to add anything more than customerId: string. Implementation may not be easy, but it is not impossible either, and will certainly be a great developer experience for our users.

Follow-up Tasks

  1. Implement the functionality as proposed in - Implementation Proposal.