[RFC]: DataStore category for Amplify Flutter

[Amplifiyer]

This issue is a Request For Comments (RFC). It is intended to elicit community feedback regarding support for Amplify library in Flutter platform. Please feel free to post comments or questions here.

Purpose

This RFC is in continuation of this and expands on DataStore category and discusses few approaches being proposed.

Proposed Solution

High Level Overview

Of all the Amplify categories, Datastore is the trickiest due to the use of auto-generated (codegen) models that reside in the client’s application. The models are used when making Datastore calls such as query(Model.Type), save(Model()), and delete(Model()), where Model is a codegen type.

As mentioned in the overview RFC, one goal of Amplify Flutter is to reuse as much of the native Amplify libraries as possible, and to write only the minimum high-level API code in Dart, that Flutter developers can call from their Dart code. Hence in the Datastore category, our goal is to codegen user models in Dart, such that developers can write queries and mutations against it from Dart.

The problem arises when the request crosses over from Flutter/Dart to Native (iOS and android). The request is serialized at the flutter end and sent as a binary message to the native end of the Amplify flutter library. This means that there are no static types available on the native end of the Amplify flutter library to call Amplify Native library APIs iOS and android

The proposal here is that we will add support for calling native libraries' DataStore APIs that will accept ModelSchema as input rather than concrete Model Types. Find the supporting android RFC here.

DataStore APIs

Query

It’s complicated to mimic the exact behavior of query API of iOS and Android in flutter because of Dart language limitations. For instance, Dart Type objects are not generic (i.e. Type<? extends Model>) and it cannot be used to instantiate objects. To workaround this, we will create another generic “type” class ModelType to encapsulate the model types.

API Proposed

// Query the DataStore to find items of the requested model type, 
// filtered by QueryPredicate, and paginated and sorting (all three optional).
Future<List<T>> query<T extends Model>(
                   ModelType<T> modelType, 
                   {QueryPredicate where,
                    QueryPagination pagination,
                   List<QuerySortBy> sortBy}) {
  ...
  throw DataStoreException();
}

DX

Using 1) Futures
Future<List<Blog>> blogsFuture = query(Blog.classType);

Using 2) await
Blog blogById = 
    (await query(Blog.classType, where: Blog.ID.eq("123"))).get(0);

Using 3) then
query(Blog.classType, 
        where: <myQueryPredicate>, 
        pagination: <myQueryPagination>,
        sortBy: <myQuerySortBy>).then((blogs) {
    print("Retrieved Models " + blogs.size());
}

// classType is a static variable on Blog class which is an 
// instance of it's representative "type" class. Blog type/class will be auto-generated

Save

API Proposed

// Saves an item into the DataStore
// If predicate is present, check if data being overwritten matches the predicate. 
// This is useful for making sure that no data is 
// overwritten with an outdated/incorrect assumption.
Future<T> save<T extends Model>(
            @required T model, 
            {QueryPredicate predicate}) {
  ...
  throw DataStoreException();
}

DX

// Only showing `then` here for brevity
save(blogInstance).then((item) {
    print("Saved model" + item.toString());
});

delete

delete API is tricky since we have two distinct uses cases to support in this API but dart doesn't support method overloading.

1. Delete an item given it's model instance and an optional query predicate. Delete the instance conditionally if the provided query predicate is true (queryPredicate is sent to remote in delete mutations) e.g. delete this post if status.eq("draft")
2. Delete multiple items given modelType and a query predicate. Delete all the items that match the query predicate. e.g. delete all posts with status.eq("draft")

Instead of cramming everything in one delete() API we are proposing two APIs for each use case.

API Proposed

Future<T> deleteInstance<T extends Model>(
            @required T model, 
            {QueryPredicate when}) {
  ...
  throw DataStoreException();
}

Future<List<T>> deleteWhere<T extends Model>(
                   @required ModelType<T> modelType, 
                   {QueryPredicate where}) {
  ...
  throw DataStoreException();
}

DX

Future<Blog> deletedInstance = deleteInstance(blogInstance, when: Blog.status.eq("DRAFT"));

Future<List<Blog>> deletedBlogsFuture = deleteWhere(Blog.classType, where: Blog.rating.le(4));

clear

API Proposed

Future<void> clear()

DX

await clear();
print("successfully cleared datastore");

// Or
clear().then((justAVoid) {
    print("successfully cleared datastore");
});

observe

Currently there is no way to dynamically create an eventChannel (https://api.flutter.dev/flutter/services/EventChannel-class.html) between dart and native platform plugins so it’s not possible to create a new event channel for each subscription or even for every model (unless we codegen the bridge code)

Rather we will create only one data streaming channel for communicating all the datastore subscriptions. Both android and iOS flutter platform plugins will subscribe to all Models and transmit over a single EventChannel() i.e. all subscriptions for all models will be transmitted over this event channel. The filtering byId and by modelType will happen in Dart side of flutter (ok, some pun intended).

API Proposed

Stream<T> observe<T extends Model>({ModelType<T> classType, QueryPredicate where}) {
  const _channel = EventChannel('events');
  var allModelsStreamFromMethodChannel = _channel.receiveBroadcastStream() as Stream<Model>;
  return allModelsStreamFromMethodChannel.where((event) {
    var filterByModelType = classType != null ? event.instanceType == classType : true;
    var filterByPredicate = where != null ? evaluatePredicate(where, event) : true;
    return filterByPredicate && filterByModelType;
  }).asBroadcastStream() as Stream<T>;
}

DX

// DX 1.1 -> All callbacks provided in `listen()`
var streamSubscription_1 = observe(classType: Blog.classType).listen(
    (event) => print('Receved event + ' + event.toString()),
    onDone: () => print('Subscription has ended'),
    onError: (error) => print('Received error: $error'),
    cancelOnError: false);

// To cancel
await streamSubscription_1.cancel();

// DX 1.2 -> Only `onData()` provided in listen while others can be attached on streamSubscription
var streamSubscription_2 = observe(classType: Blog.classType)
    .listen((event) => print('Receved event + ' + event.toString()));

// Optionally attach onDone() later
streamSubscription_2.onDone(() {
  print('Subscription has ended');
});

// Optionally attach onError() later
streamSubscription_2.onError((error) {
  print('Received error: $error');
});

// to cancel
await streamSubscription_2.cancel();

// DX 1.3 -> Using streams as Iterable<Futures>
try {
  await for (var event in observe(classType: Blog.classType)) {
    print('Receved event + ' + event.toString());
}
  print('Subscription has ended');
} catch (error) {
  print('Received error: $error');
}

Appendix 1

Sample model that will be codegen by Amplify based on user's graphql schema

@immutable
class Blog extends Model {
  //@override
  final String id;

  final String name;

  final List<Post> posts;

  @override
  String getId() {
    return id;
  }

  // Constructors
  const Blog._internal({@required this.id, @required this.name, this.posts});

  factory Blog({String id, @required String name, List<Post> posts}) {
    return Blog._internal(
        id: id == null ? UUID.getUUID() : id,
        name: name,
        posts: posts != null ? List.unmodifiable(posts) : posts);
  }

  // Utility methods for immutability
  Blog copyWith(
      {@required String id, @required String name, List<Post> posts}) {
    return Blog(
        id: id ?? this.id, name: name ?? this.name, posts: posts ?? this.posts);
  }

  // De/serialization methods
  Blog.fromJson(Map<String, dynamic> json)
      : id = json['id'],
        name = json['name'],
        posts = json['posts'] is List
            ? (json['posts'] as List)
                .map((e) => Post.fromJson(e as Map<String, dynamic>))
                .toList()
            : null;

  Map<String, dynamic> toJson() => {'id': id, 'name': name, 'posts': posts};

  // Equals and toString() and hash()
  @override
  bool operator ==(Object other) {
    if (identical(other, this)) return true;
    return other is Blog &&
        id == other.id &&
        name == other.name &&
        DeepCollectionEquality().equals(posts, other.posts);
  }

  bool equals(Object other) {
    return this == other;
  }

  // Type metadata
  static const classType = BlogType();
}

class BlogType extends ModelType<Blog> {
  const BlogType();

  @override
  Blog fromJson(Map<String, dynamic> json) {
    return Blog.fromJson(json);
  }
}

DX

Blog someBlog; 
List<Comment> someComments; 

Post postWithComments = Post(
  title: "my post with comments", 
  blog: someBlog,
  comments: someComments,
); 

[Amplifiyer]

Amplify DataStore for flutter is now available as a developer preview. Get started here

[NimaSoroush]

I started integrating Amplify DataStore to my flutter project but there is a mismatch between what version of CLI to use for codegen

Flutter preview suggests to use npm install -g @aws-amplify/cli@flutter-preview

But

amplify codegen models is only available on npm install -g @aws-amplify/cli@latest

As aws-amplify/cli is installed globally we can't have these two versions installed simultaneously

[Ashish-Nanda]

@NimaSoroush you do not need the preview tag any more. You can go ahead and use @latest:

npm install -g @aws-amplify/cli

I'll make a PR to update the docs. Thanks for catching this!

[jamesonwilliams]

@NimaSoroush Can you please create a separate bug report for that issue, and remove it from this RFC? This is more discussing roadmap.