Datastore: Support Multi-user data access #3388

dgagnon · 2023-07-08T17:19:12Z

Description

By following the Getting started instructions and setting up the datastore Cloud Feature at this page: https://docs.amplify.aws/lib/datastore/sync/q/platform/flutter/#setup-cloud-sync

We can read this line: "DataStore's cloud synchronization uses the API category behind the scenes. Therefore, the first step is to add the API plugin."

This leads people to believing that they can use the documentation at the linked page.

In the same manner, the Setup authorization rules page (at https://docs.amplify.aws/lib/datastore/setup-auth-rules/q/platform/flutter/ ) refers a few times to the AppSync cli page (at https://docs.amplify.aws/cli/graphql/authorization-rules/#per-user--owner-based-data-access ).

On this page, a section is name "Multi-user data access". I can attest that the code in this section will not work with Datastore, I suspect due to the limitation on subscription and multi-owner models.

I tried all possible configuration (transformer v1, removing multi auth, experimental pipeline, other feature flags, as a single model, etc) and all tests were done with an empty datastore. After working 3 days on this, I am convinced there is no way to have this working in a native way.

Having a documentation that is representative of the available feature is of paramount importance to us.

Note: While this could be framed as a bug in the datastore/appsync plugins for this specific example, this is a recurring theme with these two plugins. See #1566 (comment) for technical details.

Steps to Reproduce

Follow the getting started instructions and create a new project
Add auth (default settings without social + iam as secondary auth)
Add api and datastore as per the documentation
Add a multi-owner model
Try to start the app.

Screenshots

Relevant Logs:

[  +45 ms] E/amplify:aws-datastore( 6140): Failure encountered while attempting to start API sync.
[        ] E/amplify:aws-datastore( 6140): DataStoreException{message=DataStore subscriptionProcessor failed to start., cause=GraphQLResponseException{message=Subscription error for Task: [GraphQLResponse.Error{message='Validation error of type UnknownArgument: Unknown field argument authors @ 'onCreateTask'', locations='null', path='null', extensions='null'}], errors=[GraphQLResponse.Error{message='Validation error of type UnknownArgument: Unknown field argument authors @ 'onCreateTask'', locations='null', path='null', extensions='null'}], recoverySuggestion=See attached list of GraphQLResponse.Error objects.}, recoverySuggestion=Check your internet.}
[        ] E/amplify:aws-datastore( 6140): 	at com.amplifyframework.datastore.syncengine.Orchestrator.lambda$startApiSync$3$com-amplifyframework-datastore-syncengine-Orchestrator(Orchestrator.java:306)
[        ] E/amplify:aws-datastore( 6140): 	at com.amplifyframework.datastore.syncengine.Orchestrator$$ExternalSyntheticLambda6.subscribe(Unknown Source:2)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.internal.operators.completable.CompletableCreate.subscribeActual(CompletableCreate.java:40)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.core.Completable.subscribe(Completable.java:2850)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.internal.operators.completable.CompletablePeek.subscribeActual(CompletablePeek.java:51)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.core.Completable.subscribe(Completable.java:2850)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.internal.operators.completable.CompletablePeek.subscribeActual(CompletablePeek.java:51)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.core.Completable.subscribe(Completable.java:2850)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.internal.operators.completable.CompletablePeek.subscribeActual(CompletablePeek.java:51)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.core.Completable.subscribe(Completable.java:2850)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.internal.operators.completable.CompletableSubscribeOn$SubscribeOnObserver.run(CompletableSubscribeOn.java:64)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.core.Scheduler$DisposeTask.run(Scheduler.java:614)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:65)
[        ] E/amplify:aws-datastore( 6140): 	at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:56)
[        ] E/amplify:aws-datastore( 6140): 	at java.util.concurrent.FutureTask.run(FutureTask.java:264)
[        ] E/amplify:aws-datastore( 6140): 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:307)
[        ] E/amplify:aws-datastore( 6140): 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[        ] E/amplify:aws-datastore( 6140): 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
[        ] E/amplify:aws-datastore( 6140): 	at java.lang.Thread.run(Thread.java:1012)
[        ] E/amplify:aws-datastore( 6140): Caused by: GraphQLResponseException{message=Subscription error for Task: [GraphQLResponse.Error{message='Validation error of type UnknownArgument: Unknown field argument authors @ 'onCreateTask'', locations='null', path='null', extensions='null'}], errors=[GraphQLResponse.Error{message='Validation error of type UnknownArgument: Unknown field argument authors @ 'onCreateTask'', locations='null', path='null', extensions='null'}], recoverySuggestion=See attached list of GraphQLResponse.Error objects.}

Generated code:

input ModelSubscriptionTaskFilterInput {
  content: ModelSubscriptionStringInput
  and: [ModelSubscriptionTaskFilterInput]
  or: [ModelSubscriptionTaskFilterInput]
  _deleted: ModelBooleanInput
}

type Subscription {
  onCreateTask(filter: ModelSubscriptionTaskFilterInput): Task @aws_subscribe(mutations: ["createTask"]) @aws_iam @aws_cognito_user_pools
  onUpdateTask(filter: ModelSubscriptionTaskFilterInput): Task @aws_subscribe(mutations: ["updateTask"]) @aws_iam @aws_cognito_user_pools
  onDeleteTask(filter: ModelSubscriptionTaskFilterInput): Task @aws_subscribe(mutations: ["deleteTask"]) @aws_iam @aws_cognito_user_pools
}

Code to replicate:

  Future<void> configureAmplify() async {
    // Add any Amplify plugins you want to use
    final authPlugin = AmplifyAuthCognito();
    // await Amplify.addPlugin(authPlugin);

    // Add the following lines to your app initialization to add the DataStore plugin
    final datastorePlugin = AmplifyDataStore(
      modelProvider: ModelProvider.instance,
      // Be sure to add the authModeStrategy
      authModeStrategy: AuthModeStrategy.multiAuth,
    );
    // await Amplify.addPlugin(datastorePlugin);

    final api = AmplifyAPI();

    // You can use addPlugins if you are going to be adding multiple plugins
    await Amplify.addPlugins([authPlugin, api, datastorePlugin]);
    // await Amplify.addPlugins([authPlugin]);

    // Once Plugins are added, configure Amplify
    // Note: Amplify can only be configured once.
    try {
      await Amplify.configure(amplifyconfig);
    } on AmplifyAlreadyConfiguredException {
      safePrint("Tried to reconfigure Amplify; this can occur when your app restarts on Android.");
    }
  }

Platforms

Flutter Version

3.10.5

Amplify Flutter Version

1.2.0

Deployment Method

Amplify CLI

Schema

type Task @model @auth(rules: [{ allow: owner, ownerField: "authors" }]) {
    content: String
    authors: [String]
}

The text was updated successfully, but these errors were encountered:

fjnoyp · 2023-07-10T18:08:35Z

Hi @dgagnon thank you very much for sharing this issue, and apologies you had to spend a lot of time as a result ...

You are right, the documentation is misleading and there is no multi owner support in Datastore at the moment. We'll look to update the docs to clarify the limits of Datastore functionality.

dgagnon · 2023-07-11T02:04:16Z

This is a feature we are very much interested in, you can count this a feature request as well!

dgagnon · 2023-07-31T18:11:17Z

I have a work around that works pretty well, using VTL resolvers and adding a "sharedWith" field on models. Maintainable easily for queries and subscriptions, but a bit weird for mutations. If there is any interest, I will take the time to write it up here.

James1R · 2023-08-08T12:18:55Z

I have a work around that works pretty well, using VTL resolvers and adding a "sharedWith" field on models. Maintainable easily for queries and subscriptions, but a bit weird for mutations. If there is any interest, I will take the time to write it up here.

@dgagnon If you could that be great thank you.
My scenario is a chat room where chat room hosts can create a chat room and other users can be added read-only to the chat room.
Group isn't sufficient as auth rules need to be applied for each chat room. I would end up with an enournmas amount of groups.

dgagnon · 2023-08-08T19:15:43Z

@James1R I will do a better write up later, but here is the schema and the required VTL for it. Note that you also need a way to get the uuid of the user you wish to share to. You will need to adjust for your actual auth settings, as this affects the VTL generated. I build these by copying from the current ones I had. This is for a multi-auth config, with IAM, owner and groups.

schema.graphql:

enum Status {
    ACTIVE
    COMPLETED
    DELETED
}

enum UserStatus {
    ACTIVE
    SUSPENDED
    DELETED
}

type User @model @auth(rules: [{ allow: owner, ownerField: "owner"}, {allow: private, provider: iam}]) {
    id: ID! @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}])
    username: String! @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}])
    data: AWSJSON @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}])
    attributes: AWSJSON @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}])
    # https://docs.amplify.aws/lib/datastore/sync/q/platform/flutter/#advanced-use-case---query-instead-of-scan
    status: UserStatus @default(value: "ACTIVE" ) @index(name: "byStatus") @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}])
    owner: String @index(name: "byOwner") @auth(rules: [{ allow: owner, ownerField: "owner", operations: [read, create, delete] }, { allow: private, provider: iam }])
    tasks: [Task] @hasMany(indexName: "byUser", fields: ["id"])
    sharedWith: [String] @auth(rules: [{ allow: owner, ownerField: "owner" }, { allow: private, provider: iam }])
}
type Project @model @auth(rules: [{ allow: owner, ownerField: "owner"}, {allow: private, operations: [read]}, {allow: private, provider: iam}, { allow: groups, groups: ["admin"] }]) {
    id: ID! @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, {allow: private, operations: [read]}, { allow: groups, groups: ["admin"] }])
    title: String! @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, {allow: private, operations: [read]}, { allow: groups, groups: ["admin"] }])
    description: String @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, {allow: private, operations: [read]}, { allow: groups, groups: ["admin"] }])
    data: AWSJSON @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, {allow: private, operations: [read]}, { allow: groups, groups: ["admin"] }])
    status: Status @default(value: "ACTIVE" ) @index(name: "byStatus") @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, {allow: private, operations: [read]}, { allow: groups, groups: ["admin"] }])
    dueAt: AWSDateTime! @auth(rules: [{ allow: owner, ownerField: "owner"}, { allow: private, provider: iam }, {allow: private, operations: [read]}, { allow: groups, groups: ["admin"] }])
    owner: String @index(name: "byOwner") @auth(rules: [{ allow: owner, ownerField: "owner", operations: [read, create, delete] }, { allow: private, provider: iam }, {allow: private, operations: [read]},{ allow: groups, groups: ["admin"] }])
    tasks: [Task] @hasMany(indexName: "byProject", fields: ["id"]) #https://docs.amplify.aws/cli/graphql/data-modeling/#belongs-to-relationship
}
type Task @model @auth(rules: [{ allow: owner, ownerField: "owner"}, {allow: private, provider: iam}, { allow: groups, groups: ["admin"], operations: [read] }]) {
    id: ID @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, { allow: groups, groups: ["admin"], operations: [read] }])
    title: String! @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, { allow: groups, groups: ["admin"], operations: [read] }])
    description: String @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, { allow: groups, groups: ["admin"], operations: [read] }])
    data: AWSJSON @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, { allow: groups, groups: ["admin"], operations: [read] }])
    status: Status @default(value: "ACTIVE" ) @index(name: "byStatus") @auth(rules: [{ allow: owner, ownerField: "owner" }, {allow: private, provider: iam}, { allow: groups, groups: ["admin"] }])
    dueAt: AWSDateTime! @auth(rules: [{ allow: owner, ownerField: "owner" }, { allow: private, provider: iam }, { allow: groups, groups: ["admin"], operations: [read] }])
    # https://docs.amplify.aws/cli/graphql/authorization-rules/#multi-user-data-access
    #    owners: [String]
    # BUG: https://github.com/aws-amplify/amplify-flutter/issues/1566
    owner: String @index(name: "byOwner") @auth(rules: [{ allow: owner, ownerField: "owner", operations: [read, create, delete] }, { allow: private, provider: iam }, { allow: groups, groups: ["admin"], operations: [read] }])
    projectID: ID! @index(name: "byProject") #https://docs.amplify.aws/cli/graphql/data-modeling/#belongs-to-relationship
    project: Project! @belongsTo(fields: ["projectID"])
    sharedWith: [String] @auth(rules: [{ allow: owner, ownerField: "owner" }, { allow: private, provider: iam }, { allow: groups, groups: ["admin"], operations: [read] }])
    userID: ID @index(name: "byUser") #https://docs.amplify.aws/cli/graphql/data-modeling/#belongs-to-relationship
    user: User @belongsTo(fields: ["userID"])
}

amplify/backup/api*/resolvers/Query.getTask.postAuth.2.req.vtl:

## [Start] Enable sharing. **
#if( !$util.isNull($ctx.stash.authFilter) )
    #set( $authFilter = $ctx.stash.get("authFilter") )
    #set( $ownerClaim0 = $util.defaultIfNull($ctx.identity.claims.get("sub"), null) )
    #set( $currentClaim1 = $util.defaultIfNull($ctx.identity.claims.get(
        "username"), $util.defaultIfNull($ctx.identity.claims.get("cognito:username"), null)) )
    #if( !$util.isNull($ownerClaim0) && !$util.isNull($currentClaim1) )
        #set( $ownerClaim0 = "$ownerClaim0::$currentClaim1" )
        #if( !$util.isNull($ownerClaim0) )
            $util.qr($authFilter.or.add({"sharedWith": { "contains": $ownerClaim0 }}))
        #end
    #end
    #set( $role0_0 = $util.defaultIfNull($ctx.identity.claims.get("sub"), null) )
    #if( !$util.isNull($role0_0) )
        $util.qr($authFilter.or.add({"sharedWith": { "contains": $role0_0 }}))
    #end
    #set( $role0_1 = $util.defaultIfNull($ctx.identity.claims.get(
        "username"), $util.defaultIfNull($ctx.identity.claims.get("cognito:username"), null)) )
    #if( !$util.isNull($role0_1) )
        $util.qr($authFilter.or.add({"sharedWith": { "contains": $role0_1 }}))
    #end
    $util.qr($ctx.stash.put("authFilter", $authFilter))
#end
$util.toJson({})
## [End] Enable sharing. **

amplify/backup/api*/resolvers/Query.listTasks.postAuth.2.req.vtl:

## [Start] Enable sharing. **
#if( !$util.isNull($ctx.stash.authFilter) )
    #set( $authFilter = $ctx.stash.get("authFilter") )
    #set( $ownerClaim0 = $util.defaultIfNull($ctx.identity.claims.get("sub"), null) )
    #set( $currentClaim1 = $util.defaultIfNull($ctx.identity.claims.get(
        "username"), $util.defaultIfNull($ctx.identity.claims.get("cognito:username"), null)) )
    #if( !$util.isNull($ownerClaim0) && !$util.isNull($currentClaim1) )
        #set( $ownerClaim0 = "$ownerClaim0::$currentClaim1" )
        #if( !$util.isNull($ownerClaim0) )
            $util.qr($authFilter.or.add({"sharedWith": { "contains": $ownerClaim0 }}))
        #end
    #end
    #set( $role0_0 = $util.defaultIfNull($ctx.identity.claims.get("sub"), null) )
    #if( !$util.isNull($role0_0) )
        $util.qr($authFilter.or.add({"sharedWith": { "contains": $role0_0 }}))
    #end
    #set( $role0_1 = $util.defaultIfNull($ctx.identity.claims.get(
        "username"), $util.defaultIfNull($ctx.identity.claims.get("cognito:username"), null)) )
    #if( !$util.isNull($role0_1) )
        $util.qr($authFilter.or.add({"sharedWith": { "contains": $role0_1 }}))
    #end
    $util.qr($ctx.stash.put("authFilter", $authFilter))
#end
$util.toJson({})
## [End] Enable sharing. **

amplify/backup/api*/resolvers/Subscription.onDeleteTask.postAuth.2.req.vtl:

## [Start] Enable sharing. **
#if( !$util.isNull($ctx.args.filter) )
    #if( !$util.isNull($ctx.args.filter.or) )
        #set( $authFilter = $ctx.args.filter.or )
    #else
        #set( $authFilter = $ctx.args.filter.and[0].or )
    #end

    #set( $ownerClaim0 = $util.defaultIfNull($ctx.identity.claims.get("sub"), null) )
    #set( $currentClaim1 = $util.defaultIfNull($ctx.identity.claims.get(
        "username"), $util.defaultIfNull($ctx.identity.claims.get("cognito:username"), null)) )
    #if( !$util.isNull($ownerClaim0) && !$util.isNull($currentClaim1) )
        #set( $ownerClaim0 = "$ownerClaim0::$currentClaim1" )
        $util.qr($authFilter.add({ "sharedWith": { "contains": $currentClaim1 } }))
    #end

    #if( !$util.isNull($ctx.args.filter.or) )
        #set( $ctx.args.filter = { "or": $authFilter } )
    #else
        #set( $ctx.args.filter.and[0] = { "or": $authFilter } )
    #end
#end
$util.toJson({})
## [End] Enable sharing. **

amplify/backup/api*/resolvers/Subscription.onUpdateTask.postAuth.2.req.vtl:

## [Start] Enable sharing. **
#if( !$util.isNull($ctx.args.filter) )
    #if( !$util.isNull($ctx.args.filter.or) )
        #set( $authFilter = $ctx.args.filter.or )
    #else
        #set( $authFilter = $ctx.args.filter.and[0].or )
    #end

    #set( $ownerClaim0 = $util.defaultIfNull($ctx.identity.claims.get("sub"), null) )
    #set( $currentClaim1 = $util.defaultIfNull($ctx.identity.claims.get(
        "username"), $util.defaultIfNull($ctx.identity.claims.get("cognito:username"), null)) )
    #if( !$util.isNull($ownerClaim0) && !$util.isNull($currentClaim1) )
        #set( $ownerClaim0 = "$ownerClaim0::$currentClaim1" )
        $util.qr($authFilter.add({ "sharedWith": { "contains": $currentClaim1 } }))
    #end

    #if( !$util.isNull($ctx.args.filter.or) )
        #set( $ctx.args.filter = { "or": $authFilter } )
    #else
        #set( $ctx.args.filter.and[0] = { "or": $authFilter } )
    #end
#end
$util.toJson({})
## [End] Enable sharing. **

amplify/backup/api*/resolvers/Subscription.onCreateTask.postAuth.2.req.vtl:

## [Start] Enable sharing. **
#if( !$util.isNull($ctx.args.filter) )
    #if( !$util.isNull($ctx.args.filter.or) )
        #set( $authFilter = $ctx.args.filter.or )
    #else
        #set( $authFilter = $ctx.args.filter.and[0].or )
    #end

    #set( $ownerClaim0 = $util.defaultIfNull($ctx.identity.claims.get("sub"), null) )
    #set( $currentClaim1 = $util.defaultIfNull($ctx.identity.claims.get(
        "username"), $util.defaultIfNull($ctx.identity.claims.get("cognito:username"), null)) )
    #if( !$util.isNull($ownerClaim0) && !$util.isNull($currentClaim1) )
        #set( $ownerClaim0 = "$ownerClaim0::$currentClaim1" )
        $util.qr($authFilter.add({ "sharedWith": { "contains": $currentClaim1 } }))
    #end

    #if( !$util.isNull($ctx.args.filter.or) )
        #set( $ctx.args.filter = { "or": $authFilter } )
    #else
        #set( $ctx.args.filter.and[0] = { "or": $authFilter } )
    #end
#end
$util.toJson({})
## [End] Enable sharing. **

James1R · 2023-08-11T10:50:04Z

Thank you for putting this up. @dgagnon

James1R · 2024-02-15T22:33:48Z

@fjnoyp is there any progress on this as a feature?

osehmathias · 2024-02-28T10:25:04Z

Also interested. 😓

fjnoyp added Documentation Improvements or fixes to public documentation (docs.amplify.aws, pub.dev, readmes). datastore Issues related to the DataStore Category labels Jul 8, 2023

fjnoyp self-assigned this Jul 10, 2023

fjnoyp added the feature-request A request for a new feature or an enhancement to an existing API or category. label Jul 11, 2023

fjnoyp changed the title ~~Documentation for flutter datastore is inadequate.~~ Datastore: Support Multi-user data access Jul 11, 2023

fjnoyp mentioned this issue Jul 12, 2023

fix: datastore clarify api boundary with api/cli aws-amplify/docs#5646

Closed

15 tasks

osehmathias mentioned this issue Feb 28, 2024

Is there a good way to selectively sync based on userID after auth? #4491

Closed

14 tasks

Equartey unassigned fjnoyp Feb 28, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Datastore: Support Multi-user data access #3388

Datastore: Support Multi-user data access #3388

dgagnon commented Jul 8, 2023 •

edited

Loading

fjnoyp commented Jul 10, 2023

dgagnon commented Jul 11, 2023

dgagnon commented Jul 31, 2023

James1R commented Aug 8, 2023

dgagnon commented Aug 8, 2023

James1R commented Aug 11, 2023 •

edited

Loading

James1R commented Feb 15, 2024

osehmathias commented Feb 28, 2024

Datastore: Support Multi-user data access #3388

Datastore: Support Multi-user data access #3388

Comments

dgagnon commented Jul 8, 2023 • edited Loading

Description

Categories

Steps to Reproduce

Screenshots

Platforms

Flutter Version

Amplify Flutter Version

Deployment Method

Schema

fjnoyp commented Jul 10, 2023

dgagnon commented Jul 11, 2023

dgagnon commented Jul 31, 2023

James1R commented Aug 8, 2023

dgagnon commented Aug 8, 2023

James1R commented Aug 11, 2023 • edited Loading

James1R commented Feb 15, 2024

osehmathias commented Feb 28, 2024

dgagnon commented Jul 8, 2023 •

edited

Loading

James1R commented Aug 11, 2023 •

edited

Loading