Pagination with Filtering in DynamoDB Template

[sid88in]

Schema:

type Query {
	getUserTwitterFeed(handle: String!): User
}

type Tweet {
	tweet_id: String!
	tweet: String!
	retweeted: Boolean!
	retweet_count: Int
	favorited: Boolean!
}

type TweetConnection {
	items: [Tweet]
	nextToken: String
}

type User {
	name: String!
	handle: String!
	location: String!
	description: String!
	followers_count: Int!
	friends_count: Int!
	favourites_count: Int!
	followers: [String]
	tweets(limit: Int, nextToken: String): TweetConnection
}

schema {
	query: Query
}

Request Mapping Template:

{
    "version": "2017-02-28",
    "operation": "Query",
    "query": {
        "expression": "handle = :handle",
        "expressionValues": {
            ":handle": {
                "S": "$context.source.handle"
            }
        }
    },    
    "limit": #if($context.arguments.limit) $context.arguments.limit #else 10 #end,
    "nextToken": #if($context.arguments.nextToken) "$context.arguments.nextToken" #else null #end
    #if(${context.arguments.keyword})
    ,"filter": {
        "expression": "contains (#tweet, :keyword)",
        "expressionNames" : {
          	"#tweet" : "tweet"
        },        
        "expressionValues": {
            ":keyword": { "S": "${context.arguments.keyword}" }
        }
      }
    #end   
}

Response Mapping Template:

{
    "items": $util.toJson($context.result.items),
    "nextToken": $util.toJson($context.result.nextToken)
}

Query:

I am trying to write a mapping template to paginate tweets with a given filter.
My dynamodb table has simple structure: hash key = handle, sort key = tweet_id

When I don't pass keyword parameter, everything runs fine. But now I added another condition to the mapping template i.e if the keyword is an input then filter tweets by keyword. But I see weird error in the above screenshot. Took reference to add Query Filters from here

Any idea what is happening? Thanks!

[mlshon]

Hi,
Thank you for reporting this issue.
We have a fix out, could you try it again?

Thanks.

[sid88in]

It worked!! :D:D thanks!

[harleyguru]

Hi @sid88in can you let me know how you handled the pagination on the filtered result?
The limitation is working based on just query before filtering.

[ghost]

Hi, have you find a way to get the pagination on the filtered result?
As for @harleyguru, limit is applied before filtering, thus meaning that if you limit to 10 tweets and they are not containing your keyword you will get no result with a nextToken to get next 10 tweets.

@mlshon is there any suggestion to get right pagination on filtered results?

[stock1232]

I am also interested in a solution for this. I would like to be able to filter a pagination result before the limit is taken into consideration

[stock1232]

I see that it works. However, I am looking to use a sort key "created" which contains a timestamp that I would like to sort based on the most recent timestamp created.

[tahafarooqui]

Is that fixed ?

[tahafarooqui]

I am using "version" : "2017-02-28", Still facing this issue.

[tahafarooqui]

@stock1232 @eleva Did you find any solution.

[mattiLeBlanc]

I find this whole dynamodb / appsync / filtering stuff unworkable.
If you filter you can't trust the nextToken for pagination, because you don't know if the next page has results that might be filtered.
The only way around it would be to have something in your sortkey that you can sort on so you group the records you want to keep after the filter and you can do some kind of predictive pagination.

We are really hitting limits with Appsync and Dynamo. Imagine having pagination in the middle of a 5 function pipeline resolver. You would have to run the whole pipeline 5 times :)