Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ParamValidationError when using DynamoDB paginator #2300

Open
usegev opened this issue Feb 20, 2020 · 12 comments
Open

ParamValidationError when using DynamoDB paginator #2300

usegev opened this issue Feb 20, 2020 · 12 comments
Labels
documentation This is a problem with documentation. dynamodb p2 This is a standard priority issue pagination

Comments

@usegev
Copy link

usegev commented Feb 20, 2020

I create a DynamoDB paginator like this:

    paginator = client.get_paginator('query')
    page_iterator = paginator.paginate(
        TableName=TABLE_NAME,
        IndexName=INDEX_NAME,
        KeyConditionExpression=Key('X').eq(x) & Key('Y').lte(y)
    )
    for page in page_iterator:
        print (page['Items'])
    return

When I run it, I receive the following error:

Exception has occurred: ParamValidationError
Parameter validation failed:
Invalid type for parameter KeyConditionExpression, value: <boto3.dynamodb.conditions.And object at 0x111524110>, type: <class 'boto3.dynamodb.conditions.And'>, valid types: <class 'str'>

The exact same KeyConditionExpression works when I query the table directly:

    result = table.query(
        IndexName=INDEX_NAME,
        KeyConditionExpression=Key('X').eq(x) & Key('Y').lte(y)
    )

BTW: Documentation says that KeyConditionExpression should be a string and not some condition built this way.

@swetashre swetashre self-assigned this Feb 20, 2020
@swetashre
Copy link
Contributor

@usegev - Thank you for your post. I am able to reproduce the issue. Looking into it.

@swetashre swetashre added the investigating This issue is being investigated and/or work is in progress to resolve the issue. label Feb 20, 2020
@swetashre
Copy link
Contributor

@usegev - We have a customization around resources that converts KeyConditionExpression type to string format that's why you are not getting error when query the table directly.

You can use the same format with paginator by using resource.meta.client. Something like this:

import boto3
res = boto3.resource('dynamodb')

paginator = res.meta.client.get_paginator('query')
page_iterator = paginator.paginate(
        TableName=TABLE_NAME,
        IndexName=INDEX_NAME,
        KeyConditionExpression=Key('X').eq(x) & Key('Y').lte(y)
    )

Hope it helps and please let me know if you have any questions.

@swetashre swetashre added closing-soon This issue will automatically close in 4 days unless further comments are made. and removed investigating This issue is being investigated and/or work is in progress to resolve the issue. labels Feb 24, 2020
@usegev
Copy link
Author

usegev commented Feb 26, 2020

Thanks @swetashre, works as you described. Will the documentation be updated? Or will the client paginator support it as well?

@no-response no-response bot removed the closing-soon This issue will automatically close in 4 days unless further comments are made. label Feb 26, 2020
@swetashre
Copy link
Contributor

We are tracking this issue internally. We are working to document this behavior.

@swetashre swetashre added the documentation This is a problem with documentation. label Feb 26, 2020
@saintross
Copy link

thanks @swetashre this was making me scratch my head

@kulbida
Copy link

kulbida commented Apr 17, 2020

I am having the same issue with KeyConditionExpression for query and paginator

dynamodb = boto3.client('dynamodb',  . . . )
items = dynamodb.query(TableName='test',
    KeyConditionExpression=Key('user_email').eq('test@example.com'))

using paginator:

paginator = dynamodb.get_paginator('query')
items = paginator.paginate(TableName='test',
    KeyConditionExpression=Key('user_email').eq('test@example.com'))

getting:

Invalid type for parameter KeyConditionExpression,
value: <boto3.dynamodb.conditions.Equals object at 0x7f286128e990>,
type: <class 'boto3.dynamodb.conditions.Equals'>, valid types: <class 'str'>

@ChetanaYogeesh-zz
Copy link

What is the status on this issue ?

@alexsanderp
Copy link

I'm having this same problem.

@swetashre swetashre assigned kdaily and unassigned swetashre Mar 25, 2021
@ianscottknight
Copy link

Any updates on this? I'm getting the same issue as @kulbida

@koren-at-fundbox
Copy link

Any update?

@sebas-nicholls
Copy link

I am having the same issue on October 2021

@aadidubey7
Copy link

aadidubey7 commented Apr 21, 2022

response = client.query(
TableName=table_name,
IndexName="user_id",
KeyConditionExpression="user_id = :user_id",
ExpressionAttributeValues= {
":user_id": {
"S": "XYZ"
}
}
)

This works for me!

@aBurmeseDev aBurmeseDev added the p2 This is a standard priority issue label Nov 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation This is a problem with documentation. dynamodb p2 This is a standard priority issue pagination
Projects
None yet
Development

No branches or pull requests