Response contains both an empty result list and a NextToken when using filters

[stekern]

What issue did you see ?
I am getting both an empty result list AND a NextToken when calling boto3.client("ssm").list_command_invocations with a specific set of filters.

What does this response mean? Can I safely stop querying the API when I receive the first empty response list, or do I have to keep querying until there's no NextToken in the response? The former approach takes 0.5s, while the latter takes around 15s -- so not a trivial difference. In both cases the end result is still an empty list.

It is my understanding that the Filters argument applies the filters server-side. If I have to keep querying until there's no NextToken, however, I could've just as well performed the filtering client-side.

Steps to reproduce

import boto3

if __name__ == "__main__":
    ssm = boto3.client("ssm")
    response = ssm.list_command_invocations(
        Filters=[{"key": "Status", "value": "InProgress"}]
    )
    command_invocations = response["CommandInvocations"]
    while response.get("NextToken", ""):
        response = ssm.list_command_invocations(
            Filters=[{"key": "Status", "value": "InProgress"}],
            NextToken=response["NextToken"],
        )
        command_invocations += response["CommandInvocations"]

[swetashre]

@stekern - Thank you for your post. It’s possible if the command has no invocations but it should not give NextToken.
Are you getting any result in the end after calling with nextToken or every time you are getting an empty list ?

[stekern]

@swetashre The code I supplied above results in 23 requests being issued, and all of them have "CommandInvocations": [], so the final result is simply an empty list. The final response does not have NextToken set, which implies that whatever the server deems the "result set" has finally been exhausted -- even though all responses have contained empty lists.

Since there in my case are usually only 0 or 1-2 running (i.e., InProgress) command invocations at any given time, I would expect a single call to list_command_invocations with a filter on InProgress to give me all the results I need. Instead I need to issue tens of requests that offer empty results.

For what it's worth, I'm getting the same behavior when issuing the same request using AWS CLI (v1.18.107).

[swetashre]

@stekern - Thank you for your post. The response what you are seeing is coming from service. That's why you are getting the same behavior in AWS CLI as well.
I agree it should not take 23 calls to get 1-2 results. I can't debug this on client side as the response is coming from service. We would need more account specific information in order to debug this.
I would recommend reaching out to AWS Support for this.