First image get_images for KVS is always missing

[daveisfera]

Describe the bug

The get_images call for KVS can return multiple images from a stream, but there is never an image for the first result and at least 2 have to be requested to get a valid image.

Expected Behavior

The first image would be valid when there was video to generate an image from

Current Behavior

ImageContent is never present on the first result

Reproduction Steps

#!/usr/bin/env python3

from base64 import b64decode
from datetime import datetime, timedelta, timezone

import boto3
import botocore.exceptions


def _main():
    kvs = boto3.client("kinesisvideo")

    stream_name = "mytest"

    response = kvs.get_data_endpoint(
        APIName="GET_IMAGES",
        StreamName=stream_name,
    )

    print(response["DataEndpoint"])

    kvs_media = boto3.client("kinesis-video-archived-media", endpoint_url=response["DataEndpoint"])

    try:
        start = datetime.now(tz=timezone.utc) - timedelta(seconds=10)
        response = kvs_media.get_images(
            StreamName=stream_name,
            #ImageSelectorType="SERVER_TIMESTAMP",
            ImageSelectorType='PRODUCER_TIMESTAMP',
            StartTimestamp=start,
            EndTimestamp=start + timedelta(seconds=6),
            SamplingInterval=3000,
            Format="JPEG",
            WidthPixels=640,
            HeightPixels=360,
            MaxResults=3,
        )

        print(response["ResponseMetadata"])

        print(len(response["Images"]))
        for n, img in enumerate(response["Images"]):
            print(n, img['TimeStamp'], list(img.keys()))
            try:
                if content := img['ImageContent']:
                    with open("tmp.jpg", "wb") as output:
                        output.write(b64decode(content))
                    print("Wrote")
                    break
            except KeyError:
                print("Missing")
    except botocore.exceptions.ClientError as error:
        print("ClientError", error)
    except botocore.exceptions.ParamValidationError as error:
        print("ParamValidationError", error)

Possible Solution

Return a valid result for the first image

Additional Information/Context

No response

SDK version used

1.26.134

Environment details (OS name and version, etc.)

macOS 13.3

[aBurmeseDev]

Hi @daveisfera - thanks for reaching out. I attempted to reproduce with similar code for get_images client but was unable to. A few questions to clarify:

1. Have you tried higher integer for MaxResults or not using it at all?
2. Have you verified the parameters you use that's filtering the outputs? ( ImageSelectorType, StartTimestamp etc..)?

I'd also suggest trying this get_paginator method from this GetImages API doc. If you're still see the same behavior, please share the debug logs (without any sensitive info) by adding boto3.set_stream_logger('') to your script. The logs would give us more insight into the issue.

Best,
John

[daveisfera]

Hi @daveisfera - thanks for reaching out. I attempted to reproduce with similar code for get_images client but was unable to. A few questions to clarify:

1. Have you tried higher integer for MaxResults or not using it at all?

I was trying to limit the response size. We just want 1 image and would preferably just get that one back. We've changed it to request 3 because occasionally the second will be empty as well

2. Have you verified the parameters you use that's filtering the outputs? ( ImageSelectorType, StartTimestamp etc..)?

Yes, the video is valid at that time and we can view it as HLS and get_clip

I'd also suggest trying this get_paginator method from this GetImages API doc.

Isn't that for getting large number of results? If I'm understanding the documentation correctly, the get_images API will automatically paginate if there's more than 100 images in the response for a request

If you're still see the same behavior, please share the debug logs (without any sensitive info) by adding boto3.set_stream_logger('') to your script. The logs would give us more insight into the issue.

I've attached the logs and in this case the first 2 images were missing.
logs_get_images.txt

[daveisfera]

Anything we can do to help get this resolved?

[aBurmeseDev]

Hi @daveisfera - sorry for not getting back to you sooner. I'm reaching out to Service API team to take a look at this. Meanwhile, can you try running this simplified version of your code?

kvs_client = boto3.client("kinesisvideo", region_name='us-west-2')

endpoint = kvs_client.get_data_endpoint(
    StreamName=stream_name,
    APIName="GET_IMAGES"
)['DataEndpoint']

kvs_archived_media_client = boto3.client("kinesis-video-archived-media",
                                         endpoint_url=endpoint,
                                         region_name='us-west-2')


resp = kvs_archived_media_client.get_images(StreamName=stream_name,
                                            ImageSelectorType='PRODUCER_TIMESTAMP',
                                            StartTimestamp=start_timestamp,
                                            EndTimestamp=end_timestamp,
                                            SamplingInterval=3000,
                                            Format='JPEG')
print(resp)

[github-actions]

Greetings! It looks like this issue hasn’t been active in longer than five days. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.

[daveisfera]

Here's the response, so it's the same:

{'ResponseMetadata': {'RequestId': 'a53c62f3-1738-4189-8a6f-6c3aa1372523', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'a53c62f3-1738-4189-8a6f-6c3aa1372523', 'content-type': 'application/json', 'content-length': '258729', 'date': 'Mon, 12 Jun 2023 20:28:43 GMT'}, 'RetryAttempts': 0}, 'Images': [{'TimeStamp': datetime.datetime(2023, 6, 2, 11, 37, tzinfo=tzlocal()), 'Error': 'NO_MEDIA'}, {'TimeStamp': datetime.datetime(2023, 6, 2, 11, 37, 3, tzinfo=tzlocal()), 'ImageContent': '...'}, {'TimeStamp': datetime.datetime(2023, 6, 2, 11, 37, 6, tzinfo=tzlocal()), 'ImageContent':, '...'}]}

[aBurmeseDev]

@daveisfera - thank you for giving it a shot. I'm still waiting to hear back from service team and will post update as soon as I hear back.

P91109273

[aBurmeseDev]

Hi @daveisfera - thanks for your patience. I just heard back from a Service API team member and here's their response.

getImages as well as other KVS consumption APIs like getClips, and HLS, DASH work by querying fragments in a time range. If the start time falls in the middle of a fragment boundary instead of aligning with the start of the fragment boundary, it results in the fragment containing start time to be not included in the query results. This leads to the missing image for the start time. We apologize for the clunky customer experience in this case. Customer can work around this behavior by subtracting an offset of fragment duration from the start time.

Please let me know if that resolves the issue for you and let me know if there are anymore questions.
Best,
John

[daveisfera]

That makes sense, but why return the empty/invalid entry? For example, with getClips, it will start the returned video with first valid fragment after the requested time. Can't getImages behave the same way and return the first valid entry instead of always starting with an empty/invalid one?

[aBurmeseDev]

Hi Dave @daveisfera, following up here as I just heard back from a service team member. I was advised that getImages doesn't behave the same way as getClips does, at the moment. getImages will try to return image for all the time stamp provided in the request input, if the fragment wasn't found in the query result, it will include empty/invalid image entry. The workaround is by filtering out the empty/invalid image entry in the result.

I understand this doesn't make your workflow easier but I can put in a feature request although I can't guarantee if that will get prioritized. Please let me know, thanks again!

[daveisfera]

Sounds good. Thanks for your help on this

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.