"400 Bad Request" error when uploading media from an in-memory binary stream in v4.0.0

[dprgarner]

I've recently tried to upgrade an app to use Tweepy v4, but I've been unable to as it throws errors whenever I try to upload media.

I'm attempting to upload a file that's been downloaded from AWS S3 and written to an in-memory binary stream. A simplified version of the code is as follows:

# The AWS S3 Client
s3 = boto3.client("s3")

with io.BytesIO(b"") as in_memory_bytes_io:
    # This downloads an image from S3 and writes the binary bytes to the in-memory IO stream.
    # https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.download_fileobj
    s3.download_fileobj("mybucket", "myfile.png", in_memory_bytes_io)

    # With Tweepy v3 this works fine, but with Tweepy v4 this raises a "400 Bad Request" error.
    uploaded_image = api.media_upload(
        filename="test.png",
        file=in_memory_bytes_io,
        chunked=True,
    )

I've ruled out issues with the S3 download method because this approach worked fine with Tweepy v3, and also because I see the same error if I fill in_memory_bytes_io in another way, e.g. if I replaced the s3.download_fileobj with the following:

with open("./mylocalfile.png", "rb") as local_file:
    in_memory_bytes_io.write(local_file.read())

The underlying error from the Twitter API response contains the message "Segments do not add up to provided total file size". I guess changes were made in this version which means that it no longer accurately finds the file's total size?

Any help would be much appreciated.

[Harmon758]

Tweepy v4.0.0 no longer automatically seeks to the beginning of the file object when uploading media.
This was an intentional change to allow uploading starting from a specific position in the stream.

Since you're writing to the BytesIO instance after it's initialized, the file pointer is at the end after the write, and media_upload is attempting to upload the equivalent of an empty file.
Adding in_memory_bytes_io.seek(0) before the media_upload call should fix this.

[dprgarner]

Seeking to the beginning of the stream fixed the issue. Thanks!