Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s3 integrity exception when skipping content #232

Closed
brackxm opened this issue Jun 13, 2014 · 4 comments
Closed

s3 integrity exception when skipping content #232

brackxm opened this issue Jun 13, 2014 · 4 comments

Comments

@brackxm
Copy link

brackxm commented Jun 13, 2014

Skipping from an inputstream from S3Object.getObjectContent() results in an AmazonClientException when reading the content.

exception:

com.amazonaws.AmazonClientException: Unable to verify integrity of data download.  Client calculated content hash didn't match hash calculated by Amazon S3.  The data may be corrupt.
    at com.amazonaws.services.s3.internal.DigestValidationInputStream.validateMD5Digest(DigestValidationInputStream.java:79)
    at com.amazonaws.services.s3.internal.DigestValidationInputStream.read(DigestValidationInputStream.java:61)
    at java.io.FilterInputStream.read(FilterInputStream.java:133)
    at java.io.FilterInputStream.read(FilterInputStream.java:107)
    at com.amazonaws.util.IOUtils.toByteArray(IOUtils.java:34)

test case:

    @Test
    public void test() throws IOException {
        final String bucketName = "mybucket";
        final String key = "mykey";
        final AmazonS3 client = new AmazonS3Client();
        {
            final InputStream input = new ByteArrayInputStream(new byte[2]);
            client.putObject(bucketName, key, input, new ObjectMetadata());
        }
        {
            final S3Object object = client.getObject(bucketName, key);
            final InputStream content = object.getObjectContent();
            content.skip(1);
            IOUtils.toByteArray(content);
            content.close();
        }
    }
@brackxm
Copy link
Author

brackxm commented Jun 13, 2014

The digest is only calculated on the partial content.
The read() from the skip() is not percolated to the DigestInputStream.

@hansonchar
Copy link
Contributor

Taking a look

@hansonchar
Copy link
Contributor

This should be fixed in the next release (or so). Thanks for reporting!

@david-at-aws
Copy link
Contributor

This has been fixed in 1.8.0: http://aws.amazon.com/releasenotes/2874991743005522

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants