SelectObjectContent response error: missing :content-type header

[huachaohuang]

Bug Report

Version

├── aws-sdk-s3 v0.0.17-alpha (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14)
│   ├── aws-auth v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14)
│   │   ├── aws-types v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14)
│   │   │   ├── smithy-async v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14)
│   │   ├── smithy-async v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   │   ├── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14)
│   │   │   ├── smithy-eventstream v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14)
│   │   │   │   └── smithy-types v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14)
│   │   │   ├── smithy-types v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   ├── aws-endpoint v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14)
│   │   ├── aws-types v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   │   └── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   ├── aws-http v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14)
│   │   ├── aws-types v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   │   ├── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   │   ├── smithy-types v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   ├── aws-hyper v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14)
│   │   ├── aws-auth v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   │   ├── aws-endpoint v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   │   ├── aws-http v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   │   ├── aws-sig-auth v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14)
│   │   │   ├── aws-auth v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   │   │   ├── aws-sigv4 v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14)
│   │   │   │   └── smithy-eventstream v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   │   │   ├── aws-types v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   │   │   ├── smithy-eventstream v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   │   │   ├── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   │   ├── smithy-client v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14)
│   │   │   ├── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   │   │   ├── smithy-http-tower v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14)
│   │   │   │   ├── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   │   │   ├── smithy-types v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   │   ├── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   │   ├── smithy-http-tower v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   │   ├── smithy-types v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   ├── aws-sig-auth v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   ├── aws-types v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   ├── smithy-client v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   ├── smithy-eventstream v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   ├── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   ├── smithy-types v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
│   └── smithy-xml v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14)
├── aws-types v0.1.0 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)
├── smithy-http v0.0.1 (https://github.com/awslabs/aws-sdk-rust?tag=v0.0.17-alpha#74cd8a14) (*)

Platform

• Darwin Kernel Version 20.3.0
• Linux 5.4.0-1045-aws - Ubuntu SMP aarch64

AWS Services

S3

Description

Trying to use S3 SELECT command with Rust SDK.

I tried this code:

    async fn select(&self, key: &str, expr: &str) -> Result<SelectObjectContentOutput> {
        let parquet_input = InputSerialization::builder()
            .parquet(ParquetInput::builder().build())
            .build();
        let json_output = OutputSerialization::builder()
            .json(JsonOutput::builder().build())
            .build();
        let output = self
            .client
            .select_object_content()
            .bucket(self.bucket.clone())
            .key(key)
            .expression(expr)
            .expression_type(ExpressionType::Sql)
            .input_serialization(parquet_input)
            .output_serialization(json_output)
            .send()
            .await?;
        Ok(output)
    }

I ran it with the expression SELECT count(*) FROM S3Object on a large Parquet object (200MB).
It returned this error:

Error: ResponseError { err: Unmarshalling("expected response to include :content-type header, but it was missing"), raw: Message { headers: [Header { name: StrBytes { bytes: b":message-type" }, value: String(StrBytes { bytes: b"event" }) }, Header { name: StrBytes { bytes: b":event-type" }, value: String(StrBytes { bytes: b"Cont" }) }], payload: b"" } }

However, the same code seems to work on small Parquet files (1MB).

[jdisanti]

Thanks for reporting this! I was able to reproduce the issue and will work on a fix.

[jdisanti]

The fix for this went out in v0.0.20-alpha.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.