AuthorizationHeaderMalformed when using S3 with VPCE

[klopfdreh]

Type:
Bug

Component:
S3

Describe the bug
When using a VPCE endpoint the region detection is not working correctly. I also tried to override the region by setting it with cloud.aws.s3.region, but this is ignored anyway. This might be an issue in the AmazonS3ClientFactory of spring-cloud-aws-core-2.4.1.jar, because here the region is derived from the host or if not matching the AmazonS3URI is used to get the region. The property is not read here. Is there any configuration which is not applied correctly?

Sample

pom.xml:

        <dependencyManagement>
          <dependencies>
            <dependency>
                <groupId>io.awspring.cloud</groupId>
                <artifactId>spring-cloud-aws-dependencies</artifactId>
                <version>2.4.1</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
          </dependencies>
        </dependencyManagement>

        <dependency>
            <groupId>io.awspring.cloud</groupId>
            <artifactId>spring-cloud-starter-aws</artifactId>
        </dependency>

application.yml (note: the endpoint has been modified and is no existing)

cloud:
  aws:
    s3:
      endpoint: https://bucket.vpce-0xx72x6xxx5d2842x-5icphead.s3.eu-central-1.vpce.amazonaws.com
      region: eu-central-1
    stack:
      auto: false
    region:
      static: eu-central-1
    credentials:
      accessKey: ...
      secretKey: ...

Error:

"Error while reading resources from path: s3://mybucket/subpath/*.gz","context":"default","exception":"com.amazonaws.services.s3.model.AmazonS3Exception: The authorization header is malformed; the region 'vpce' is wrong; expecting 'eu-central-1' (Service: Amazon S3; Status Code: 400; Error Code: AuthorizationHeaderMalformed; Request ID: <rid>; S3 Extended Request ID: <id>; Proxy: <proxy>)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1862)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1415)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1384)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1154)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:811)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:779)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:753)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:713)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:695)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:559)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:539)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5453)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5400)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5394)
at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:928)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at io.awspring.cloud.core.io.s3.AmazonS3ProxyFactory$SimpleStorageRedirectInterceptor.invoke(AmazonS3ProxyFactory.java:118)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at com.sun.proxy.$Proxy135.listObjects(Unknown Source)
at io.awspring.cloud.core.io.s3.PathMatchingSimpleStorageResourcePatternResolver.findProgressivelyWithPartialMatch(PathMatchingSimpleStorageResourcePatternResolver.java:228)
at io.awspring.cloud.core.io.s3.PathMatchingSimpleStorageResourcePatternResolver.findPathMatchingKeyInBucket(PathMatchingSimpleStorageResourcePatternResolver.java:177)
at io.awspring.cloud.core.io.s3.PathMatchingSimpleStorageResourcePatternResolver.findPathMatchingKeys(PathMatchingSimpleStorageResourcePatternResolver.java:146)
at io.awspring.cloud.core.io.s3.PathMatchingSimpleStorageResourcePatternResolver.findPathMatchingResources(PathMatchingSimpleStorageResourcePatternResolver.java:136)
at io.awspring.cloud.core.io.s3.PathMatchingSimpleStorageResourcePatternResolver.getResources(PathMatchingSimpleStorageResourcePatternResolver.java:102)

Thanks in advance.

[klopfdreh]

I was able to fix this by adding a configuration:

@Configuration
@Order(1)
public class S3Configuration {

    @Bean(name = "amazonS3Client")
    @Primary
    @Profile("!test")
    public AmazonS3 amazonS3Client(AWSCredentialsProvider credentialsProvider,
                                   RegionProvider regionProvider,
                                   @Value("${cloud.aws.s3.endpoint}") String endpoint) {
        return AmazonS3ClientBuilder.standard()
                .withCredentials(credentialsProvider)
                .withEndpointConfiguration(
                        new AwsClientBuilder.EndpointConfiguration(endpoint, regionProvider.getRegion().getName()))
                .build();
    }
}

This is only a workaround and shouldn't be necessary. It should work by only adding spring-cloud-starter-aws and the auto configuration provided by this dependency.

[maciejwalkowiak]

This issue is already fixed in main branch pointing to 3.0.0 version. @klopfdreh PRs for fixing it in 2.4.x are welcome!

[klopfdreh]

Hey @maciejwalkowiak thanks for the answer - is there any roadmap when 3.x.x is going to be released?

[maciejwalkowiak]

Milestone 1 in June, GA later this year but hard to say exactly when. Look https://github.com/awspring/spring-cloud-aws/milestones

[klopfdreh]

Thanks a lot for the hints! 👍

[maciejwalkowiak]

Fixed in 3.0

[klopfdreh]

Migrated to 3.0.0-M2 and it is working like expected! 👍 Thanks!