S3 PutBucketMetricsConfiguration Receiving MalformedXML Response with API Example

[bflad]

Version of AWS SDK for Go?

1.8.39

Version of Go (go version)?

go version go1.8.3 darwin/amd64

What issue did you see?

Trying to use s3.PutBucketMetricsConfiguration with the simplest example of the API documentation (metrics for the whole bucket, no filters -- https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTMetricConfiguration.html#examples) is always receiving MalformedXML response, even though the request seems to match the documentation.

Steps to reproduce

package main

import (
	"log"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/s3"
)

func main() {
	bucket := "bflad-testing"
	metricsID := "EntireBucket"
	region := "us-east-1"

	sess := session.Must(session.NewSession())
	svcDebug := s3.New(sess, &aws.Config{
		LogLevel: aws.LogLevel(aws.LogDebugWithHTTPBody),
		Region:   aws.String(region),
	})
	svcInfo := s3.New(sess, &aws.Config{
		Region: aws.String(region),
	})

	// Create S3 Bucket
	log.Printf("Creating S3 bucket: %s", bucket)
	_, createBucketErr := svcInfo.CreateBucket(&s3.CreateBucketInput{
		Bucket: aws.String(bucket),
	})
	if createBucketErr != nil {
		log.Printf("Error creating S3 Bucket: %s", createBucketErr)
		return
	}

	// PUT S3 Bucket Metrics Configuration
	_, putBucketMetricsConfigurationErr := svcDebug.PutBucketMetricsConfiguration(&s3.PutBucketMetricsConfigurationInput{
		Bucket: aws.String(bucket),
		Id:     aws.String(metricsID),
		MetricsConfiguration: &s3.MetricsConfiguration{
			Id: aws.String(metricsID),
		},
	})
	if putBucketMetricsConfigurationErr != nil {
		log.Printf("Error putting S3 bucket metrics configuration: %s", putBucketMetricsConfigurationErr)
	}

	// Delete S3 Bucket
	log.Printf("Deleting S3 bucket: %s", bucket)
	_, deleteBucketErr := svcInfo.DeleteBucket(&s3.DeleteBucketInput{
		Bucket: aws.String(bucket),
	})
	if deleteBucketErr != nil {
		log.Printf("Error deleting S3 bucket: %s", deleteBucketErr)
		return
	}
}

go run put-bucket-metrics-configuration.go
2017/06/11 03:24:03 Creating S3 bucket: bflad-testing
2017/06/11 03:24:03 DEBUG: Request s3/PutBucketMetricsConfiguration Details:
---[ REQUEST POST-SIGN ]-----------------------------
PUT /?id=EntireBucket&metrics= HTTP/1.1
Host: bflad-testing.s3.amazonaws.com
User-Agent: aws-sdk-go/1.8.39 (go1.8.3; darwin; amd64)
Content-Length: 66
Authorization: AWS4-HMAC-SHA256 Credential=--OMITTED--/20170611/us-east-1/s3/aws4_request, SignedHeaders=content-length;host;x-amz-content-sha256;x-amz-date, Signature=a8f212911af4509161b578258e95661ecfa1ce34b37743b1c4901d93041a0202
X-Amz-Content-Sha256: 26226fc3ca57da9870772d88f45bb646f22eac611ac8bc7709309a6fb064cab5
X-Amz-Date: 20170611T072403Z
Accept-Encoding: gzip

<MetricsConfiguration><Id>EntireBucket</Id></MetricsConfiguration>
-----------------------------------------------------
2017/06/11 03:24:05 DEBUG: Response s3/PutBucketMetricsConfiguration Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 400 Bad Request
Connection: close
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Sun, 11 Jun 2017 07:24:04 GMT
Server: AmazonS3
X-Amz-Id-2: kJTIV+LQ2p3Bj/0bg71x4PcRLPhSzdhBN3I/KzNuIEusmWyZdxbqTuqXNgrp+TNWWHWaeYnHxkE=
X-Amz-Request-Id: 57647934848FFBFC


-----------------------------------------------------
2017/06/11 03:24:05 <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>MalformedXML</Code><Message>The XML you provided was not well-formed or did not validate against our published schema</Message><RequestId>57647934848FFBFC</RequestId><HostId>kJTIV+LQ2p3Bj/0bg71x4PcRLPhSzdhBN3I/KzNuIEusmWyZdxbqTuqXNgrp+TNWWHWaeYnHxkE=</HostId></Error>
2017/06/11 03:24:05 Error putting S3 bucket metrics configuration: MalformedXML: The XML you provided was not well-formed or did not validate against our published schema
	status code: 400, request id: 57647934848FFBFC, host id: kJTIV+LQ2p3Bj/0bg71x4PcRLPhSzdhBN3I/KzNuIEusmWyZdxbqTuqXNgrp+TNWWHWaeYnHxkE=
2017/06/11 03:24:05 Deleting S3 bucket: bflad-testing

https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTMetricConfiguration.html#examples shows request as:

PUT /?metrics&id=EntireBucket HTTP/1.1
Host: examplebucket.s3.amazonaws.com
x-amz-date: Thu, 15 Nov 2016 00:17:21 GMT
Authorization: signatureValue
Content-Length: 159

<?xml version="1.0" encoding="UTF-8"?>
<MetricsConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <Id>EntireBucket</Id>
</MetricsConfiguration>

[jasdel]

Thanks for reporting this issue @bflad. I investigated this issue and it looks like this API requires the xmlns="http://s3.amazonaws.com/doc/2006-03-01/ attribute on MetricsConfiguration. This seems to be more strict than other S3 APIs which no not require this namespace such as PutBucketAcl.

It looks like the SDK's code generation for S3's protocol drop's the modeled xmlNamespace attribute for API types. Marking this as a bug so the SDK can prioritize getting this issue fixed in the S3 service client.

As a rough workaround you could write the request body directly in a build handler to workaround this issue until the SDK fixes this bug.

// PUT S3 Bucket Metrics Configuration
_, err := svcDebug.PutBucketMetricsConfigurationWithContext(context.Background(), &s3.PutBucketMetricsConfigurationInput{
	Bucket: aws.String(bucket),
	Id:     aws.String(metricsID),
	MetricsConfiguration: &s3.MetricsConfiguration{
		Id: aws.String(metricsID),
	},
}, func(r *request.Request) {
	r.Handlers.Build.PushBack(func(req *request.Request) {
		req.SetReaderBody(bytes.NewReader([]byte(`<?xml version="1.0" encoding="UTF-8"?>
<MetricsConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Id>EntireBucket</Id>
</MetricsConfiguration>`)))
	})
})
if err != nil {
	log.Printf("Error putting S3 bucket metrics configuration: %s", err)
}

[jasdel]

Hi @bflad thanks for reporting this issue. I've merged in a fix that corrects this issue for S3 API operations. This change is available on tip and will be included in the SDK's next release.