PutObjectTaggingRequest: XML malformed error on Dell ECS

[ashishdhingra]

Discussed in #2114

^{Originally posted by Tobias-08 September 20, 2022}
Hi,

we are using Amazon S3 SDK (.NET) with Dell ECS. When sending a PutObjectTaggingRequest Dell ECS responds with a "xml malformed" error. I debugged this issue and learned that it is caused by a missing xmlns-attribute (xmlns="http://s3.amazonaws.com/doc/2006-03-01/") in the tagging-element. When I added the xmlns-attribute (manually in Fiddler) the request succeeded.

My question: Is it possible to inject the xmlns-attribute with means of the Amazon SDK (are there extensions points?)?

Thanks!
Tobias

[ashishdhingra]

While, PutObjectTagging does specify the below syntax for request, looks like the use of xmlns is not fully enforced by AWS S3 service.

PUT /{Key+}?tagging&versionId=VersionId HTTP/1.1
Host: Bucket.s3.amazonaws.com
Content-MD5: ContentMD5
x-amz-sdk-checksum-algorithm: ChecksumAlgorithm
x-amz-expected-bucket-owner: ExpectedBucketOwner
x-amz-request-payer: RequestPayer
<?xml version="1.0" encoding="UTF-8"?>
<Tagging xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
   <TagSet>
      <Tag>
         <Key>string</Key>
         <Value>string</Value>
      </Tag>
   </TagSet>
</Tagging>

Tried using Fiddler, and see the below request sent to AWS service:

PUT https://<<some-bucket>>.s3.us-east-2.amazonaws.com/src.txt?tagging HTTP/1.1
User-Agent: aws-sdk-dotnet-coreclr/3.7.9.56 aws-sdk-dotnet-core/3.7.13.7 .NET_Core/6.0.9 OS/Microsoft_Windows_10.0.19043 ClientAsync
amz-sdk-invocation-id: 9bc9098c-8aa2-4e07-91c9-a6a24387513f
amz-sdk-request: attempt=1; max=5
x-amz-security-token: <<REMOVED>>
Host: <<some-bucket>>.s3.us-east-2.amazonaws.com
X-Amz-Date: 20220921T213857Z
X-Amz-Content-SHA256: <<REMOVED>>
Authorization: AWS4-HMAC-SHA256 Credential=<<REMOVED>>/20220921/us-east-2/s3/aws4_request, SignedHeaders=content-md5;content-type;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=<<REMOVED>>
Content-Length: 130
Content-Type: application/xml
Content-MD5: NI28Rt55/vKkjuZr+ZitIg==

<Tagging><TagSet><Tag><Key>tag1</Key><Value>value1</Value></Tag><Tag><Key>tag2</Key><Value>value2</Value></Tag></TagSet></Tagging>

Java SDK does add xmlns attribute to request XML:

2022-09-22 11:48:40 [main] DEBUG software.amazon.awssdk.request:85 - Sending Request: DefaultSdkHttpFullRequest(httpMethod=PUT, protocol=https, host=duck-mcbucket-asia-pacific-two.s3.ap-south-1.amazonaws.com, port=443, encodedPath=/this_file_has_%C2%AE_in_the_name.txt, headers=[amz-sdk-invocation-id, Content-Length, Content-MD5, Content-Type, User-Agent], queryParameters=[tagging])
2022-09-22 11:48:41,673 [main] DEBUG org.apache.http.wire -  http-outgoing-0 >> "PUT /this_file_has_%C2%AE_in_the_name.txt?tagging HTTP/1.1[\r][\n]"
2022-09-22 11:48:41,673 [main] DEBUG org.apache.http.wire -  http-outgoing-0 >> "Host: <<some-bucket>>.s3.ap-south-1.amazonaws.com[\r][\n]"
2022-09-22 11:48:41,673 [main] DEBUG org.apache.http.wire -  http-outgoing-0 >> "amz-sdk-invocation-id: 63ee5888-348b-48fc-8e5a-eaf23d6e8b58[\r][\n]"
2022-09-22 11:48:41,673 [main] DEBUG org.apache.http.wire -  http-outgoing-0 >> "amz-sdk-request: attempt=1; max=4[\r][\n]"
2022-09-22 11:48:41,673 [main] DEBUG org.apache.http.wire -  http-outgoing-0 >> "Authorization: AWS4-HMAC-SHA256 Credential=<<REMOVED>>/20220922/ap-south-1/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-length;content-md5;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=<<REMOVED>>[\r][\n]"
2022-09-22 11:48:41,673 [main] DEBUG org.apache.http.wire -  http-outgoing-0 >> "Content-MD5: vN/wdX/b89dvCvFA7dnJuQ==[\r][\n]"
2022-09-22 11:48:41,673 [main] DEBUG org.apache.http.wire -  http-outgoing-0 >> "Content-Type: application/xml[\r][\n]"
2022-09-22 11:48:41,673 [main] DEBUG org.apache.http.wire -  http-outgoing-0 >> "User-Agent: aws-sdk-java/2.17.267 Mac_OS_X/12.6 OpenJDK_64-Bit_Server_VM/11.0.14+9-LTS Java/11.0.14 kotlin vendor/Amazon.com_Inc. io/sync http/Apache cfg/retry-mode/legacy ft/s3-transfer[\r][\n]"
2022-09-22 11:48:41,673 [main] DEBUG org.apache.http.wire -  http-outgoing-0 >> "x-amz-content-sha256: UNSIGNED-PAYLOAD[\r][\n]"
2022-09-22 11:48:41,673 [main] DEBUG org.apache.http.wire -  http-outgoing-0 >> "X-Amz-Date: 20220922T184840Z[\r][\n]"
2022-09-22 11:48:41,673 [main] DEBUG org.apache.http.wire -  http-outgoing-0 >> "X-Amz-Security-Token: <<REMOVED>>[\r][\n]"
2022-09-22 11:48:41,673 [main] DEBUG org.apache.http.wire -  http-outgoing-0 >> "Content-Length: 169[\r][\n]"
2022-09-22 11:48:41,673 [main] DEBUG org.apache.http.wire -  http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
2022-09-22 11:48:41,673 [main] DEBUG org.apache.http.wire -  http-outgoing-0 >> "Expect: 100-continue[\r][\n]"
2022-09-22 11:48:41,673 [main] DEBUG org.apache.http.wire -  http-outgoing-0 >> "[\r][\n]"
2022-09-22 11:48:41,951 [main] DEBUG org.apache.http.wire -  http-outgoing-0 << "HTTP/1.1 100 Continue[\r][\n]"
2022-09-22 11:48:41,951 [main] DEBUG org.apache.http.wire -  http-outgoing-0 << "[\r][\n]"
2022-09-22 11:48:41,953 [main] DEBUG org.apache.http.wire -  http-outgoing-0 >> "<?xml version="1.0" encoding="UTF-8"?><Tagging xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><TagSet><Tag><Key>tag1</Key><Value>value1</Value></Tag></TagSet></Tagging>"
2022-09-22 11:48:42,251 [main] DEBUG org.apache.http.wire -  http-outgoing-0 << "HTTP/1.1 200 OK[\r][\n]"
2022-09-22 11:48:42,251 [main] DEBUG org.apache.http.wire -  http-outgoing-0 << "x-amz-id-2: DGaAROftR+4y4X6wHYcmoNLmMWaJn6boPH4hsTbPD5YA69WdT886fJX3uphtxMm967e/bXXpZJc=[\r][\n]"
2022-09-22 11:48:42,251 [main] DEBUG org.apache.http.wire -  http-outgoing-0 << "x-amz-request-id: FMAKSKFACYQYSH9F[\r][\n]"
2022-09-22 11:48:42,251 [main] DEBUG org.apache.http.wire -  http-outgoing-0 << "Date: Thu, 22 Sep 2022 18:48:42 GMT[\r][\n]"
2022-09-22 11:48:42,251 [main] DEBUG org.apache.http.wire -  http-outgoing-0 << "Server: AmazonS3[\r][\n]"
2022-09-22 11:48:42,251 [main] DEBUG org.apache.http.wire -  http-outgoing-0 << "Content-Length: 0[\r][\n]"
2022-09-22 11:48:42,251 [main] DEBUG org.apache.http.wire -  http-outgoing-0 << "[\r][\n]"
2022-09-22 11:48:42 [main] DEBUG software.amazon.awssdk.request:85 - Received successful response: 200, Request ID: FMAKSKFACYQYSH9F, Extended Request ID: DGaAROftR+4y4X6wHYcmoNLmMWaJn6boPH4hsTbPD5YA69WdT886fJX3uphtxMm967e/bXXpZJc=

Even though S3 service does not complain and customer is using 3rd party service, we should be complaint with the API specification.

• Looks like AmazonS3Util. SerializeTaggingToXml() is not setting the xmlns attribute.
• Since S3 service is customized in SDK, we should check for other S3 service API operations that have the requirement of xmlns attribute.

[ashishdhingra]

Looks like almost all S3 bucket PUT operations (https://docs.aws.amazon.com/AmazonS3/latest/API/API_Operations_Amazon_Simple_Storage_Service.html) which send XML content in the request should have xmlns attribute set to http://s3.amazonaws.com/doc/2006-03-01/ in root element.

[ashishdhingra]

@Tobias-08 This has been fixed in latest version of AWSSDK.S3.

Thanks,
Ashish

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.