Skip to content

S3 DeleteObjects fails to serialize with certain unicode keys #547

@aajtodd

Description

@aajtodd

Describe the bug

Certain key names can trigger serialization issues with XML pull serializer.

Expected behavior

All valid keys should be serializable to XML

Current behavior


S3BucketOpsIntegrationTest > executionError FAILED                                                    
    java.lang.IllegalStateException: character 1 is not allowed in output (text value='\u0001')                    
        at org.xmlpull.mxp1_serializer.MXSerializer.writeElementContent(MXSerializer.java:977)                                                                 
        at org.xmlpull.mxp1_serializer.MXSerializer.text(MXSerializer.java:801)                                                                                    
        at aws.smithy.kotlin.runtime.serde.xml.XmlPullSerializer.text(XmlStreamWriterXmlPull.kt:69)
        at aws.smithy.kotlin.runtime.serde.xml.XmlSerializer.serializeString(XmlSerializer.kt:164)                                                  
        at aws.smithy.kotlin.runtime.serde.xml.XmlSerializer$field$3.invoke(XmlSerializer.kt:119)                                     
        at aws.smithy.kotlin.runtime.serde.xml.XmlSerializer$field$3.invoke(XmlSerializer.kt:119)                                     
        at aws.smithy.kotlin.runtime.serde.xml.XmlSerializer$tagOrAttribute$1.invoke(XmlSerializer.kt:93)                     
        at aws.smithy.kotlin.runtime.serde.xml.XmlSerializer$tagOrAttribute$1.invoke(XmlSerializer.kt:93)                     
        at aws.smithy.kotlin.runtime.serde.xml.XmlSerializerKt.writeTag(XmlSerializer.kt:357)              
        at aws.smithy.kotlin.runtime.serde.xml.XmlSerializerKt.access$writeTag(XmlSerializer.kt:1)         
        at aws.smithy.kotlin.runtime.serde.xml.XmlSerializer.tagOrAttribute(XmlSerializer.kt:93)            
        at aws.smithy.kotlin.runtime.serde.xml.XmlSerializer.field(XmlSerializer.kt:119)
        at aws.sdk.kotlin.services.s3.transform.ObjectIdentifierDocumentSerializerKt.serializeObjectIdentifierDocument(ObjectIdentifierDocumentSerializer.kt:33)
        at aws.sdk.kotlin.services.s3.transform.DeletePayloadSerializerKt$serializeDeletePayloadWithXmlNameDelete$1$1$1.invoke(DeletePayloadSerializer.kt:41)
        at aws.sdk.kotlin.services.s3.transform.DeletePayloadSerializerKt$serializeDeletePayloadWithXmlNameDelete$1$1$1.invoke(DeletePayloadSerializer.kt:41)
        at aws.smithy.kotlin.runtime.serde.SdkSerializableLambda.serialize(SdkSerializable.kt:25)
        at aws.smithy.kotlin.runtime.serde.xml.XmlListSerializer$serializeSdkSerializable$1.invoke(XmlSerializer.kt:330)
        at aws.smithy.kotlin.runtime.serde.xml.XmlListSerializer$serializeSdkSerializable$1.invoke(XmlSerializer.kt:329)
        at aws.smithy.kotlin.runtime.serde.xml.XmlSerializerKt.writeTag(XmlSerializer.kt:357)
        at aws.smithy.kotlin.runtime.serde.xml.XmlSerializerKt.access$writeTag(XmlSerializer.kt:1)
        at aws.smithy.kotlin.runtime.serde.xml.XmlListSerializer.serializeSdkSerializable(XmlSerializer.kt:329)
        at aws.sdk.kotlin.services.s3.transform.DeletePayloadSerializerKt$serializeDeletePayloadWithXmlNameDelete$1$1.invoke(DeletePayloadSerializer.kt:41)
        at aws.sdk.kotlin.services.s3.transform.DeletePayloadSerializerKt$serializeDeletePayloadWithXmlNameDelete$1$1.invoke(DeletePayloadSerializer.kt:39)
        at aws.smithy.kotlin.runtime.serde.xml.XmlSerializer.listField(XmlSerializer.kt:404)               
        at aws.sdk.kotlin.services.s3.transform.DeletePayloadSerializerKt.serializeDeletePayloadWithXmlNameDelete(DeletePayloadSerializer.kt:39)
        at aws.sdk.kotlin.services.s3.transform.DeleteObjectsOperationSerializer.serialize(DeleteObjectsOperationSerializer.kt:44)
        at aws.sdk.kotlin.services.s3.transform.DeleteObjectsOperationSerializer.serialize(DeleteObjectsOperationSerializer.kt:20)
        at aws.smithy.kotlin.runtime.http.operation.SdkOperationExecutionKt$decorate$2.invoke(SdkOperationExecution.kt:93)
        at aws.smithy.kotlin.runtime.http.operation.SdkOperationExecutionKt$decorate$2.invoke(SdkOperationExecution.kt:93)
        at aws.smithy.kotlin.runtime.http.operation.SerializeHandler.call(SdkOperationExecution.kt:120)
        at aws.smithy.kotlin.runtime.http.operation.SerializeHandler.call(SdkOperationExecution.kt:107)
        at aws.smithy.kotlin.runtime.http.operation.InitializeHandler.call(SdkOperationExecution.kt:104)
        at aws.smithy.kotlin.runtime.io.middleware.Phase.handle(Phase.kt:60)
        at aws.smithy.kotlin.runtime.io.middleware.DecoratedHandler.call(Middleware.kt:40)
        at aws.smithy.kotlin.runtime.http.operation.SdkHttpOperationKt.execute(SdkHttpOperation.kt:83)                                         
        at aws.smithy.kotlin.runtime.http.operation.SdkHttpOperationKt.roundTrip(SdkHttpOperation.kt:66)
        at aws.sdk.kotlin.services.s3.DefaultS3Client.deleteObjects(DefaultS3Client.kt:1650)
        at aws.sdk.kotlin.e2etest.S3TestUtils$deleteBucketAndAllContents$2$3$1.invokeSuspend(S3TestUtils.kt:80)                                     
        (Coroutine boundary)
        at aws.sdk.kotlin.e2etest.S3BucketOpsIntegrationTest$cleanup$1.invokeSuspend(S3IntegrationTest.kt:47)

Steps to Reproduce

This is easily triggered by our presign tests if you switch the deleteBucketAndAllContents to use batch delete.

But also roughly:

 client.deleteObject {
        bucket = bucketName
        key = `\u0001`
    }

Possible Solution

Likely will be OBE when #538 lands

Context

No response

AWS Kotlin SDK version used

v0.13.1-beta

Platform (JVM/JS/Native)

JVM

Operating System and version

All

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions