DynamoDB enhanced client: EnhancedDocument.getJson() produces invalid JSON

[twbecker]

Describe the bug

EnhancedDocument.getJson() incorrectly produces json that has single quotes backslash-escaped. Such escapes are neither required nor legal in JSON.

Example:

        dynamoTable.putItem(EnhancedDocument.builder()
                .putString("pk", "partitionKey")
                .putJson("json", "{ \"aKey\": \"A value with 'single quotes'\" }")
                .build());

        String invalidJson = dynamoTable.getItem(Key.builder().partitionValue("partitionKey").build())
                .getJson("json");
        System.out.println(invalidJson);

Produces {"aKey":"A value with \'single quotes\'"}. Again this is not valid and any JSON parser will choke on it:

com.fasterxml.jackson.core.JsonParseException: Unrecognized character escape ''' (code 39)
 at [Source: (String)"{"aKey":"A value with \'single quotes\'"}"; line: 1, column: 25]

	at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1840)
	at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:712)
...

Expected Behavior

I expect the getJson() method to produce valid JSON.

Current Behavior

See above, resulting JSON contains strings with invalid escape sequences (\').

Reproduction Steps

JUnit5 test, requires Jackson mapper and DynamoDbLocal:

import com.amazonaws.services.dynamodbv2.local.main.ServerRunner;
import com.amazonaws.services.dynamodbv2.local.server.DynamoDBProxyServer;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import software.amazon.awssdk.enhanced.dynamodb.*;
import software.amazon.awssdk.enhanced.dynamodb.document.EnhancedDocument;
import software.amazon.awssdk.services.dynamodb.DynamoDbClient;

import java.net.URI;

public class DeserializationBugTest {
    private static DynamoDbTable<EnhancedDocument> dynamoTable;

    @BeforeAll
    static void suiteSetUp() throws Exception {
        System.setProperty("sqlite4java.library.path", "target/native-libs");
        DynamoDBProxyServer dynamoServer = ServerRunner.createServerFromCommandLineArgs(new String[]{"-inMemory"});
        dynamoServer.start();

        DynamoDbEnhancedClient dynamoClient = DynamoDbEnhancedClient.builder()
                .dynamoDbClient(
                        DynamoDbClient.builder()
                                .endpointOverride(URI.create("http://localhost:8000"))
                                .build())
                .build();
        dynamoTable = dynamoClient
                .table("test-table", TableSchema.documentSchemaBuilder()
                        .addIndexPartitionKey(TableMetadata.primaryIndexName(), "pk", AttributeValueType.S)
                        .build());
        dynamoTable.createTable();
    }

    @Test
    void invalidJson() throws JsonProcessingException {
        dynamoTable.putItem(EnhancedDocument.builder()
                .putString("pk", "partitionKey")
                .putJson("json", "{ \"aKey\": \"A value with 'single quotes'\" }")
                .build());

        String invalidJson = dynamoTable.getItem(Key.builder().partitionValue("partitionKey").build())
                .getJson("json");
        System.out.println(invalidJson);
        new ObjectMapper().readTree(invalidJson);
    }
}

Possible Solution

If I understand correctly, single quotes do have to be escaped in DDB's internal JSON/map format, but those escapes should be removed when converting back to JSON.

Additional Information/Context

No response

AWS Java SDK version used

2.20.89

JDK version used

11.0.12

Operating System and version

Mac OS X 12.6.7

[joviegas]

Thanks @twbecker for reporting this.
I verified this behavior with V1 and confirmed that Single quotes should not be escaped

    @Test
    void testValidJson() throws JsonProcessingException {

        Item putItem = new Item()
                .withString("myHashKey", "partitionKey1.x''")
                .withNumber("myRangeKey", 3)
                .withJSON("json", "{ \"aKey\": \"A value   with 'single quotes'\" }");

        AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard().withRegion(Regions.US_EAST_1).build();
        DynamoDB dynamoDB = new DynamoDB(client);

        Table  dynamoDBTable = dynamoDB.getTable("test_table");

        dynamoDBTable.putItem(putItem);

        Item item = dynamoDBTable.getItem("myHashKey", "partitionKey1.x''", "myRangeKey", 3);

        System.out.println(item.toJSON());
        new ObjectMapper().readTree(item.toJSON());

I have fixed the issue.
Appreciate the details mentioned in the issues, really helpful.

Thanks @yasminetalby for letting me know about this 👍

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[jkbredsquiggles]

In which package and version is this fixed?

I see similar behaviour from the toJson method in software.amazon.awssdk.enhanced.dynamodb.internal.document.DefaultEnhancedDocument
which is in software.amazon.awssdk:dynamodb-enhanced:2.20.123