Skip to content

Commit 00fc849

Browse files
joviegasjoviegas
authored
Design of DocumentAPI for DynamoDB (#3687)
1 parent 0330376 commit 00fc849

File tree

1 file changed

+176
-0
lines changed

1 file changed

+176
-0
lines changed

‎docs/design/services/dynamodb/high-level-library/DocumentAPI.md

Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
**Design:** New Feature, **Status:** Design
2+
3+
## Problem
4+
5+
In DynamoDB, an [item](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/WorkingWithItems.html) is a
6+
collection of attributes, each of which has a name and a value. Aws-sdk-java 1.x provided Document API to access these
7+
items. The user could access these items without actually knowing the complete schema of the entire item. This feature
8+
did not exist in Aws-sdk-java 2.x. This document proposes mechanism by which user will be able to access the DDB items
9+
as documents using enhanced dynamodb client.
10+
11+
### Requested features
12+
Aws-sdk-java 2.x should provide Document API similar to that of aws-sdk-java 1.x with following APIs
13+
14+
1. APIs to access DynamoDB for complicated data models without having to use DynamoDB Mapper.
15+
For example, APIs for converting from JSON to DynamoDb items & vice versa.
16+
2. APIs to manipulate semi structured data for each of the attribute values.
17+
For example, APIs to access the AttributeValue as string sets, number sets, string list, number list etc.
18+
3. Allow direct read and write of dynamoDB elements as Documents.
19+
20+
Example Github issue: https://github.com/aws/aws-sdk-java-v2/issues/36
21+
22+
## Current functionality
23+
Aws-sdk-java 2.x currently supports Mid-level DynamoDB mapper/abstraction for Java by providing Mapper Clients.
24+
While using these mappers the user needs to define the complete Table schema at the time of mapped table creation.
25+
26+
## Naming conventions
27+
Please note that the names of classes and api mentioned in this design document are not final and might get changed
28+
based on future reviews
29+
30+
## Proposed Solution
31+
32+
Add a new DocumentSchema in existing enhanced client. This schema just needs the primary key and sort key to be defined
33+
at the time of mapped table creation. This will retrieve the dynamo db table items as Documents.
34+
User can then access the attribute values from these documents by getters.
35+
Similarly , user can create a new Document and insert it to the mapped table by using the builder apis of Document.
36+
37+
### Enhanced Client Table Schema creation API
38+
The DocumentSchema is created by supplying partitionKey, sortKey and optional attributeConverterProviders in the builder.
39+
If AttributeConverterProvider are not supplied in TableSchema the Document Table schema will use the default converter
40+
providers.
41+
~~~java
42+
// Existing way of creating enhanced client
43+
DynamoDbEnhancedClient enhancedClient = DynamoDbEnhancedClient.builder().build();
44+
45+
// New API in TableSchema to create a DocumentTableSchema
46+
DocumentTableSchema documentTableSchema =
47+
TableSchema.fromDocumentSchemaBuilder()
48+
.addIndexPartitionKey(primaryIndexName(), "sample_hash_name", AttributeValueType.S)
49+
.addIndexSortKey("gsi_index", "sample_sort_name", AttributeValueType.N)
50+
.addAttributeConverterProviders(cutomAttributeConverters)
51+
.build();
52+
53+
// Existing API to access DynamoDB table.
54+
DynamoDbTable<EnhancedDocument> documentTable = enhancedClient.table("table-name", documentTableSchema);
55+
~~~
56+
*addAttributeConverterProviders = Appends custom attribute converter providers to the defaults provided by sdk*<br>
57+
58+
### Accessing Document from DDB table
59+
#### Reading a document to DDB Table
60+
The DocumentSchema mapped table returns items as EnhancedDocument,
61+
The EnhancedDocument can then be used to retrieve attribute values.
62+
63+
~~~java
64+
// Creating a document which defined primary key of the item needs to be retrieved
65+
EnhancedDocument hashKeyDocument = EnhancedDocument.builder()
66+
.addString("sample_hash_name", "sample_value")
67+
.build();
68+
// Retrieving from existing Get operation.
69+
EnhancedDocument retrievedDocument = documentTable.getItem(hashKeyDocument);
70+
71+
// Retrieving from existing Get operation
72+
EnhancedDocument documentTableItem = documentTable.getItem(
73+
EnhancedDocument.builder()
74+
.addString("sample_hash_name", "sample_value")
75+
.build());
76+
77+
// Accessing an attribute from document using generic getter.
78+
Number sampleSortvalue = documentTableItem.get("sample_sort_name", EnhancedType.of(Number.class));
79+
80+
// Accessing an attribute from document using specific getters.
81+
sampleSortvalue = documentTableItem.getSdkNumber("sample_sort_name");
82+
83+
// Accessing an attribute of custom class using custom converters.
84+
CustomClass customClass = documentTableItem.get("custom_nested_map", new CustomAttributeConverter()));
85+
86+
// Accessing Nested set
87+
Set<List<String>> stringSet = documentTableItem.get("string_set", new EnhancedType<Set<List<<String>>>(){}));
88+
~~~
89+
90+
91+
#### Writing a document to DDB Table
92+
The EnhancedDocument provides builder method to create documents that can be put to DocumentSchema mapped table.
93+
94+
~~~java
95+
96+
// Creating a document from Json input.
97+
EnhancedDocument documentFromJson = EnhancedDocument.fromJson(("{\"sample_hash_name\": \"sample_value_2\"}"));
98+
// put to dynamo db table
99+
documentTable.putItem(documentFromJson);
100+
101+
// Creating a document from EnhanceDocumentBuilders
102+
EnhancedDocument documentFromBuilder = EnhancedDocument.builder()
103+
.addString("sample_hash_name", "sample_value_2")
104+
.addNumber("sample_sort_name", 111)
105+
.addNumberList("sample_names", 1 ,2 ,3, 4)
106+
.build();
107+
108+
// put to dynamo db table
109+
documentTable.documentFromBuilder(documentFromBuilder);
110+
111+
// retrieving a document from dynamo db and updating some attributes
112+
EnhancedDocument documentTableItem = documentTable.getItem(hashKeyDocument);
113+
// using toBuilder to make a copy of the retrieved item and then modifying the key attribute
114+
EnhancedDocument changedValue = documentTableItem.toBuilder().addString("key-to-change", "changedValue").build();
115+
// put to dynamo db table
116+
documentTable.putItem(changedValue);
117+
~~~
118+
119+
120+
#### Attribute converter providers for EnhancedDocument
121+
122+
A builder method would be provided to add Attribute converters for an EnhancedDocument.
123+
The default value of attribute converter field would be null for EnhancedDocument.
124+
125+
Q: What converter providers will be used for EnhancedDocument retrieved from Dynamo db?<br>
126+
A: For the EnhancedDocuments retrieved from the SDK get/scan/query operations the DefaultAttributeConverterProviders would
127+
be assigned by default. If the user has provided attribute converter providers at the time of table creation then these
128+
converters will be used.
129+
130+
Q: What converter providers would be used for EnhancedDocument created by the user ?<br>
131+
A: The converter providers supplied by the user in the EnhancedDocument builders. If no converter providers are provided
132+
then user will get error while trying to get the attribute values. Thus, user should always supply defaultConverterProviders
133+
while creating the EnhancedDocuments for which user wants to access the attriute values latter.
134+
135+
136+
#### Getters and Setters for EnhancedDocuments
137+
138+
Q: What kind of getter API would be available for EnhancedDocument?<br>
139+
A: Following getter API would be available in EnhancedDocument
140+
1. Class specific getters like getString(), getNumber, getMap() same as V1.
141+
2. Generic getter API for any EnhancedType.
142+
3. Getters with ConverterProviders in ares
143+
144+
Q: What kind of setter API would be available for EnhancedDocument?<br>
145+
A: Following builder API would be available in EnhancedDocument
146+
1. Class specific builder like addString(), addNumber, addMap() same as V1.
147+
2. Generic builder API for any EnhancedType like add(value, EnhancedType).
148+
3. Builders for custom classes with Custom converter providers/
149+
150+
## Appendix B: Alternative solutions
151+
152+
### Design alternative: Using existing Document APIs from Sdk-core
153+
This design approach enhances the software.amazon.awssdk.core.document.Document with additional APIs enhancing it for
154+
better experience writing and reading open content.
155+
156+
#### Reading flat structures from DynamoDB
157+
~~~java
158+
Document document = table.getItem(Key.builder().partitionValue("0").build());
159+
// Current Document API
160+
String id = document.asMap().get("id").asString();
161+
Instant time = Instant.parse(document.asMap().get("time").asString());
162+
163+
// Document Converters API
164+
String id = document.get("id", String.class);
165+
Instant time = document.get("time", Instant.class);
166+
167+
// Document JSON API
168+
String json = document.toJsonString();
169+
170+
~~~
171+
172+
**Decision**
173+
174+
This alternative was discarded since it doesnot make use of existing converter providers.
175+
The Document will be required to be converted from key-AttributeValue map to Documents , for which new jsonNode converters
176+
needs to be implemented.

0 commit comments

Comments
 (0)