DynamoDb TransactWrite error 'We expected a VALUE token but got: ObjectStart' #1279
Description
When TransactWriteItems update/put hits a conditional check, the sdk sometimes throws the following error.
AmazonDynamoDBClient 102|2019-05-04T01:15:31.944Z|ERROR|Failed to unmarshall a service error response. --> Amazon.Runtime.AmazonClientException: We expected a VALUE token but got: ObjectStart
at Amazon.Runtime.Internal.Transform.JsonUnmarshallerContext.ReadText () [0x000d6] in <f265973e530645a2b3c9e875d4a9e300>:0
at Amazon.Runtime.Internal.Transform.SimpleTypeUnmarshaller`1[T].Unmarshall (Amazon.Runtime.Internal.Transform.JsonUnmarshallerContext context) [0x00007] in <f265973e530645a2b3c9e875d4a9e300>:0
at Amazon.Runtime.Internal.Transform.StringUnmarshaller.Unmarshall (Amazon.Runtime.Internal.Transform.JsonUnmarshallerContext context) [0x00000] in <f265973e530645a2b3c9e875d4a9e300>:0
at Amazon.Runtime.Internal.Transform.JsonErrorResponseUnmarshaller.GetValuesFromJsonIfPossible (Amazon.Runtime.Internal.Transform.JsonUnmarshallerContext context, System.String& type, System.String& message, System.String& code) [0x0003a] in <f265973e530645a2b3c9e875d4a9e300>:0
at Amazon.Runtime.Internal.Transform.JsonErrorResponseUnmarshaller.Unmarshall (Amazon.Runtime.Internal.Transform.JsonUnmarshallerContext context) [0x0003c] in <f265973e530645a2b3c9e875d4a9e300>:0
at Amazon.DynamoDBv2.Model.Internal.MarshallTransformations.TransactWriteItemsResponseUnmarshaller.UnmarshallException (Amazon.Runtime.Internal.Transform.JsonUnmarshallerContext context, System.Exception innerException, System.Net.HttpStatusCode statusCode) [0x0001a] in <c117f003185048cb9e815b6dea2cdc82>:0
at Amazon.Runtime.Internal.Transform.JsonResponseUnmarshaller.UnmarshallException (Amazon.Runtime.Internal.Transform.UnmarshallerContext input, System.Exception innerException, System.Net.HttpStatusCode statusCode) [0x00015] in <f265973e530645a2b3c9e875d4a9e300>:0
at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleException (Amazon.Runtime.IExecutionContext executionContext, Amazon.Runtime.Internal.HttpErrorResponseException exception) [0x0008a] in <f265973e530645a2b3c9e875d4a9e300>:0
<very long stacktrace cut>
The sdk incorrectly unmarshals dynamoDB's response. JsonErrorResponseUnmarshaller checks for json-keys ending with '__type', 'message', code'. If the value corresponding to that key is not a string, an exception is thrown.
This is no longer a safe assumption, since the dynamo API returns the items that failed the condition check.
example httpErrorResponse.ResponseBody after a conditioncheck failed (pretty-printed for readability):
{
"__type": "com.amazonaws.dynamodb.v20120810#TransactionCanceledException",
"CancellationReasons": [
{
"Item": {
"message": { "S": "hello" },
"sk": { "S": "1556926280" },
"pk": { "S": "N1CVl79PYi2HOoI0imR" }
},
"Code": "ConditionalCheckFailed",
"Message": "The conditional request failed"
},
{ "Code": "None" }
],
"message": "Transaction cancelled, please refer cancellation reasons for specific reasons [ConditionalCheckFailed, None]"
}
Notice that 'Item' contains an attribute with the name 'message', but the json value is a dictionary {"S":"hello"}, not a string.
relevant sdk code from JsonErrorResponseUnmarshaller.cs
(TestExpression uses EndsWith(v, StringComparison.OrdinalIgnoreCase))
private static void GetValuesFromJsonIfPossible(JsonUnmarshallerContext context, out string type, out string message, out string code)
{
code = null;
type = null;
message = null;
while (TryReadContext(context))
{
if (context.TestExpression("__type"))
{
type = StringUnmarshaller.GetInstance().Unmarshall(context);
continue;
}
if (context.TestExpression("message"))
{
message = StringUnmarshaller.GetInstance().Unmarshall(context);
continue;
}
if (context.TestExpression("code"))
{
code = StringUnmarshaller.GetInstance().Unmarshall(context);
continue;
}
}
}
One workaround (for applications) is to Try/Catch, check for the specific unmarshalling error, and treat it an a TransactionCanceledException. Needless to say, this is ugly and risky.
Expected Behavior
TransactWriteItems should throw TransactionCanceledException regardless of application attributeNames.
Current Behavior
TransactWriteItems throws AmazonClientException when attributeNames end with "__type", "message", "code". (case-insentive)
Possible Solution
TransactWriteItemsResponseUnmarshaller.UnmarshallException can parse the error similar to how QueryResponseUnmarshaller parses the response-body.
The Items can then be added to the exception. The java-sdk does exactly this.
Steps to Reproduce (for bugs)
TransactWriteItems that contains an Update with ConditionExpression, on an existing item.
The existing item must contain an attributeName that ends with "__type", "message", "code"
Your Environment
- AWSSDK.Core version used: sdk commit 0585ea6, built with dotnet 2.2.104
- Service assembly and version used: see above
- Operating System and version: windows10 64bit
- Visual Studio version: 15.7.5
- Targeted .NET platform: netstandard-2.0