Create dynamo.StreamRecord to avoid json.Unmarshall issue #38

Merged
merged 1 commit into from Jan 8, 2017

Projects

None yet

3 participants

@tleef
Contributor
tleef commented Jan 5, 2017

The current use of dynamodbstreams.StreamRecord causes an error when the dynamo.Handle function tries to json.Unmarshall the data. This is because json.Unmarshall can't parse the unix timestamp as time.Time for ApproximateCreationDateTime.

Currently you get an error similar to this:

{"errorMessage":"parsing time \"1483585620\" as \"\"2006-01-02T15:04:05Z07:00\"\": cannot parse \"1483585620\" as \"\"\""}

There is an issue open for this on the aws-sdk-go repo here however, it can easily be fixed from this side by defining our own StreamRecord. I left Keys, NewImage, OldImage as map[string]*dynamodb.AttributeValue because they json.Unmarshall just fine and it is convenient for using dynamodbattribute.UnmarshalMap to convert them into application specific types when handling the event.

There was also an issue for this on the go-apex repo here but it is closed (I assume because the root cause is actually coming from the aws-sdk-go repo)

@harlow
Contributor
harlow commented Jan 7, 2017

Feels like a reasonable solution. We're using a patched version of the aws-sdk, let me know if you need some help testing.

+ OldImage map[string]*dynamodb.AttributeValue
+ SequenceNumber string
+ SizeBytes int64
+ StreamViewType string
@tj
tj Jan 7, 2017 Member

these ones don't need tags as well?

@tleef
tleef Jan 8, 2017 Contributor

No, tags are not required since the JSON keys have the same casing.

Example:

{
  "eventID": "a4667189b69049238be6e19dca1b0b68",
  "eventName": "INSERT",
  "eventVersion": "1.1",
  "eventSource": "aws:dynamodb",
  "awsRegion": "us-east-1",
  "dynamodb": {
    "ApproximateCreationDateTime": 1483834200,
    "Keys": {
      "id": {
        "S": "123"
      }
    },
    "NewImage": {
      "id": {
        "S": "123"
      },
      "name": {
        "S": "bob"
      }
    },
    "SequenceNumber": "123456700000000012345678901",
    "SizeBytes": 100,
    "StreamViewType": "NEW_IMAGE"
  },
  "eventSourceARN": "arn:aws:dynamodb:us-east-1:123456789012:table\/table_name\/stream\/2017-01-01T00:00:00.000"
}

docs

@tj
tj Jan 8, 2017 Member

weird ok cool

@tj
Member
tj commented Jan 7, 2017

Looks good to me if it looks fine to you guys haha, I'm not using Dynamo at the moment

@tj tj merged commit b962b60 into apex:master Jan 8, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment