Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LambdaInvokerFactory doesn't work with the event sources that AWS Lambda natively accepts #1344

Closed
DaichiUeura opened this issue Oct 16, 2017 · 4 comments
Labels
guidance Question that needs advice or information.

Comments

@DaichiUeura
Copy link

DaichiUeura commented Oct 16, 2017

For instance, when an input data based on the following POJO is passed to a Lambda function through aws-java-sdk-lambda SDK library, the Lambda function receives an empty data instead.

        private String eventSource;

        /**
         * Gets the event source
         * @return event source
         */
        public String getEventSource() {
            return eventSource;
        }

        /**
         * Sets the event source
         * @param eventSource A string containing the event source
         */
        public void setEventSource(String eventSource) {
            this.eventSource = eventSource;
        }

(this is a snippet from https://github.com/aws/aws-lambda-java-libs/blob/master/aws-lambda-java-events/src/main/java/com/amazonaws/services/lambda/runtime/events/SNSEvent.java)

The raw string after the serialization (by ObjectMapper in LambdaInvokerFactory) is below.

{"eventSource":"foobar"}

This results in an empty input to the Lambda function even if the same POJO is used. Not 'eventSource' but 'EventSource' seems to be expected for the JSON deserialization logic in AWS Lambda service.

@DaichiUeura
Copy link
Author

DaichiUeura commented Oct 18, 2017

In DynamoDB Streams case (i.e. input POJO class is DynamodbEvent), just using the upper camel case naming strategy on JSON element names(like the example case I described at first) doesn't work. It seems AWS SDK has the special marshallers and unmarshallers such as RecordMarshaller and StreamRecordMarshaller for JSON serialization/deserialization.

To resolve the issue, for instance, would it be possible to utilize them in LambdaInvokerFactory? so that we can invoke Lambda functions which have trigger configurations with DynamoDB, Kinesis and so on.

@DaichiUeura DaichiUeura changed the title Upper camel case is expected on JSON element name (lambda) LambdaInvokerFactory doesn't work with the event sources that AWS Lambda natively accepts Oct 18, 2017
@shorea
Copy link
Contributor

shorea commented Oct 30, 2017

@DaichiUeura I've asked the service team if I can share how the deserialize/serialize these POJOs. Waiting for a response.

@shorea
Copy link
Contributor

shorea commented Oct 30, 2017

So spoke with the service team and I think you have two options.

Option 1:
You can maintain a separate set of POJOs that follows the serialization contract Lambda expects more closely.

Option 2:
You can add a mixin to your Jackson configuration to make the existing Lambda event POJOs serialize correctly according to the interface Lambda expects. Here's an example of one such mixin for the SNSEvent POJO that changes the serialization of fields to match what Lambda expects.

public abstract class SNSEventMixin {

    // needed because Jackson expects "records" instead of "Records"
    @JsonProperty("Records") abstract List<?> getRecords();
    @JsonProperty("Records") abstract void setRecords(List<?> records);

    public abstract class SNSRecordMixin {

        // needed because Jackson expects "getSns" instead of "getSNS"
        @JsonProperty("Sns") abstract Object getSNS();
        @JsonProperty("Sns") abstract void setSns(Object sns);

    }

}

Option 2 seems like the better approach to me.

@shorea shorea closed this as completed Oct 30, 2017
@shorea
Copy link
Contributor

shorea commented Oct 30, 2017

Oh and for Dynamo here's an example of what that Mixin might look like.

public abstract class DynamodbEventMixin {

    // needed because jackson expects "records" instead of "Records"
    @JsonProperty("Records") abstract List<?> getRecords();
    @JsonProperty("Records") abstract void setRecords(List<?> records);

    public abstract class DynamodbStreamRecordMixin {

        // needed because Jackson cannot distinguish between Enum eventName from String eventName
        @JsonProperty("eventName") abstract String getEventName();
        @JsonProperty("eventName") abstract void setEventName(String eventName);
        // needed because Jackson expects "eventSourceArn" instead of "eventSourceARN"
        @JsonProperty("eventSourceARN") abstract String getEventSourceArn();
        @JsonProperty("eventSourceARN") abstract void setEventSourceArn(String eventSourceArn);
    }

    public abstract class StreamRecordMixin {

        // needed because Jackson expects "keys" instead of "Keys"
        @JsonProperty("Keys") abstract Map<String, ?> getKeys();
        @JsonProperty("Keys") abstract void setKeys(Map<String, ?> keys);
        // needed because Jackson expects "sizeBytes" instead of "SizeBytes"
        @JsonProperty("SizeBytes") abstract Long getSizeBytes();
        @JsonProperty("SizeBytes") abstract void setSizeBytes(Long sizeBytes);
        // needed because Jackson expects "sequenceNumber" instead of "SequenceNumber"
        @JsonProperty("SequenceNumber") abstract String getSequenceNumber();
        @JsonProperty("SequenceNumber") abstract void setSequenceNumber(String sequenceNumber);
        // needed because Jackson expects "streamViewType" instead of "StreamViewType"
        @JsonProperty("StreamViewType") abstract String getStreamViewType();
        @JsonProperty("StreamViewType") abstract void setStreamViewType(String streamViewType);
        // needed because Jackson expects "newImage" instead of "NewImage"
        @JsonProperty("NewImage") abstract Map<String, ?> getNewImage();
        @JsonProperty("NewImage") abstract void setNewImage(Map<String, ?> newImage);
        // needed because Jackson expects "oldImage" instead of "OldImage"
        @JsonProperty("OldImage") abstract Map<String, ?> getOldImage();
        @JsonProperty("OldImage") abstract void setOldImage(Map<String, ?> oldImage);
        // needed because Jackson expects "approximateCreationDateTime" instead of "ApproximateCreationDateTime"
        @JsonProperty("ApproximateCreationDateTime") abstract Date getApproximateCreationDateTime();
        @JsonProperty("ApproximateCreationDateTime") abstract void setApproximateCreationDateTime(Date approximateCreationDateTime);

    }

    public abstract class AttributeValueMixin {

        // needed because Jackson expects "s" instead of "S"
        @JsonProperty("S") abstract String getS();
        @JsonProperty("S") abstract void setS(String s);
        // needed because Jackson expects "n" instead of "N"
        @JsonProperty("N") abstract String getN();
        @JsonProperty("N") abstract void setN(String n);
        // needed because Jackson expects "b" instead of "B"
        @JsonProperty("B") abstract ByteBuffer getB();
        @JsonProperty("B") abstract void setB(ByteBuffer b);
        // needed because Jackson expects "null" instead of "NULL"
        @JsonProperty("NULL") abstract Boolean isNULL();
        @JsonProperty("NULL") abstract void setNULL(Boolean nU);
        // needed because Jackson expects "bool" instead of "BOOL"
        @JsonProperty("BOOL") abstract Boolean getBOOL();
        @JsonProperty("BOOL") abstract void setBOOL(Boolean bO);
        // needed because Jackson expects "ss" instead of "SS"
        @JsonProperty("SS") abstract List<String> getSS();
        @JsonProperty("SS") abstract void setSS(List<String> sS);
        // needed because Jackson expects "ns" instead of "NS"
        @JsonProperty("NS") abstract List<String> getNS();
        @JsonProperty("NS") abstract void setNS(List<String> nS);
        // needed because Jackson expects "bs" instead of "BS"
        @JsonProperty("BS") abstract List<String> getBS();
        @JsonProperty("BS") abstract void setBS(List<String> bS);
        // needed because Jackson expects "m" instead of "M"
        @JsonProperty("M") abstract Map<String, ?> getM();
        @JsonProperty("M") abstract void setM(Map<String, ?> val);
        // needed because Jackson expects "l" instead of "L"
        @JsonProperty("L") abstract List<?> getL();
        @JsonProperty("L") abstract void setL(List<?> val);

    }
}

https://github.com/FasterXML/jackson-docs/wiki/JacksonMixInAnnotations

@srchase srchase added guidance Question that needs advice or information. and removed Question labels Jan 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
guidance Question that needs advice or information.
Projects
None yet
Development

No branches or pull requests

4 participants