Skip to content
This repository has been archived by the owner on May 16, 2023. It is now read-only.

SQS / SNS nested object parsing #103

Merged
merged 2 commits into from
Jun 21, 2022

Conversation

sdd
Copy link
Contributor

@sdd sdd commented Jun 20, 2022

Resolves #102.

Adds a set of alternate structs that you can use if you are working with nested objects inside your SQS and/or SNS messages, instead of plain string messages.

For example, a common use case in AWS is to have a Lambda that subscribes to an SQS Queue that is in turn subscribed to an SNS topic. If you are posting serialized JSON objects to that SNS topic, and intending to deserialize them inside a Lambda, prior to this change you would need to manually deserialize the SNS message from within the body of an SqsMessage, and then do the same to get a deserialized instance of a custom struct out of the message field of that SnsMessage.

So whereas before you might need to do this:

#[derive(Deserialize)]
struct CustomStruct {
   foo: String,
   bar: i32
}

fn lambda_handler(e: LambdaEvent<SqsEvent>) -> Result<(), Error> {
    for sqs_msg in e.payload.records.into_iter() {
        let sns_msg: SnsMessage = serde_json::from_str(&sqs_msg.body)?;
        let obj: CustomStruct = serde_json::from_str(&sns_msg.message)?;
        msg_handler(obj);
    }

    Ok(())
}

fn message_handler(msg: CustomStruct) -> () {
    // ... 
}

This PR allows you to do this much more cleanly:

#[derive(Deserialize)]
struct CustomStruct {
   foo: String,
   bar: i32
}

fn lambda_handler(e: LambdaEvent<SqsEventObj<SnsMessageObj<SolveResponse>>>) -> () {
    for sqs_msg in e.payload.records.into_iter() {
        msg_handler(sqs_msg.body.message);
    }
}

fn message_handler(msg: CustomStruct) -> () {
    // ... 
}

@sdd sdd force-pushed the sqs-and-sns-nested-objects branch from a1295f4 to 0706f53 Compare June 20, 2022 19:05
@sdd sdd force-pushed the sqs-and-sns-nested-objects branch from 0706f53 to 5ccc891 Compare June 20, 2022 19:12
Copy link
Owner

@calavera calavera left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a great addition, thanks for putting it together! I have a small request so we don't make compilation time higher when sqs and sns events are not compiled.

@sdd
Copy link
Contributor Author

sdd commented Jun 21, 2022

You're welcome! I've made serde_with optional now, and only pulled in by the sns and sqs features 👍🏼

@calavera calavera merged commit 8b30be5 into calavera:master Jun 21, 2022
@CumpsD
Copy link
Contributor

CumpsD commented Sep 29, 2022

Is it possible these should be optional? https://github.com/LegNeato/aws-lambda-events/blob/master/aws_lambda_events/src/sns/mod.rs#L136-L137

In my SNS message (being posted on SQS) I dont have these fields?

{
    "Records": [
        {
            "attributes": {
                "AWSTraceHeader": "Root=1-633375e5-6992f4381edc004f3f3beed3;Parent=7428100f69529302;Sampled=0",
                "ApproximateFirstReceiveTimestamp": "1664316902967",
                "ApproximateReceiveCount": "1",
                "MessageDeduplicationId": "9",
                "MessageGroupId": "Geert",
                "SenderId": "xxx",
                "SentTimestamp": "1664316902967",
                "SequenceNumber": "18872809200869103616"
            },
            "awsRegion": "eu-central-1",
            "body": "{\n  \"Type\" : \"Notification\",\n  \"MessageId\" : \"f5458878-95d1-5aad-82d9-d3d5edf775d9\",\n  \"SequenceNumber\" : \"10000000000000027000\",\n  \"TopicArn\" : \"arn:aws:sns:eu-central-1:xxx:dev-events.fifo\",\n  \"Message\" : \"{\\\"CustomerDepositedMoney\\\":{\\\"amount\\\":1000.0,\\\"balance\\\":9000.0}}\",\n  \"Timestamp\" : \"2022-09-27T22:15:02.834Z\",\n  \"UnsubscribeURL\" : \"https://sns.eu-central-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:eu-central-1:xxx:dev-events.fifo:aab561d9-73b3-4cdf-b527-9cbfa357eb47\"\n}",
            "eventSource": "aws:sqs",
            "eventSourceARN": "arn:aws:sqs:eu-central-1:xxx:dev-sqs1.fifo",
            "md5OfBody": "4fcc9a1791061d7fc91022734f84850f",
            "messageAttributes": {},
            "messageId": "bd9feb74-23c6-40b7-8b27-8af29b4c140a",
            "receiptHandle": "AQEB8m4uUDMmdKPi0AGmlyV30Eln411zJAi7GTQ0FZdkYOBzDlTwHUdEmXRBqWEa0Re3SwaOvhYx/4kj1DR5mLtJM7adYFj+ovAnUBHljr0gfaQe3WHY0l0vM9RVb/vqUI9weHH5fMCgqYXNanMjoA4KyplNewFvJo/KwxY3cF7LZIpaEZdFREJN5uU1h5u2tQKnCKvMValbToW6JU2LHuArXlo8y9VuS6mIHQVUCDYHn+QCze1JoEWC7EUuiPPaFbT2GjTmCuVr85DpYPH8WyDXvpoaujUWkWoj4rgtVp3AdCA="
        }
    ]
}

Which means my SNS message looks like:

{
  "Type" : "Notification",
  "MessageId" : "f5458878-95d1-5aad-82d9-d3d5edf775d9",
  "SequenceNumber" : "10000000000000027000",
  "TopicArn" : "arn:aws:sns:eu-central-1:xxx:dev-events.fifo",
  "Message" : "{\"CustomerDepositedMoney\":{\"amount\":1000.0,\"balance\":9000.0}}",
  "Timestamp" : "2022-09-27T22:15:02.834Z",
  "UnsubscribeURL" : "https://sns.eu-central-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:eu-central-1:xxx:dev-events.fifo:aab561d9-73b3-4cdf-b527-9cbfa357eb47"
}

@CumpsD
Copy link
Contributor

CumpsD commented Sep 29, 2022

Made a PR for it: #115

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Nested events and embedded deserialisable objects
3 participants