Mule 4 app created to scan items from Amazon DynamoDB and insert them in Salesforce Data Cloud.
- Main flow can be found in dynamodb-to-datacloud.xml
- Global configurations can be found in global.xml
- Secured/encrypted properties can be found in secure.config.yaml
- Regular properties can be found in config.yaml
- DataWeave transformation can be found in dynamodb-response.dwl
- Example input payload for the DataWeave script can be found in payload.json
- The encryption key property is called
encryption.key
If you need additional resources to do your configs, check out the following links:
MuleSoft
- How to add JVM/Command-line arguments to the Mule 4 Runtime in Anypoint Code Builder (ACB)
- How to secure properties before deployment in Anypoint Studio
- Quick guide to secure/encrypt your properties in MuleSoft
Salesforce
AWS
- Managing access keys for IAM users
- Getting started with DynamoDB
- Getting started with the DataWeave extension for Visual Studio Code
Besides making sure you have all the configuration ready in AWS/Salesforce, this is how you have to set up your Mule project to run locally.
- Download or clone this repo to your local computer
- Open it in Studio or ACB (preferably ACB)
- Add your values to the config.yaml file
cdp.api.name
- the Source API Name from your Ingestion API in Salesforcecdp.object.name
- the Object Name from your Ingestion API / Data Stream in Salesforcedynamodb.table.name
- the name of the table to scan items from in DynamoDB
- Add your encrypted values to the secure.config.yaml file
cdp.consumer.key
- the Consumer Key from your Connected App in Salesforcecdp.consumer.secret
- the Consumer Secret from your Connected App in Salesforcesalesforce.username
- the username to log in to your Salesforce accountsalesforce.password
- the password to log in to your Salesforce accountdynamodb.accessKey
- the Access Key for your AWS accountdynamodb.secretKey
- the Secret Key for your AWS account
- Modify the DataWeave code in dynamodb-response.dwl to match your DynamoDB/Data Cloud output/input
- The code in the file is an example for my specific use case. Use the two functions (
removeDynamodbKeys
&flattenObject
) to transform the data, but make sure you map the keys/values to your specific structure - If you're using ACB, you can use the DataWeave extension for VS Code to see the preview of your script before deploying
- The code in the file is an example for my specific use case. Use the two functions (
- Make sure to pass the
encryption.key
property to the runtime (see resources for more info) - Run the application locally
- Send a request to
localhost:8081/sync
to trigger the flow - You should receive a 200-OK response
- Wait 2-5 minutes for the insertion to be added to Data Cloud
If you're experiencing issues, please make sure your credentials are correct and your Salesforce/AWS settings have been properly set.
You can also raise an issue to this repo in case I missed something in the code.
This is intended to be a simple POC. This app only performs the scan from the given AWS table and does an insert (streaming) to Data Cloud every time you send a request to the /sync
path.
This app does not perform deletions or queries in Data Cloud. Check out this GitHub repo for delete/query operations.
Since the Streaming - Insert operation is being used for Data Cloud, there is a max of 200 records set by Data Cloud. You can implement the Size-Based Aggregator to the app if you intend to use more than 200 records or switch the operation to Batch instead of Streaming.