diff --git a/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/ArchDiagram.png b/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/ArchDiagram.png new file mode 100644 index 000000000..a52e2ae77 Binary files /dev/null and b/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/ArchDiagram.png differ diff --git a/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/README.md b/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/README.md new file mode 100644 index 000000000..7257a3ba7 --- /dev/null +++ b/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/README.md @@ -0,0 +1,108 @@ +# Cross-account cross-region replication for FSx for OpenZFS volumes with AWS Lambda +Amazon EventBridge triggers an AWS Lambda function to replicate FSx for OpenZFS volumes across file systems located in the same account and region, or across different accounts and regions. + +![Architecture diagram](./ArchDiagram.png) + +The [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) (AWS SAM) template deploys an Amazon EventBridge Scheduler to trigger an AWS Lambda function based on a user-defined schedule. This function copies the snapshot of a volume and transfers it to the target FSx system, which can be located in the same or a different AWS account and/or region. + +For FSx for OpenZFS periodic volume replication in same account and same region, please refer to the Serverless Land Pattern + +The template contains a sample Lambda function that creates a snapshot of the source FSx Volume ID. Once the snapshot becomes available, it invokes another Lambda function in the destination AWS account and/or region, which initiates the replication by calling the copy_snapshot_and_update_volume API. This solution also notifies users via an Amazon SNS topic of any errors and snapshot creation details. + +Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example. + +## Requirements + +* [Create two AWS accounts for cross account setup](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have, create them and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources. + +* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configure two profiles with credentials for the individual accounts as below: + + ``` + [default] + [crossaccount] + ``` + + +- [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured +- [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) +- [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) (AWS SAM) installed +- Make sure that you have the ID of the source and destination volumes that you would like to initiate the replication between. For more information on these resources, see [Creating FSx for OpenZFS file systems](https://docs.aws.amazon.com/fsx/latest/OpenZFSGuide/creating-file-systems.html), [Creating a volume](https://docs.aws.amazon.com/fsx/latest/OpenZFSGuide/creating-volumes.html), [Creating a snapshot](https://docs.aws.amazon.com/fsx/latest/OpenZFSGuide/snapshots-openzfs.html#creating-snapshots), and [Using on-demand data replication](https://docs.aws.amazon.com/fsx/latest/OpenZFSGuide/on-demand-replication.html#how-to-use-data-replication). + +## Deployment Instructions + +1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository: + ``` + git clone https://github.com/aws-samples/serverless-patterns + ``` +2. Change directory to the pattern directory: + ``` + cd eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication + ``` +3. From the command line, use the AWS SAM command listed below to deploy the AWS resources in the destination AWS account as specified in the destination-template.yaml file. Note that an AWS CLI profile named crossaccount must be configured with AWS credentials for the destination/target AWS account. + ``` + sam deploy --guided --capabilities CAPABILITY_AUTO_EXPAND CAPABILITY_IAM CAPABILITY_NAMED_IAM -t destination-template.yaml --profile crossaccount + ``` +4. During the prompts: + - Enter a target stack name + - Enter the desired AWS Region + - Enter a TargetVolumeID + - Enter a CopySnapshotAndUpdateVolume - "Options" parameter. Comma (,) separated values + - Enter a CopySnapshotAndUpdateVolume - "CopyStrategy" parameter (Default = INCREMENTAL_COPY) + - Enter source AWS account Id + - Allow SAM CLI to create IAM roles with the required permissions. + - Save arguments to configuration file [Y/n]: N +5. Once the above stack is deployed in the target account/region, use the AWS SAM command listed below to deploy the resources in the source AWS account using the source-template.yaml file. + ``` + sam deploy --guided --capabilities CAPABILITY_AUTO_EXPAND CAPABILITY_IAM CAPABILITY_NAMED_IAM -t source-template.yaml + ``` +6. During the prompts: + - Enter a source stack name + - Enter the desired AWS Region + - Enter a SourceVolumeID + - Enter a CRON schedule for snapshots (Default = [0 0/6 * * ? *] every six hours) + - Enter a value of snapshot Name (Default = fsx_scheduled_snapshot) + - Enter an Email for notifications + - Allow Success Notification (Default = Yes) + - Enter number of days to retain custom-scheduled snapshots (Default = 7 days) + - Enter target AWS account Id + - Enter target region + - Enter target stack name used previously + - Save arguments to configuration file [Y/n]: N +7. Note the outputs from the previous SAM deploy commands. These contain the resource names and/or ARNs which will be used for later review. + +## How it works + +This pattern sets up the following resources: + +- An Amazon EventBridge Scheduler that triggers a Lambda function based on the schedule defined by you to create snapshots of the provided FSx Source Volume ID. +- A sample [Lambda](https://docs.aws.amazon.com/lambda/latest/dg/welcome.html) function that creates snapshots of the source FSx Volume ID and invokes another Lambda function in the destination AWS account and/or region, which will initiate the replication by calling the [CopySnapshotAndUpdateVolume](https://docs.aws.amazon.com/fsx/latest/APIReference/API_CopySnapshotAndUpdateVolume.html) API. +- The function also deletes the older snapshots in the source and target AWS account/region based on the configured retention period. +- An SNS topic that sends notifications for any success or failure events while creating or replicating snapshots. + +## Testing +1. Based on the provided schedule, monitor the CloudWatch logs and the FSx snapshots that are created. +2. The Lambda function will send various success and failure notifications to the configured email address via an SNS topic. + +## Cleanup +1. Change directory to the pattern directory: + ``` + cd serverless-patterns/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication + ``` +2. Delete resources from the source account. + ``` + sam delete --stack-name + ``` + +3. Delete resources from the destination/target account. + ``` + sam delete --stack-name --profile crossaccount --config-env crossacct + `````` +3. During the prompts: + * Enter all details as per requirement. + +--- + +Copyright 2025 Amazon.com, Inc. or its affiliates. All Rights Reserved. + +SPDX-License-Identifier: MIT-0 + diff --git a/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/destination-template.yaml b/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/destination-template.yaml new file mode 100644 index 000000000..9568285e9 --- /dev/null +++ b/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/destination-template.yaml @@ -0,0 +1,142 @@ +AWSTemplateFormatVersion: "2010-09-09" + +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy of +# this software and associated documentation files (the "Software"), to deal in +# the Software without restriction, including without limitation the rights to +# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of +# the Software, and to permit persons to whom the Software is furnished to do so. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS +# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR +# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER +# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +Transform: AWS::Serverless-2016-10-31 +Description: > + Configure periodic replication schedule for your Amazon FSx for OpenZFS file system volumes. + +Parameters: + TargetVolumeID: + Description: Amazon FSx for OpenZFS Target Volume ID + Type: String + AllowedPattern: "^fsvol-[A-Za-z0-9]+" + + # CopySnapshotAndUpdateVolume - "Options" parameter. + # Comma (,) separated values such as "DELETE_INTERMEDIATE_SNAPSHOTS,DELETE_INTERMEDIATE_DATA,DELETE_CLONED_VOLUMES" + Options: + Description: Options parameter value for the CopySnapshotAndUpdateVolume API + Type: String + + # CopySnapshotAndUpdateVolume - "CopyStrategy" parameter. + # example = INCREMENTAL_COPY or FULL_COPY + CopyStrategy: + Description: CopyStrategy parameter value for the CopySnapshotAndUpdateVolume API + Type: String + Default: INCREMENTAL_COPY + + SourceAWSAcctId: + Description: FSx Source AWS Account ID + Type: String + AllowedPattern: ^\d{12} + +Resources: + # + # IAM role going to be assumed by Source Lambda function to invoke target Lambda function. + # + CrossAcctIAMRole: + Type: AWS::IAM::Role + Properties: + RoleName: !Sub CrossAcctIAMRole-${AWS::StackName} + AssumeRolePolicyDocument: + Version: "2012-10-17" + Statement: + - Effect: Allow + Principal: + AWS: !Sub "arn:aws:iam::${SourceAWSAcctId}:root" + Action: sts:AssumeRole + Path: / + Policies: + - PolicyName: !Sub CrossAcctIAMRole-Policy-${AWS::StackName} + PolicyDocument: + Version: "2012-10-17" + Statement: + - Effect: Allow + Action: + - lambda:InvokeFunction + Resource: !GetAtt CopySnapshotAndUpdateVolumeLambda.Arn + + # + # Lambda Execution Role + # + CopySnapshotAndUpdateVolumeLambdaRole: + Type: AWS::IAM::Role + Properties: + RoleName: !Sub CopySnapshotAndUpdateVolumeLambda-Role-${AWS::StackName} + AssumeRolePolicyDocument: + Version: "2012-10-17" + Statement: + - Effect: Allow + Principal: + Service: lambda.amazonaws.com + Action: sts:AssumeRole + Path: / + Policies: + - PolicyName: !Sub CopySnapshotAndUpdateVolumeLambda-Policy-${AWS::StackName} + PolicyDocument: + Version: "2012-10-17" + Statement: + - Effect: Allow + Action: + - logs:CreateLogGroup + - logs:CreateLogStream + - logs:PutLogEvents + Resource: !Sub "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/*" + - Effect: Allow + Action: + - fsx:CreateSnapshot + - fsx:DescribeSnapshots + - fsx:DescribeVolumes + - fsx:DeleteSnapshot + - fsx:TagResource + - fsx:ListTagsForResource + - fsx:CopySnapshotAndUpdateVolume + Resource: + - !Sub "arn:aws:fsx:${AWS::Region}:${AWS::AccountId}:*" + - !Sub "arn:aws:fsx:*:${SourceAWSAcctId}:*" + # + # Lambda function that will make CopySnapshotAndUpdateVolume API call. + # + CopySnapshotAndUpdateVolumeLambda: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Sub CopySnapshotAndUpdateVolumeLambda-${AWS::StackName} + Description: Lambda function that will make CopySnapshotAndUpdateVolume API call. + Environment: + Variables: + DEST_VOLUME_ID: !Ref TargetVolumeID + OPTIONS: !Ref Options + COPY_STRATEGY: !Ref CopyStrategy + Handler: CopySnapshotAndUpdateVolume.lambda_handler + Role: !GetAtt CopySnapshotAndUpdateVolumeLambdaRole.Arn + CodeUri: src/CopySnapshotAndUpdateVolume.py + Runtime: python3.13 + Timeout: 600 + +# +# Stack output section +# +Outputs: + CrossAcctIAMRole: + Description: Cross account IAM role going to be assumed by Source Lambda function to invoke target Lambda function. + Value: !GetAtt CrossAcctIAMRole.Arn + LambdaFunction: + Description: Target account Lambda Function that will make CopySnapshotAndUpdateVolume API call + Value: !GetAtt CopySnapshotAndUpdateVolumeLambda.Arn + LambdaExecutionRole: + Description: IAM role used by target Lambda function + Value: !GetAtt CopySnapshotAndUpdateVolumeLambdaRole.Arn + diff --git a/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication.json b/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication.json new file mode 100644 index 000000000..c82b3854c --- /dev/null +++ b/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication.json @@ -0,0 +1,79 @@ +{ + "title": "Replicate FSx-OpenZFS volumes across file systems", + "description": "Periodic Amazon FSx for OpenZFS volume replication across AWS Regions and accounts using Amazon EventBridge Scheduler and AWS Lambda", + "language": "Python", + "level": "200", + "framework": "SAM", + "introBox": { + "headline": "How it works", + "text": [ + "The AWS SAM template deploys an Amazon EventBridge Scheduler to trigger an AWS Lambda function based on a user-defined schedule. This function copies snapshots of the volume and replicates them to the target FSx system available in a different AWS account and/or region. The template contains a sample Lambda function that creates a snapshot of the source FSx VolumeID. Once the snapshot becomes available, it invokes another Lambda function in the destination AWS account or region, which initiates the replication by calling the copy_snapshot_and_update_volume API. This solution also notifies users using an SNS topic for any errors and snapshot creation details.", + "This pattern sets up the following resources:", + "An Amazon EventBridge Scheduler that triggers a Lambda function based on the schedule defined by the customer to take snapshots of the provided FSx Source VolumeID.", + "An SNS topic that sends notifications for any failures while creating snapshots.", + "The function also deletes older snapshots.", + "Sample Lambda functions that create snapshots of the source FSx VolumeID and replicate them by invoking another Lambda function, which calls the copy_snapshot_and_update_volume API for the target VolumeID in the destination AWS account or region." + ] + }, + "gitHub": { + "template": { + "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication", + "templateURL": "serverless-patterns/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication", + "projectFolder": "eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication", + "templateFile": "source-template.yaml" + } + }, + "deploy": { + "text": [ + "sam deploy --guided --capabilities CAPABILITY_AUTO_EXPAND CAPABILITY_IAM CAPABILITY_NAMED_IAM -t source-template.yaml" + ] + }, + "testing": { + "text": [ + "See the GitHub repo for detailed testing instructions." + ] + }, + "cleanup": { + "text": [ + "Delete the stack: sam delete --stack-name ." + ] + }, + "authors": [ + { + "name": "Anup Rajpara", + "image": "https://drive.google.com/file/d/1MqpPNLCqbU4kvvtTspNXZBqD99aVIJI9/view?usp=sharing", + "bio": "Anup is passionate about serverless & event-driven architectures.", + "linkedin": "anup-rajpara-developer/" + } + ], + "patternArch": { + "icon1": { + "x": 20, + "y": 50, + "service": "eventbridge-scheduler", + "label": "EventBridge Scheduler" + }, + "icon2": { + "x": 50, + "y": 50, + "service": "lambda", + "label": "AWS Lambda" + }, + "icon3": { + "x": 80, + "y": 50, + "service": "fsx", + "label": "FSx for OpenZFS" + }, + "line1": { + "from": "icon1", + "to": "icon2", + "label": "" + }, + "line2": { + "from": "icon2", + "to": "icon3", + "label": "" + } + } +} diff --git a/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/example-pattern.json b/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/example-pattern.json new file mode 100644 index 000000000..497e2d527 --- /dev/null +++ b/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/example-pattern.json @@ -0,0 +1,49 @@ +{ + "title": "Replicate FSx-OpenZFS volumes across file systems", + "description": "Periodic Amazon FSx for OpenZFS volume replication across AWS Regions and accounts using Amazon EventBridge Scheduler and AWS Lambda", + "language": "Python", + "level": "200", + "framework": "SAM", + "introBox": { + "headline": "How it works", + "text": [ + "The AWS SAM template deploys an Amazon EventBridge Scheduler to trigger an AWS Lambda function based on a user-defined schedule. This function copies snapshots of the volume and replicates them to the target FSx system available in a different AWS account and/or region. The template contains a sample Lambda function that creates a snapshot of the source FSx VolumeID. Once the snapshot becomes available, it invokes another Lambda function in the destination AWS account or region, which initiates the replication by calling the copy_snapshot_and_update_volume API. This solution also notifies users using an SNS topic for any errors and snapshot creation details.", + "This pattern sets up the following resources:", + "An Amazon EventBridge Scheduler that triggers a Lambda function based on the schedule defined by the customer to take snapshots of the provided FSx Source VolumeID.", + "An SNS topic that sends notifications for any failures while creating snapshots.", + "The function also deletes older snapshots.", + "Sample Lambda functions that create snapshots of the source FSx VolumeID and replicate them by invoking another Lambda function, which calls the copy_snapshot_and_update_volume API for the target VolumeID in the destination AWS account or region." + ] + }, + "gitHub": { + "template": { + "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication", + "templateURL": "serverless-patterns/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication", + "projectFolder": "eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication", + "templateFile": "source-template.yaml" + } + }, + "deploy": { + "text": [ + "sam deploy --guided --capabilities CAPABILITY_AUTO_EXPAND CAPABILITY_IAM CAPABILITY_NAMED_IAM -t source-template.yaml" + ] + }, + "testing": { + "text": [ + "See the GitHub repo for detailed testing instructions." + ] + }, + "cleanup": { + "text": [ + "Delete the stack: sam delete --stack-name ." + ] + }, + "authors": [ + { + "name": "Anup Rajpara", + "image": "https://drive.google.com/file/d/1MqpPNLCqbU4kvvtTspNXZBqD99aVIJI9/view?usp=sharing", + "bio": "Anup is passionate about serverless & event-driven architectures.", + "linkedin": "anup-rajpara-developer/" + } + ] +} diff --git a/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/source-template.yaml b/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/source-template.yaml new file mode 100644 index 000000000..30f4dcb2d --- /dev/null +++ b/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/source-template.yaml @@ -0,0 +1,221 @@ +AWSTemplateFormatVersion: "2010-09-09" + +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy of +# this software and associated documentation files (the "Software"), to deal in +# the Software without restriction, including without limitation the rights to +# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of +# the Software, and to permit persons to whom the Software is furnished to do so. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS +# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR +# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER +# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +Transform: AWS::Serverless-2016-10-31 +Description: > + Configure periodic replication schedule for your Amazon FSx for OpenZFS file system volumes. + +Parameters: + SourceVolumeID: + Description: Amazon FSx for OpenZFS Source Volume ID + Type: String + AllowedPattern: "^fsvol-[A-Za-z0-9]+" + + # Schedule for creating snapshot and calling CopySnapshotAndUpdateVolume API + CronSchedule: + Description: CRON schedule for snapshots (default every 6 hours) + Type: String + Default: "0 0/6 * * ? *" + + # Value of snapshot Name (shows in Name column in snapshots list in console) + SnapshotName: + Description: The name of snapshots starts with (shown in FSx console) + Type: String + Default: "fsx_scheduled_snapshot" + AllowedPattern: "^[a-zA-Z0-9_:.-]{1,179}$" + + # Email for notifications + Email: + Description: Email for CopySnapshotAndUpdateVolume notifications + Type: String + + # If customer wants notification for successful snapshots + SuccessNotification: + Description: Do you want to be notified for successful replication initiation? For failures, you will always be notified + Type: String + AllowedValues: + - "Yes" + - "No" + Default: "Yes" + + # Number of days of snapshots you want to retain + RetainDays: + Description: Number of days to retain custom-scheduled snapshots + Type: Number + Default: 7 + + # Destination AWS Account ID + DestinationAWSAcctId: + Description: FSx Destination AWS Account ID + Type: String + AllowedPattern: ^\d{12} + + # Destination region + DestinationRegion: + Description: Destination region + Type: String + + # target account/region stack name + TargetStackName: + Description: Name of the Target region/account stack + Type: String + +Resources: + # + # SNS topic to notify + # + SNSTopic: + Type: AWS::SNS::Topic + Properties: + DisplayName: !Sub ${AWS::StackName}-notification + TopicName: !Sub ${AWS::StackName}-notification + Subscription: + - Endpoint: !Ref Email + Protocol: "email" + + # + # Lambda Execution Role + # + SnapshotLambdaRole: + Type: AWS::IAM::Role + Properties: + RoleName: !Sub SnapshotLambdaRole-${AWS::StackName} + AssumeRolePolicyDocument: + Version: "2012-10-17" + Statement: + - Effect: Allow + Principal: + Service: lambda.amazonaws.com + Action: sts:AssumeRole + Path: / + Policies: + - PolicyName: !Sub SnapshotLambda-Policy-${AWS::StackName} + PolicyDocument: + Version: "2012-10-17" + Statement: + - Effect: Allow + Action: + - logs:CreateLogGroup + - logs:CreateLogStream + - logs:PutLogEvents + Resource: !Sub "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/*" + - Effect: Allow + Action: + - fsx:CreateSnapshot + - fsx:DescribeSnapshots + - fsx:DescribeVolumes + - fsx:DeleteSnapshot + - fsx:TagResource + - fsx:ListTagsForResource + Resource: !Sub "arn:aws:fsx:${AWS::Region}:${AWS::AccountId}:*" + - Effect: Allow + Action: + - lambda:InvokeFunction + Resource: !Sub "arn:aws:lambda:${DestinationRegion}:${DestinationAWSAcctId}:function:CopySnapshotAndUpdateVolumeLambda-${TargetStackName}" + - Effect: Allow + Action: + - sts:AssumeRole + Resource: !Sub "arn:aws:iam::${DestinationAWSAcctId}:role/CrossAcctIAMRole-${TargetStackName}" + - Effect: Allow + Action: + - sns:Publish + Resource: !GetAtt SNSTopic.TopicArn + + # + # Lambda function + # + SnapshotLambda: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Sub ${AWS::StackName}-Lambda + Environment: + Variables: + SRC_VOLUME_ID: !Ref SourceVolumeID + SNS_TOPIC_ARN: !GetAtt SNSTopic.TopicArn + SUCCESS_NOTIFICATION: !Ref SuccessNotification + SNAPSHOT_NAME: !Ref SnapshotName + SNAPSHOT_TAG_VALUE: !Sub ${AWS::StackName}_${SourceVolumeID} + SNAPSHOT_RETAIN_DAYS: !Ref RetainDays + DEST_LAMBDA_ARN: !Sub "arn:aws:lambda:${DestinationRegion}:${DestinationAWSAcctId}:function:CopySnapshotAndUpdateVolumeLambda-${TargetStackName}" + DEST_IAM_ROLE: !Sub "arn:aws:iam::${DestinationAWSAcctId}:role/CrossAcctIAMRole-${TargetStackName}" + DEST_LAMBDA_REGION: !Ref DestinationRegion + Handler: PeriodicReplication.lambda_handler + Role: !GetAtt SnapshotLambdaRole.Arn + CodeUri: src/PeriodicReplication.py + Runtime: python3.13 + Timeout: 900 + + # + # EventBridge Scheduler Role + # + FSxEventBridgeSchedulerRole: + Type: AWS::IAM::Role + Properties: + AssumeRolePolicyDocument: + Version: "2012-10-17" + Statement: + - Effect: Allow + Principal: + Service: + - scheduler.amazonaws.com + Action: + - sts:AssumeRole + Policies: + - PolicyName: FSxEventBridgeSchedulerRolePolicy + PolicyDocument: + Version: "2012-10-17" + Statement: + - Effect: Allow + Action: + - "lambda:InvokeFunction" + Resource: + - !GetAtt SnapshotLambda.Arn + + # + # EventBridge Scheduler to trigger a Lambda function + # + FSxEventBridgeScheduler: + Type: AWS::Scheduler::Schedule + Properties: + Name: !Sub ${AWS::StackName}-Scheduler + ScheduleExpression: !Sub "cron(${CronSchedule})" + FlexibleTimeWindow: + Mode: "OFF" + Target: + Arn: !GetAtt SnapshotLambda.Arn + RoleArn: !GetAtt FSxEventBridgeSchedulerRole.Arn + +# +# Stack output section +# +Outputs: + SNSTopic: + Description: SNS Topic + Value: !Ref "SNSTopic" + EventBridgeSchedulerRole: + Description: EventBridge Scheduler Role + Value: !GetAtt FSxEventBridgeSchedulerRole.Arn + EventBridgeScheduler: + Description: EventBridge Scheduler + Value: !GetAtt FSxEventBridgeScheduler.Arn + LambdaExecutionRole: + Description: Lambda Execution Role + Value: !GetAtt SnapshotLambdaRole.Arn + LambdaFunction: + Description: Lambda Function + Value: !GetAtt SnapshotLambda.Arn + diff --git a/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/src/CopySnapshotAndUpdateVolume.py b/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/src/CopySnapshotAndUpdateVolume.py new file mode 100644 index 000000000..8a9844d5b --- /dev/null +++ b/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/src/CopySnapshotAndUpdateVolume.py @@ -0,0 +1,114 @@ +import json +import os +import time +import datetime +import boto3 +import botocore +import logging +logger = logging.getLogger() +logger.setLevel("INFO") + +logger.info("boto3 version: " + boto3.__version__) +logger.info("botocore version: " + botocore.__version__) + +session = boto3.session.Session() +fsx_client = session.client(service_name='fsx') +volId = os.environ.get("DEST_VOLUME_ID") + +def deleteSnapshotIfOlderThanRetention(snapshot, retain_days): + snapshot_id = snapshot['SnapshotId'] + created = snapshot['CreationTime'] + created_date = created.date() + now_date = datetime.datetime.now().date() + delta = now_date - created_date + + try: + logger.info("Examining OpenZFS volume snapshot " + snapshot['Name'] + " with Sanpshot ID = " + snapshot_id) + if delta.days > retain_days: + fsx_client.delete_snapshot(SnapshotId=snapshot_id) + logger.info("Deleted FSx for OpenZFS volume snapshot " + snapshot['Name'] + " with Sanpshot ID = " + snapshot_id) + else: + logger.info("Skipping (retaining) FSx for OpenZFS volume " + snapshot['Name'] + " with Sanpshot ID = " + snapshot_id) + except Exception as e: + logger.info("The error is: " + str(e)) + +def deleteSnapshots(retain_days, snapshot_name): + logger.info ("deleting snapshots") + + # query the FSx API for existing snapshots + logger.info ("Getting snapshots for volume id = " + volId) + next_token = None + while True: + # Prepare the base request parameters + params = { + 'Filters':[{'Name': 'volume-id', 'Values': [volId]}], + 'MaxResults': 2 # 20 snapshots per API call + } + + # Add NextToken if it exists + if next_token: + params['NextToken'] = next_token + + # Make the API call + response = fsx_client.describe_snapshots(**params) + + # Process snapshots in current response + snapshots = response.get('Snapshots', []) + logger.info(snapshots) + + # loop thru the results, checking the snapshot date-time and call Fsx API to remove those older than x hours/days + logger.info("Starting purge of snapshots older than " + str(retain_days) + " days for volume " + volId) + + for snapshot in snapshots: + if snapshot['Name'].startswith(snapshot_name): + deleteSnapshotIfOlderThanRetention(snapshot, retain_days) + + # Check if there are more results + next_token = response.get('NextToken') + if not next_token: + break + +def lambda_handler(event, context): + logger.info(event) + src_snapshot_arn = event["src_snapshot_ResourceARN"] + retain_days = event["snapshot_retain_days"] + snapshot_name = event["snapshot_name"] + + rspJson = {} + try: + logger.info ("Starting copy snapshot operation ...") + options_string = os.environ.get("OPTIONS") + options = [] if len(options_string) == 0 else list(options_string.split(", ")) + option_list = [] + for item in options: + option_list.append(item.strip()) + logger.info ("CopySnapshotAndUpdateVolume Options: ") + logger.info (option_list) + + response = fsx_client.copy_snapshot_and_update_volume( + VolumeId=volId, + SourceSnapshotARN=src_snapshot_arn, + Options=option_list, + CopyStrategy=os.environ.get("COPY_STRATEGY") + ) + logger.info (response) + msg = "Started CopySnapshotAndUpdateVolume operation\n\n" + msg += "Destination Volume ID : " + os.environ.get("DEST_VOLUME_ID") + "\n" + msg += "Source Snapshot ARN : " + src_snapshot_arn + logger.info (msg) + rspJson["Message"] = msg + rspJson["Subject"] = 'Success Notification: CopySnapshotAndUpdateVolume' + + except Exception as e: + logger.info("The error is: " + str(e)) + msg = "Error while initiating CopySnapshotAndUpdateVolume operation\n\n" + msg += "Destination Volume ID = " + os.environ.get("DEST_VOLUME_ID") + "\n" + msg += "Source Snapshot ARN = " + src_snapshot_arn + "\n" + str(e) + logger.info (msg) + rspJson["Message"] = msg + rspJson["Subject"] = 'Error Notification: CopySnapshotAndUpdateVolume' + + logger.info (rspJson) + deleteSnapshots(retain_days, snapshot_name) + logger.info ("function completed") + return rspJson \ No newline at end of file diff --git a/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/src/PeriodicReplication.py b/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/src/PeriodicReplication.py new file mode 100644 index 000000000..90f688c5f --- /dev/null +++ b/eventbridge-lambda-fsx-openzfs-cross-account-region-periodic-replication/src/PeriodicReplication.py @@ -0,0 +1,176 @@ +import json +import os +import time +import datetime +import boto3 +import botocore +import logging +logger = logging.getLogger() +logger.setLevel("INFO") + +logger.info("boto3 version: " + boto3.__version__) +logger.info("botocore version: " + botocore.__version__) + +session = boto3.session.Session() +fsx_client = session.client(service_name='fsx') +sns_client = boto3.client('sns') + +sns_notification = os.environ.get('SUCCESS_NOTIFICATION', "No") == 'Yes' +retain_days = int(os.environ.get('SNAPSHOT_RETAIN_DAYS')) +snapshot_name = os.environ.get('SNAPSHOT_NAME') + +def send_sns_notification(msg, subject): + sns_client.publish( + TopicArn=os.environ.get("SNS_TOPIC_ARN"), + Subject=subject, + Message=msg + ) + +def deleteSnapshotIfOlderThanRetention(snapshot): + snapshot_id = snapshot['SnapshotId'] + created = snapshot['CreationTime'] + created_date = created.date() + now_date = datetime.datetime.now().date() + delta = now_date - created_date + + try: + logger.info("Examining OpenZFS volume snapshot " + snapshot['Name'] + " with Sanpshot ID = " + snapshot_id) + if delta.days > retain_days: + fsx_client.delete_snapshot(SnapshotId=snapshot_id) + logger.info("Deleted FSx for OpenZFS volume snapshot " + snapshot['Name'] + " with Sanpshot ID = " + snapshot_id) + else: + logger.info("Skipping (retaining) FSx for OpenZFS volume " + snapshot['Name'] + " with Sanpshot ID = " + snapshot_id) + except Exception as e: + logger.info("The error is: %s", str(e)) + +def deleteSnapshots(): + logger.info ("deleting snapshots") + volId = os.environ.get("SRC_VOLUME_ID") + + # query the FSx API for existing snapshots + logger.info ("Getting snapshots for volume id = " + volId) + next_token = None + while True: + # Prepare the base request parameters + params = { + 'Filters':[{'Name': 'volume-id', 'Values': [volId]}], + 'MaxResults': 20 # 20 snapshots per API call + } + + # Add NextToken if it exists + if next_token: + params['NextToken'] = next_token + + # Make the API call + response = fsx_client.describe_snapshots(**params) + + # Process snapshots in current response + snapshots = response.get('Snapshots', []) + logger.info(snapshots) + + # loop thru the results, checking the snapshot date-time and call Fsx API to remove those older than x hours/days + logger.info("Starting purge of snapshots older than " + str(retain_days) + " days for volume " + volId) + + for snapshot in snapshots: + if snapshot['Name'].startswith(snapshot_name): + deleteSnapshotIfOlderThanRetention(snapshot) + + # Check if there are more results + next_token = response.get('NextToken') + if not next_token: + break + +def lambda_handler(event, context): + try: + # call the FSx snapshot API + logger.info ("Creating a snapshot for the volume = " + os.environ.get("SRC_VOLUME_ID")) + response = fsx_client.create_snapshot( + # append datetime to ensure snap name is unique + Name=os.environ.get("SNAPSHOT_NAME") + datetime.datetime.utcnow().strftime("_%Y-%m-%d_%H:%M:%S.%f")[:-3], + VolumeId=os.environ.get("SRC_VOLUME_ID"), + Tags=[{'Key': 'CreatedBy','Value': os.environ.get("SNAPSHOT_TAG_VALUE") },] + ) + logger.info(response) + src_snapshot = response["Snapshot"] + if sns_notification: + logger.info ("Sending SNS notification for successful snapshot creation") + msg = "Snapshot Created Successfully\n\n" + msg += "Snapshot ID : " + src_snapshot["SnapshotId"] + "\n" + msg += "ResourceARN : " + src_snapshot["ResourceARN"] + "\n" + msg += "Snapshot Name : " + src_snapshot["Name"] + "\n" + msg += "Snapshot Tags : " + json.dumps(src_snapshot["Tags"]) + "\n" + msg += "Snapshot Lifecycle : " + src_snapshot["Lifecycle"] + send_sns_notification (msg, 'Success Notification: CreateSnapshot') + + except Exception as e: + logger.info("The error is: " + str(e)) + errMessage = "Error while creating a snapshot from the Source VolumeId = " + os.environ.get("SRC_VOLUME_ID") + "\n" + errMessage += "Error = " + str(e) + send_sns_notification (errMessage, 'Error Notification: CreateSnapshot') + deleteSnapshots() + return + + # call the FSx describe snapshot API to confirm created snapshot is in AVAILABLE state + copy_snapshot = False + for i in range(1, 10): + time.sleep(10) + logger.info ("Describe Snapshot - Attempt=" + str(i)) + ret = fsx_client.describe_snapshots(SnapshotIds=[src_snapshot["SnapshotId"]]) + logger.info("Snapshot = " + src_snapshot["SnapshotId"] + " is in " + ret["Snapshots"][0]["Lifecycle"] +" state.") + if ret["Snapshots"][0]["Lifecycle"] == "AVAILABLE": + copy_snapshot = True + break + + if not copy_snapshot: + logger.info ("ERROR - The snapshot does not transition to AVAILABLE state for some reason - Snapshot ID : " + src_snapshot["SnapshotId"]) + msg = "ERROR - The snapshot does not transition to AVAILABLE state for some reason !!\n\n" + msg += "Snapshot ID : " + src_snapshot["SnapshotId"] + "\n" + msg += "Snapshot Name : " + src_snapshot["Name"] + "\n" + send_sns_notification (msg, 'Error Notification: CreateSnapshot') + else: + try: + logger.info ("Assuming role in target ...") + sts_connection = boto3.client('sts') + target_role = sts_connection.assume_role( + RoleArn=os.environ.get("DEST_IAM_ROLE"), + RoleSessionName="target_lambda_role" + ) + + # create a lambda client using the assumed role credentials + lambda_client = boto3.client( + 'lambda', + region_name=os.environ.get("DEST_LAMBDA_REGION"), + aws_access_key_id=target_role['Credentials']['AccessKeyId'], + aws_secret_access_key=target_role['Credentials']['SecretAccessKey'], + aws_session_token=target_role['Credentials']['SessionToken'], + ) + + # prepare payload to invoke the target/destination lambda function + payload = { + "src_snapshot_ResourceARN": src_snapshot["ResourceARN"], + "snapshot_retain_days":retain_days, + "snapshot_name":snapshot_name + } + payload = json.dumps(payload) + + logger.info ("Invoking target lambda function ...") + response = lambda_client.invoke( + FunctionName=os.environ.get("DEST_LAMBDA_ARN"), + InvocationType='RequestResponse', + Payload=payload + ) + + logger.info ("Received response from a target lambda function ...") + lambda_rsp = json.load(response["Payload"]) + logger.info(lambda_rsp) + send_sns_notification (lambda_rsp["Message"], lambda_rsp["Subject"]) + + except Exception as e: + logger.info("The error is: " + str(e)) + errMessage = "Error while invoking target lambda function\n\n" + errMessage += "Source Snapshot ARN = " + src_snapshot["ResourceARN"] + "\n" + str(e) + logger.info(errMessage) + send_sns_notification (errMessage, 'Error Notification: Invoke Lambda Function') + + deleteSnapshots() + logger.info ("function completed")