Demo code for the meetup presentation on 14.06.2023:
The meetup was live streamed on LinkedIn and YouTube.
Please find the recording of the talk on YouTube.
The code is based on clemenspeters/ARIMA.
Use
terraform apply
to setup the required infrastructure in your AWS account.
aws lambda invoke --function-name publishLinkToSNS lambda_output_publisher.txt
aws s3 sync s3://preprocessed-serverless ./results_preprocessed_s3
Use
terraform destroy
to remove all created infrastructure from your AWS account.
terraform import aws_s3_bucket.preprocessed_serverless preprocessed-serverless
terraform import aws_sns_topic.preprocessor arn:aws:sns:eu-central-1:710731193510:preprocessor
terraform import aws_ecr_repository.preprocessor preprocessor
terraform import aws_ecr_repository_policy.lambdaPolicy preprocessor
terraform import aws_lambda_function.preprocess arn:aws:lambda:eu-central-1:710731193510:function:preprocess
terraform import aws_sns_topic_subscription.snsPreprocessor arn:aws:sns:eu-central-1:710731193510:preprocessor:79976f18-6a21-470b-8478-cc3976a90788
terraform import aws_lambda_function.publisher arn:aws:lambda:eu-central-1:710731193510:function:publishLinkToSNS
terraform import aws_iam_role.publish_to_sns publishToSNS
terraform import aws_iam_role.preprocessLambdaRole preprocessLambdaRole
-
Package your code
(cd ./lambda_functions/publishLinkToSNS && zip -r ../../publishLinkToSNS.zip lambda_function.py)
-
Upload the code package to an S3 bucket
aws s3 cp publishLinkToSNS.zip s3://preprocessed-serverless/publishLinkToSNS.zip
-
Update the Lambda function with the new code
aws lambda update-function-code --function-name publishLinkToSNS --s3-bucket preprocessed-serverless --s3-key publishLinkToSNS.zip
cd local
. ./download.sh
python3 invoke_processor.py
rm -rf *.parquet
terraform plan
terraform apply
date && aws lambda invoke --function-name publishLinkToSNS lambda_output_publisher.txt
aws s3 sync s3://preprocessed-serverless ./results_preprocessed_s3
terraform destroy
terraform destroy
rm -rf results_preprocessed_s3