Centralize all logs in a separate AWS account.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
LICENSE
README.md
cloudwatch_logs_sink.yml
cloudwatch_logs_source.yml
es.yml
firehose.yml

README.md

log-pipeline

Centralize all logs in a separate AWS account. See blog post.

Deploying

  1. Launch AWS Elasticsearch Domain. $ aws cloudformation create-stack --stack-name LogEsDomain --template-body file://es.yml --parameters ParameterKey=KibanaJumpboxPublicIpAddress,ParameterValue=x.x.x.x --profile isolated-account-admin

  2. Launch AWS Kinesis Firehose that delivers logs to Elasticsearch Domain in step 1. $ aws cloudformation create-stack --stack-name LogFirehose --template-body file://firehose.yml --parameters ParameterKey=ElasticsearchDomainArn,ParameterValue=<from_step_1> ParameterKey=FirehoseBackupS3KmsKeyArn,ParameterValue=<kms_key> --capabilities CAPABILITY_IAM --profile isolated-account-admin

  3. Launch log sink stack that accepts logs from another account and forwards them to the AWS Kinesis Firehose in step 2. $ aws cloudformation create-stack --stack-name LogSink --template-body file://cloudwatch_logs_sink.yml --parameters ParameterKey=LogSinkFirehoseDeliveryStream,ParameterValue=<from_step_2> ParameterKey=SubscriptionFilterAccountId,ParameterValue=111111111111 --capabilities CAPABILITY_IAM --profile isolated-account-admin

  4. Launch log source in each AWS account to send logs to the log sink in the isolated AWS account. $ aws cloudformation create-stack --stack-name LogSource --template-body file://cloudwatch_logs_source.yml --parameters ParameterKey=DestinationArn,ParameterValue=<from_step_3> --profile 111111111111-admin