Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding terraform references for some buckets #1229

Merged
merged 2 commits into from
Apr 7, 2020

Conversation

ryandeivert
Copy link
Contributor

to: @airbnb/streamalert-maintainers

Background

A race condition could cause errors like:

Error: Error putting S3 notification configuration: NoSuchBucket: The specified bucket does not exist
	status code: 404, request id: 5A5ACCB6EB4055B8, host id: vIlTAABEslxBZXQXmF5QRhFi6z4kQ0wW2YUeJdMLKjGpJn4u5irjwctlzAPPDIY1RKsktnKyh5s=
  on modules/tf_athena/main.tf line 100, in resource "aws_s3_bucket_notification" "bucket_notification":
 100: resource "aws_s3_bucket_notification" "bucket_notification" {
...

Changes

  • Using references to terraform resources/outputs for bucket names to enforce the creation of the resources before we try to use their "value".

Testing

Updates to unit tests.

@ryandeivert ryandeivert added this to the 3.2.0 milestone Apr 7, 2020
@coveralls
Copy link

Coverage Status

Coverage increased (+0.005%) to 95.43% when pulling 3a6752b on ryandeivert-adding-tf-bucket-refs into b552a05 on release-3-2-0.

Copy link
Contributor

@Ryxias Ryxias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it works, it isn't stupid

That said, would this be a candidate for using a depends_on block?

@ryandeivert
Copy link
Contributor Author

@Ryxias a reference to a resource is an implicit depends_on.. actually, depends_on should rarely be needed at all (if ever in most of our cases). see here:

Most resource dependencies are handled automatically. Terraform analyses any expressions within a resource block to find references to other objects, and treats those references as implicit ordering requirements when creating, updating, or destroying resources. Since most resources with behavioral dependencies on other resources also refer to those resources' data, it's usually not necessary to manually specify dependencies between resources.

@ryandeivert ryandeivert merged commit 771ce8a into release-3-2-0 Apr 7, 2020
@ryandeivert ryandeivert deleted the ryandeivert-adding-tf-bucket-refs branch April 7, 2020 18:34
ryandeivert added a commit that referenced this pull request Apr 9, 2020
* bumping version to 3.2.0

* migrating Athena function to use tf_lambda module (#1217)

* rename of athena function

* updating terraform generation code to use tf_lambda module

* updating tf_athena module to remove lambda code

* updates for packaging, rollback, and deploy

* misc updates related to config path renaming, etc

* removing no-longer-used method (athena is default)

* addressing PR feedback

* adding more granular time prefix to athena client

* fixing duplicate resource issues (#1218)

* fixing duplicate resource issues

* fixing some other bugs in #1217

* fixing tf targets for athena deploy (#1220)

* adding "--config-dir" flag to CLI to support specifying path for config files (#1224)

* adding support for supplying path to config via CLI flag

* misc touchups

* updating publishers to accept configurable paths (#1223)

* moving matchers outside of rules directory

* updating rules for new matcher path

* updating unit test for consistency

* making publisher locations configurable

* fixing typo

* updating tf_lambda module to remove extra resources (#1225)

* fixing rollback for all functions, removing 'all' flag for function deploys (#1222)

* updating rollback functionality to include all funcs

* updating tests to check for rollback of all funcs

* updating docs

* fixing tf cycle and index issue (#1226)

* Add missing dependency (#1228)

* Implements a v2 Lambda Output with AssumeRole (#1227)

* First draft of aws-lambda-v2

* Tests

* Fixup

* Fixup

* Fioxup

* Fixup

* fixup

* adding terraform references for some buckets (#1229)

* adding athena terraform references instead of literals

* fixing tests

* GitHub Actions (#1231)

* port to github actions

* remove travis

* cover the 3.2 branch for now too

* initial updates to simplify lambda packaging logic (#1232)

* moving some precompiled files

* initial revamp to packaging to remove multiple pacakges

* taking out more trash

* update scheduled queries module

* updating deploy logic to suck garbage slightly less

* updates to unit tests

* addressing pr feedback

* addressing PR feedback

* small update to docs (#1233)

Co-authored-by: Ryxias <derek.wang@airbnb.com>
Co-authored-by: Paul Kehrer <paul.l.kehrer@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants