Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(core) add firehose delivery ingestion stage #71

Merged
merged 25 commits into from
Apr 4, 2022

Conversation

malachi-constant
Copy link
Contributor

Feature

  • Adding stage for firehose ingestion to an S3 bucket.

Detail

    • Resource Factory: FirehoseFactory()

Relates

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@malachi-constant malachi-constant added enhancement New feature or request core labels Mar 29, 2022
@malachi-constant malachi-constant added this to the DDK Stages milestone Mar 29, 2022
@malachi-constant malachi-constant self-assigned this Mar 29, 2022
@malachi-constant malachi-constant linked an issue Mar 29, 2022 that may be closed by this pull request
@malachi-constant malachi-constant marked this pull request as draft March 29, 2022 19:20
@malachi-constant malachi-constant added this to In Progress in Roadmap Mar 31, 2022
@malachi-constant malachi-constant marked this pull request as ready for review March 31, 2022 19:40
core/aws_ddk_core/resources/_kinesis.py Outdated Show resolved Hide resolved
core/aws_ddk_core/resources/_kinesis.py Outdated Show resolved Hide resolved
core/aws_ddk_core/stages/firehose_s3.py Outdated Show resolved Hide resolved
core/aws_ddk_core/stages/firehose_s3.py Outdated Show resolved Hide resolved
core/aws_ddk_core/stages/firehose_s3.py Outdated Show resolved Hide resolved
@kukushking kukushking moved this from In Progress to Review in Roadmap Apr 1, 2022

This construct allows to configure parameters of the firehose delivery stream using ddk.json
configuration file depending on the `environment_id` in which the function is used.
Supported parameters are:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor - missing supported params

elif data_stream:
self._data_stream = data_stream
else:
self._data_stream = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor - I guess you can just leave else: self._data_stream = data_stream

core/aws_ddk_core/resources/_kinesis.py Outdated Show resolved Hide resolved
core/aws_ddk_core/resources/_kinesis.py Outdated Show resolved Hide resolved
core/aws_ddk_core/resources/_kinesis.py Outdated Show resolved Hide resolved
core/aws_ddk_core/resources/_kinesis.py Outdated Show resolved Hide resolved
core/aws_ddk_core/resources/_kinesisfirehose.py Outdated Show resolved Hide resolved
core/aws_ddk_core/stages/firehose_s3.py Outdated Show resolved Hide resolved
core/aws_ddk_core/resources/_kinesisfirehose.py Outdated Show resolved Hide resolved
core/aws_ddk_core/resources/_kinesisfirehose.py Outdated Show resolved Hide resolved
core/aws_ddk_core/stages/firehose_s3.py Outdated Show resolved Hide resolved
core/aws_ddk_core/stages/firehose_s3.py Outdated Show resolved Hide resolved
@jaidisido jaidisido merged commit c08d163 into main Apr 4, 2022
@jaidisido jaidisido deleted the 70-core-add-firehose-delivery-ingestion-stage branch April 4, 2022 16:57
@malachi-constant malachi-constant moved this from Review to Done in Roadmap Apr 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core enhancement New feature or request
Projects
Development

Successfully merging this pull request may close these issues.

(core) add firehose delivery ingestion stage
3 participants