New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add new firehose stream #35810
Add new firehose stream #35810
Conversation
7c65ddc
to
5f0c24a
Compare
5f0c24a
to
1021c3c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall this looks good! It could be merged as-is to unblock work moving forward. Added a couple of (optional) syntax/style comments below just for potential polish.
data_json: { | ||
level_source_id: @level_source.id | ||
}.to_json | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(optional) to reduce the diff (and/or to remove some curly braces) you could use Ruby's implicit conversion of keyword parameters into a hash passed as the last argument:
FirehoseClient.instance.put_record(FirehoseClient::ANALYSIS_EVENTS_STREAM_NAME,
study: 'share_filtering'
[...etc...]
)
lib/cdo/firehose.rb
Outdated
# string_key: 'home.heading_elementary_markdown' | ||
# } | ||
# ) | ||
I18N_STRING_TRACKING_EVENTS_STREAM_NAME = 'i18n-string-tracking-events'.freeze |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These names seem a bit verbose to me, I tend to prefer variables/constants passed around in business-logic code to be shorter/more readable based on context. How about something like a fixed hash to lookup the full stream name based on a short symbol, e.g.:
STREAMS = {
analysis: 'analysis-events',
i18n: 'i18n-string-tracking-events'
}
def put_record(stream, data)
stream_name = STREAMS[stream] || raise "Stream #{stream} not found"
[...]
end
FirehoseClient.instance.put_record(:i18n,
url: 'http://studio.code.org/courses',
string_key: 'home.heading_elementary_markdown'
)
lib/cdo/firehose.rb
Outdated
wait_at_exit: 10.0 | ||
) | ||
class Buffer < Cdo::Buffer | ||
# Initializes the @firehose to an AWS Firehose client. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this comment was already obsolete (and doubly so with this refactoring).
wait_at_exit: 10.0 | ||
) | ||
|
||
raise ArgumentError.new("stream_name must be defined") if stream_name.nil? || stream_name.blank? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can make a keyword argument required (and even better, caught at compile-time) by removing the default value, e.g.:
def initialize(stream_name:)
1021c3c
to
197b847
Compare
197b847
to
d9cd4a1
Compare
I'm implementing a new feature which will use Firehose but to a different Kinesis stream than the one we are already using. The PR just adds the ability to target different streams.
FirehoseClient.put_record
to have astream_name
parameter so we can target different streams.Links
Testing story
Reviewer Checklist: