Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support logstash 5 #2

Merged
merged 4 commits into from Feb 9, 2018
Merged

Support logstash 5 #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d34e681
Lazily register the client
Jan 24, 2018
34ea798
Remove access key options
Jan 24, 2018
6739f27
Make it purty
Jan 24, 2018
7a1e2d3
Bump versions to support Logstash 5.x
Jan 24, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
12 changes: 2 additions & 10 deletions lib/logstash/outputs/firehose.rb
View file
Open in desktop
Expand Up @@ -45,8 +45,6 @@ class LogStash::Outputs::Firehose < LogStash::Outputs::Base

# make properties visible for tests
attr_accessor :stream
attr_accessor :access_key_id
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how do you specify access and secret keys to your stream from now on?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the same way. By including the AWS Mixin these configuration options are already available.

attr_accessor :secret_access_key
attr_accessor :codec

config_name "firehose"
Expand All @@ -57,8 +55,6 @@ class LogStash::Outputs::Firehose < LogStash::Outputs::Base
# Firehose stream info
config :region, :validate => :string, :default => "us-east-1"
config :stream, :validate => :string
config :access_key_id, :validate => :string
config :secret_access_key, :validate => :string

#
# Register plugin
Expand All @@ -70,9 +66,6 @@ def register
#Aws.eager_autoload!(Aws::Firehose)
#Aws.eager_autoload!(services: %w(Firehose))

# Create Firehose API client
@firehose = aws_firehose_client

# Validate stream name
if @stream.nil? || @stream.empty?
@logger.error("Firehose: stream name is empty", :stream => @stream)
Expand Down Expand Up @@ -105,8 +98,7 @@ def receive(event)
# Build AWS Firehose client
private
def aws_firehose_client
@logger.info "Registering Firehose output", :stream => @stream, :region => @region
@firehose = Aws::Firehose::Client.new(aws_full_options)
@firehose ||= Aws::Firehose::Client.new(aws_full_options)
end

# Build and return AWS client options map
Expand Down Expand Up @@ -137,7 +129,7 @@ def push_data_into_stream(encoded_event)
@logger.debug "Pushing encoded event: #{encoded_event}"

begin
@firehose.put_record({
aws_firehose_client.put_record({
delivery_stream_name: @stream,
record: {
data: encoded_event
Expand Down
18 changes: 9 additions & 9 deletions logstash-output-firehose.gemspec
View file
Open in desktop
@@ -1,12 +1,12 @@
Gem::Specification.new do |s|
s.name = 'logstash-output-firehose'
s.version = "0.0.2"
s.licenses = ["Apache License (2.0)"]
s.summary = "Output plugin to push data into AWS Kinesis Firehose stream."
s.description = "This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program"
s.authors = ["Valera Chevtaev"]
s.email = "myltik@gmail.com"
s.homepage = "https://github.com/chupakabr/logstash-output-firehose"
s.name = 'logstash-output-firehose'
s.version = "0.0.2"
s.licenses = ["Apache License (2.0)"]
s.summary = "Output plugin to push data into AWS Kinesis Firehose stream."
s.description = "This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program"
s.authors = ["Valera Chevtaev"]
s.email = "myltik@gmail.com"
s.homepage = "https://github.com/chupakabr/logstash-output-firehose"
s.require_paths = ["lib"]

# Files
Expand All @@ -20,7 +20,7 @@ Gem::Specification.new do |s|

# Gem dependencies
s.add_runtime_dependency "stud", "~> 0.0.22"
s.add_runtime_dependency "logstash-core", ">= 2.0.0", "< 3.0.0"
s.add_runtime_dependency "logstash-core-plugin-api", ">= 1.60", "<= 2.99"
s.add_runtime_dependency "logstash-mixin-aws", ">= 2.0.2"
s.add_runtime_dependency "logstash-codec-line"
s.add_runtime_dependency "logstash-codec-json_lines"
Expand Down
2 changes: 0 additions & 2 deletions spec/outputs/firehose_spec.rb
View file
Open in desktop
Expand Up @@ -19,8 +19,6 @@

# Setup Firehose client
output.stream = "aws-test-stream"
output.access_key_id = "Key ID"
output.secret_access_key = "Secret key"
output.register
end

Expand Down