Error message doesn't work right.

[jhitze]

Version

/opt/logstash/bin/logstash --version
logstash 1.5.0

conf:

input {
  stdin {}
}

filter{ 
  geoip { source => "message" }
}

output {
  stdout { codec => rubydebug }
}

console (Send a dash "-"):

/opt/logstash/bin/logstash agent -f ~/test.conf -vvv
...{ Whole bunch of stuff I removed for brevity. }...
-
filter received {:event=>{"message"=>"-", "@version"=>"1", "@timestamp"=>"2015-06-05T01:44:49.078Z", "host"=>"vagrant-ubuntu-trusty-64"}, :level=>:debug, :file=>"(eval)", :line=>"40", :method=>"filter_func"}
IP Field contained invalid IP address or hostname {:field=>nil, :event=>#<LogStash::Event:0x794bc2a6 @metadata={}, @accessors=#<LogStash::Util::Accessors:0x5cd85e80 @store={"message"=>"-", "@version"=>"1", "@timestamp"=>"2015-06-05T01:44:49.078Z", "host"=>"vagrant-ubuntu-trusty-64"}, @lut={"host"=>[{"message"=>"-", "@version"=>"1", "@timestamp"=>"2015-06-05T01:44:49.078Z", "host"=>"vagrant-ubuntu-trusty-64"}, "host"], "message"=>[{"message"=>"-", "@version"=>"1", "@timestamp"=>"2015-06-05T01:44:49.078Z", "host"=>"vagrant-ubuntu-trusty-64"}, "message"]}>, @data={"message"=>"-", "@version"=>"1", "@timestamp"=>"2015-06-05T01:44:49.078Z", "host"=>"vagrant-ubuntu-trusty-64"}, @metadata_accessors=#<LogStash::Util::Accessors:0x5b02de9a @store={}, @lut={}>, @cancelled=false>, :level=>:error, :file=>"logstash/filters/geoip.rb", :line=>"112", :method=>"filter"}
output received {:event=>{"message"=>"-", "@version"=>"1", "@timestamp"=>"2015-06-05T01:44:49.078Z", "host"=>"vagrant-ubuntu-trusty-64"}, :level=>:debug, :file=>"(eval)", :line=>"46", :method=>"output_func"}
{
       "message" => "-",
      "@version" => "1",
    "@timestamp" => "2015-06-05T01:44:49.078Z",
          "host" => "vagrant-ubuntu-trusty-64"
}

Problem:

IP Field contained invalid IP address or hostname {:field=>nil, 

It is returning the fieldas nil. Looking at the source code it always will. I don't see anywhere where @field is instantiated. Would it make more sense to have @source be the field it is using for the error message?

I ran into this because my IIS logs would sometimes put a - into the field I am doing the GeoIP filter on. When I looked in my logs, I had no idea what was going on, since :field was nil.

[jsvd]

🤦 this should clearly be @source. thanks for spotting this. you can create a pr to change @field to @source if you want

[jhitze]

Sure thing. Just a quick question: Do all the tests fail for you?

jhitze$ be rspec
Using Accessor#strict_set for specs
Run options: exclude {:redis=>true, :socket=>true, :performance=>true, :couchdb=>true, :elasticsearch=>true, :elasticsearch_secure=>true, :export_cypher=>true, :integration=>true, :windows=>true}
FFFFFFFFFFFFFFF

Failures:

  1) LogStash::Filters::GeoIP ASN db "{"ip":"1.1.1.1"}" when processed
     Failure/Error: let(:pipeline) { LogStash::Pipeline.new(config) }
     LogStash::ConfigurationError:
       Something is wrong with your configuration.
     # (eval):7:in `initialize'
     # ./spec/filters/geoip_spec.rb:19:in `(root)'
     # ./spec/filters/geoip_spec.rb:19:in `(root)'

  2) LogStash::Filters::GeoIP ASN db "{"ip":"2a02:8071:aa1:c700:7984:22fc:c8e6:f6ff"}" when processed
     Failure/Error: let(:pipeline) { LogStash::Pipeline.new(config) }
     LogStash::ConfigurationError:
       Something is wrong with your configuration.
     # (eval):7:in `initialize'
     # ./spec/filters/geoip_spec.rb:25:in `(root)'
     # ./spec/filters/geoip_spec.rb:25:in `(root)'

  3) LogStash::Filters::GeoIP defaults "{"ip":"8.8.8.8"}" when processed
     Failure/Error: pipeline.instance_eval { @filters.each(&:register) }
     TypeError:
       can't convert nil into String
     # ./lib/logstash/filters/geoip.rb:66:in `register'
     # ./spec/filters/geoip_spec.rb:40:in `(root)'
     # ./spec/filters/geoip_spec.rb:40:in `(root)'

  4) LogStash::Filters::GeoIP defaults "{"ip":"127.0.0.1"}" when processed
     Failure/Error: pipeline.instance_eval { @filters.each(&:register) }
     TypeError:
       can't convert nil into String
     # ./lib/logstash/filters/geoip.rb:66:in `register'
     # ./spec/filters/geoip_spec.rb:53:in `(root)'
     # ./spec/filters/geoip_spec.rb:53:in `(root)'

  5) LogStash::Filters::GeoIP location field when longitude field is excluded behaves like an event with a [geoip][location] field should have a location field
     Failure/Error: plugin.register
     TypeError:
       can't convert nil into String
     Shared Example Group: "an event with a [geoip][location] field" called from ./spec/filters/geoip_spec.rb:166
     # ./lib/logstash/filters/geoip.rb:66:in `register'
     # ./spec/filters/geoip_spec.rb:150:in `(root)'

  6) LogStash::Filters::GeoIP location field when both latitude and longitude field are excluded behaves like an event with a [geoip][location] field should have a location field
     Failure/Error: plugin.register
     TypeError:
       can't convert nil into String
     Shared Example Group: "an event with a [geoip][location] field" called from ./spec/filters/geoip_spec.rb:171
     # ./lib/logstash/filters/geoip.rb:66:in `register'
     # ./spec/filters/geoip_spec.rb:150:in `(root)'

  7) LogStash::Filters::GeoIP location field when latitude field is excluded behaves like an event with a [geoip][location] field should have a location field
     Failure/Error: plugin.register
     TypeError:
       can't convert nil into String
     Shared Example Group: "an event with a [geoip][location] field" called from ./spec/filters/geoip_spec.rb:161
     # ./lib/logstash/filters/geoip.rb:66:in `register'
     # ./spec/filters/geoip_spec.rb:150:in `(root)'

  8) LogStash::Filters::GeoIP Specify the target "{"ip":"8.8.8.8"}" when processed
     Failure/Error: pipeline.instance_eval { @filters.each(&:register) }
     TypeError:
       can't convert nil into String
     # ./lib/logstash/filters/geoip.rb:66:in `register'
     # ./spec/filters/geoip_spec.rb:69:in `(root)'
     # ./spec/filters/geoip_spec.rb:69:in `(root)'

  9) LogStash::Filters::GeoIP Specify the target "{"ip":"127.0.0.1"}" when processed
     Failure/Error: pipeline.instance_eval { @filters.each(&:register) }
     TypeError:
       can't convert nil into String
     # ./lib/logstash/filters/geoip.rb:66:in `register'
     # ./spec/filters/geoip_spec.rb:82:in `(root)'
     # ./spec/filters/geoip_spec.rb:82:in `(root)'

  10) LogStash::Filters::GeoIP correct encodings with ASN db "{"ip":"187.2.0.0"}" when processed
     Failure/Error: let(:pipeline) { LogStash::Pipeline.new(config) }
     LogStash::ConfigurationError:
       Something is wrong with your configuration.
     # (eval):7:in `initialize'
     # ./spec/filters/geoip_spec.rb:134:in `(root)'
     # ./spec/filters/geoip_spec.rb:134:in `(root)'

  11) LogStash::Filters::GeoIP correct encodings with ASN db "{"ip":"189.2.0.0"}" when processed
     Failure/Error: let(:pipeline) { LogStash::Pipeline.new(config) }
     LogStash::ConfigurationError:
       Something is wrong with your configuration.
     # (eval):7:in `initialize'
     # ./spec/filters/geoip_spec.rb:137:in `(root)'
     # ./spec/filters/geoip_spec.rb:137:in `(root)'

  12) LogStash::Filters::GeoIP correct encodings with ASN db "{"ip":"1.1.1.1"}" when processed
     Failure/Error: let(:pipeline) { LogStash::Pipeline.new(config) }
     LogStash::ConfigurationError:
       Something is wrong with your configuration.
     # (eval):7:in `initialize'
     # ./spec/filters/geoip_spec.rb:131:in `(root)'
     # ./spec/filters/geoip_spec.rb:131:in `(root)'

  13) LogStash::Filters::GeoIP correct encodings with ASN db "{"ip":"161.24.0.0"}" when processed
     Failure/Error: let(:pipeline) { LogStash::Pipeline.new(config) }
     LogStash::ConfigurationError:
       Something is wrong with your configuration.
     # (eval):7:in `initialize'
     # ./spec/filters/geoip_spec.rb:140:in `(root)'
     # ./spec/filters/geoip_spec.rb:140:in `(root)'

  14) LogStash::Filters::GeoIP correct encodings with default db "{"ip":"1.1.1.1"}" when processed
     Failure/Error: pipeline.instance_eval { @filters.each(&:register) }
     TypeError:
       can't convert nil into String
     # ./lib/logstash/filters/geoip.rb:66:in `register'
     # ./spec/filters/geoip_spec.rb:101:in `(root)'
     # ./spec/filters/geoip_spec.rb:100:in `(root)'

  15) LogStash::Filters::GeoIP correct encodings with default db "{"ip":"189.2.0.0"}" when processed
     Failure/Error: pipeline.instance_eval { @filters.each(&:register) }
     TypeError:
       can't convert nil into String
     # ./lib/logstash/filters/geoip.rb:66:in `register'
     # ./spec/filters/geoip_spec.rb:110:in `(root)'
     # ./spec/filters/geoip_spec.rb:109:in `(root)'

Finished in 1.06 seconds
15 examples, 15 failures

[jhitze]

Bah. There is no vendor folder for the ASNDB. I guess I can fix that too. Is there some other way that you guys set up the test environment? Because I'm cloning just this repo and it the tests don't work. I had a similar issue with the logstash-output-riemann repo, but I assumed that was related to it having zero tests.

[jhitze]

Annnnnd I found the rake file. >.<

Sorry for the stream of consciousness comments, lol.

I'm going to update the readme.md to explain this part so the next person doesn't run into this.