Fix: hosts => "es_host:port" regression (when ssl => true)

[kares]

The change here makes sure "es_host:port" is properly split into a { host: es_host, port: port } pair.
Elasticsearch gem not seeing a port: key will add the port: 9200 default and Manticore 0.8 appends the port part while building the URL.

---

The issue did reproduce with the added integration test (under SECURE_INTEGRATION) - we did not have testing setup with secured ES, the setup has been ~ copied from ES output plugin.

There's also some additional test targets to keep testing against previous ES and Manticore gem versions.

---

CI 🔴 are caused by #157 (and logstash-plugins/logstash-input-elasticsearch#165)

[kares]

✂️ not used atm

[jsvd]

This cert expires in a couple of years, will this break tests or is curl -k the only relevant validation skip we need?

[kares]

Indeed it will break integration specs eventually, just like in ES output: https://github.com/logstash-plugins/logstash-output-elasticsearch/tree/v11.4.1/spec/fixtures/test_certs where it's copied from.

We do not have a base-line setup on generating TLS material in tests and there isn't a good enough excuse to start looking into that atm (simply due having more pressing stuff to tackle).
Hopefully the TLS setting unification effort will be a place to look into these...

[jsvd]

agreed it doesn't need to be in scope, it's a good target for refactoring if we do an integration plugin. For reference, we have some plugins that generate certs during the tests, such as http input and tcp input. example here: https://github.com/logstash-plugins/logstash-input-tcp/blob/main/spec/spec_helper.rb#L37-L52

[kares]

yeah I know of these but here we need to have PEM certificates ahead of time during Docker build ...
(to setup a secured TLS configuration in ES)

[jsvd]

LGTM.

The ES ruby client behaves very inconsistently wrt hosts input:

> begin; Elasticsearch::Client.new(hosts: "localhost:9000").transport.transport.connections.first.host; rescue => e; puts [e.class, e.message];end
=> {:host=>"localhost", :port=>9000, :protocol=>"http"}

> begin; Elasticsearch::Client.new(hosts: {host: "localhost:9000"}).transport.transport.connections.first.host; rescue => e; puts [e.class, e.message];end
URI::InvalidURIError
bad URI(is not URI?): http://localhost:9000:9200

> begin; Elasticsearch::Client.new(hosts: "http://localhost:9000").transport.transport.connections.first.host; rescue => e; puts [e.class, e.message];end
=> {:scheme=>"http", :user=>nil, :password=><REDACTED>, :host=>"localhost", :path=>"", :port=>9000, :protocol=>"http"}

> begin; Elasticsearch::Client.new(hosts: {host: "http://localhost:9000"}).transport.transport.connections.first.host; rescue => e; puts [e.class, e.message];end
=> {:host=>"http://localhost:9000", :protocol=>"http", :port=>9200}

I'll open an issue to track this on that side.