Kinesis Elasticsearch Sink: configure transport port

[xkidro]

I searched the source code from https://github.com/snowplow/snowplow/tree/master/4-storage/kinesis-elasticsearch-sink I can't seem to find where the 9300 is hard coded and can't find a way to configure it as well.

[fblundun]

Hi @xkidro ,

9300 is the default value used in the amazon-kinesis-connectors library. We override several of these default values here, but the port is not (yet) one of them.

[xkidro]

Hello @fblundun thank you for the clarification, I have a question, it's not really related with this but it is related to the other issue I posted, didn't really study these but would it be possible for the sink to use the http api instead of the transport client? Does the kinesis connectors library support this? If so, would it be hard to modify the sink to work with that? I really want to use the new AWS elasticsearch service but it doesn't support the transport client :(

[fblundun]

Unfortunately not - we only support the transport client. Compatibility with Amazon Elasticsearch Service would be nice but as you say they don't support the transport client (source). A pull request adding the option to use the HTTP client would be welcome!

[fblundun]

Note that we don't actually use the amazon-kinesis-connectors Elasticsearch emitter - we implement that ourselves:

snowplow/4-storage/kinesis-elasticsearch-sink/src/main/scala/com.snowplowanalytics.snowplow.storage.kinesis/elasticsearch/SnowplowElasticsearchEmitter.scala

Line 165 in 9ee272a

private val elasticsearchClient = new TransportClient(settings)

[alexanderdean]

Bringing forwards as pretty straightforward add