Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elastic connector #2546

Merged
merged 29 commits into from
Sep 10, 2018
Merged

Elastic connector #2546

merged 29 commits into from
Sep 10, 2018

Conversation

david-streamlio
Copy link
Contributor

Motivation

Added a sink connector that writes JSON documents into ElasticSearch

Modifications

Added new pulsar-io module and associated integration tests

Result

An ElasticSearch sink connector will be available for use.

@Test
public void testCassandraArchiveSink() throws Exception {
testSink(new CassandraSinkArchiveTester(), false);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did we delete the testCassandraArchiveSink test?

}

private void testSink(SinkTester tester, boolean builtin) throws Exception {
private void testSink(SinkTester tester) throws Exception {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a bunch of changes were reverted from master by mistake?

@srkukarni srkukarni added this to the 2.2.0-incubating milestone Sep 7, 2018
@srkukarni
Copy link
Contributor

run integration tests

@david-streamlio
Copy link
Contributor Author

How do I view the actual cause of the error..

org.apache.pulsar.tests.integration.docker.ContainerExecException: /pulsar/bin/pulsar-admin sink create --tenant public --namespace default --name test-sink-connector-elastic_search-THREAD-name-exoiydod --sink-type elastic_search --sinkConfig {"elasticSearchUrl":"http://localhost:9200","indexName":"test-index"} --inputs test-sink-connector-ELASTIC_SEARCH-THREAD-input-topic-midgxwab failed on e463877cd694fe13c88a7b3695cabe3c2840fae0f25b095b1483f2c244ad1d05 with error code 1
at org.apache.pulsar.tests.integration.utils.DockerUtils.runCommand(DockerUtils.java:234)
at org.apache.pulsar.tests.integration.containers.ChaosContainer.execCmd(ChaosContainer.java:109)
at org.apache.pulsar.tests.integration.functions.PulsarFunctionsTest.submitSinkConnector(PulsarFunctionsTest.java:171)
at org.apache.pulsar.tests.integration.functions.PulsarFunctionsTest.testSink(PulsarFunctionsTest.java:107)
at org.apache.pulsar.tests.integration.functions.PulsarFunctionsTest.testElasticSearchSink(PulsarFunctionsTest.java:89)

@sijie sijie merged commit 928c3e1 into apache:master Sep 10, 2018
@david-streamlio david-streamlio deleted the elastic-connector branch September 11, 2018 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants