Skip to content

Commit

Permalink
Merge pull request #47 from apache/cassandraql-sink
Browse files Browse the repository at this point in the history
CassandraQL sink connector example
  • Loading branch information
oscerd committed Dec 13, 2019
2 parents e6f4d0a + 1411d6b commit 79b0ebe
Show file tree
Hide file tree
Showing 2 changed files with 84 additions and 0 deletions.
57 changes: 57 additions & 0 deletions docs/try-it-out-locally.adoc
Expand Up @@ -126,6 +126,63 @@ export CLASSPATH="$(find core/target/ -type f -name '*.jar'| grep '\-package' |
$KAFKA_HOME/bin/connect-standalone.sh $KAFKA_HOME/config/connect-standalone.properties examples/CamelCassandraQLSourceConnector.properties
----

.Run the cassandraql sink:

To run this example you'll need a bit more work:

First you'll need to run a cassandra instance:

[source,bash]
----
docker run --name master_node --env MAX_HEAP_SIZE='800M' -dt oscerd/cassandra
----

Check everything is fine:

[source,bash]
----
docker exec -ti master_node /opt/cassandra/bin/nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID Rack
UN 172.17.0.2 251.32 KiB 256 100.0% 5126aaad-f143-43e9-920a-0f9540a93967 rack1
----

You'll need a local installation of cassandra, in particular the 3.11.4.
Now we can populate the database:

[source,bash]
----
<LOCAL_CASSANDRA_HOME>/bin/cqlsh $(docker inspect --format='{{ .NetworkSettings.IPAddress }}' master_node)
----

and run the script:

[source,bash]
----
create keyspace test with replication = {'class':'SimpleStrategy', 'replication_factor':3};
use test;
create table users (id uuid primary key, name text );
insert into users (id,name) values (now(), 'oscerd');
quit;
----

The output of the following command should be used in the configuration of CamelCassandraQLSourceConnector.properties

[source,bash]
----
<LOCAL_CASSANDRA_HOME>/bin/cqlsh $(docker inspect --format='{{ .NetworkSettings.IPAddress }}' master_node)
----

in particular it should be used as address instead of localhost in the `camel.sink.url`
[source,bash]
----
export CLASSPATH="$(find core/target/ -type f -name '*.jar'| grep '\-package' | tr '\n' ':')"
$KAFKA_HOME/bin/connect-standalone.sh $KAFKA_HOME/config/connect-standalone.properties examples/CamelCassandraQLSinkConnector.properties
----

.Run the file sink, just a camel file appending to /tmp/kafkaconnect.txt:
[source,bash]
----
Expand Down
27 changes: 27 additions & 0 deletions examples/CamelCassandraQLSinkConnector.properties
@@ -0,0 +1,27 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

name=CamelCassandraQLSinkConnector
topics=mytopic
tasks.max=1
connector.class=org.apache.camel.kafkaconnector.CamelSinkConnector
key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=org.apache.kafka.connect.storage.StringConverter

camel.sink.url=cql://localhost/test?cql=insert into users(id, name) values (now(), ?)


0 comments on commit 79b0ebe

Please sign in to comment.