Skip to content

Updated Cassandraql example to use Eclipse JKube #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 7, 2020
Merged

Updated Cassandraql example to use Eclipse JKube #12

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
154 changes: 82 additions & 72 deletions examples/camel-example-cassandra-kubernetes/README.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,47 +5,52 @@ Apache Camel (Cassandraql component).

This example is based on:

* Minikube 0.21.0 (Kubernetes version >= 1.7)
* Fabric8 Maven Plugin (version >= 3.5)
* Minikube 1.11.0 (Kubernetes version >= 1.17)
* https://www.eclipse.org/jkube[Eclipse JKube]

First thing you'll need to do is preparing the environment.
First thing you'll need to do is to prepare the environment.

Don't forget to use a bit more memory for your Minikube for running this
example:
Don't forget to use a bit more memory for your Minikube setup to run this
example smoothly:

....
[source,sh]
----
$ minikube start --memory 5120 --cpus=4
....
----

Once your Minikube node is up and running you'll need to run the
following command. In your src/main/resource/fabric8/ folder you'll find
two yaml file. Run the following command using them:
following command. In your `src/main/resource/jkube/` folder you'll find
two yaml files. Run the following commands using them to prepare the Cassandra cluster:

....
$ kubectl create -f src/main/resources/fabric8/cassandra-service.yaml
$ kubectl create -f src/main/resources/fabric8/cassandra-statefulset.yaml
....
[source,sh]
----
$ kubectl create -f src/main/resources/jkube/cassandra-service.yaml
$ kubectl create -f src/main/resources/jkube/cassandra-statefulset.yaml
----

To check the correct startup of the cluster run the following command:

....
$ kubectl get statefulsets
NAME DESIRED CURRENT AGE
cassandra 2 2 2h
....
[source,sh]
----
$ kubectl get statefulsets.apps
NAME READY AGE
cassandra 2/2 3h44m
----

and check the status of the pods

....
[source,sh]
----
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
cassandra-0 1/1 Running 0 2h
cassandra-1 1/1 Running 0 2h
....
----

You can also verify the health of your cluster by running

....
[source,sh]
----
$ kubectl exec <pod_name> -it nodetool status
Datacenter: DC1-K8Demo
======================
Expand All @@ -54,76 +59,81 @@ Status=Up/Down
-- Address Load Tokens Owns (effective) Host ID Rack
UN 172.17.0.4 212.14 KiB 32 53.1% 9bf81ccd-4aa1-451b-b56e-c16c5ee04836 Rack1-K8Demo
UN 172.17.0.6 170.08 KiB 32 46.9% 69cc6f60-9ccf-439d-a298-b79b643c1586 Rack1-K8Demo
....
----

=== Building and running

Navigate to the project folder and the example can be built with
Navigate to the project folder where the example can be built with

....
mvn clean -Pkubernetes-install fabric8:deploy
....
[source,sh]
----
$ mvn clean -Pkubernetes-install k8s:deploy
----

When the example runs in fabric8, you can use the Kubectl command tool
When the example runs in Kubernetes, you can use the Kubectl command tool
to inspect the status

To list all the running pods:

....
kubectl get pods
....

Then find the name of the pod that runs this quickstart, and output the
logs from the running pods with:

....
kubectl logs <name of pod>
....

and you should see something like this:

....
2017-08-06 10:43:52,209 [main ] INFO ClassPathXmlApplicationContext - Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@1068e947: startup date [Sun Aug 06 10:43:52 UTC 2017]; root of context hierarchy
2017-08-06 10:43:52,244 [main ] INFO XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [META-INF/spring/camel-context.xml]
2017-08-06 10:43:53,425 [main ] INFO GuavaCompatibility - Detected Guava >= 19 in the classpath, using modern compatibility layer
2017-08-06 10:43:53,564 [main ] INFO ClockFactory - Using native clock to generate timestamps.
2017-08-06 10:43:53,639 [main ] INFO NettyUtil - Did not find Netty's native epoll transport in the classpath, defaulting to NIO.
2017-08-06 10:43:54,054 [main ] INFO DCAwareRoundRobinPolicy - Using data-center name 'DC1-K8Demo' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor)
2017-08-06 10:43:54,056 [main ] INFO Cluster - New Cassandra host cassandra/172.17.0.2:9042 added
2017-08-06 10:43:54,056 [main ] INFO Cluster - New Cassandra host cassandra/172.17.0.4:9042 added
2017-08-06 10:43:56,845 [main ] INFO SpringCamelContext - Apache Camel 2.20.0-SNAPSHOT (CamelContext: camel-1) is starting
2017-08-06 10:43:56,846 [main ] INFO ManagedManagementStrategy - JMX is enabled
2017-08-06 10:43:57,105 [main ] INFO DefaultTypeConverter - Type converters loaded (core: 192, classpath: 1)
2017-08-06 10:43:57,225 [main ] INFO SpringCamelContext - StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
2017-08-06 10:43:57,230 [main ] INFO ClockFactory - Using native clock to generate timestamps.
2017-08-06 10:43:57,918 [main ] INFO DCAwareRoundRobinPolicy - Using data-center name 'DC1-K8Demo' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor)
2017-08-06 10:43:57,920 [main ] INFO Cluster - New Cassandra host cassandra/172.17.0.2:9042 added
2017-08-06 10:43:57,920 [main ] INFO Cluster - New Cassandra host cassandra/172.17.0.4:9042 added
2017-08-06 10:43:58,488 [main ] INFO SpringCamelContext - Route: cassandra-route started and consuming from: timer://foo?period=5000
2017-08-06 10:43:58,489 [main ] INFO SpringCamelContext - Total 1 routes, of which 1 are started.
2017-08-06 10:43:58,489 [main ] INFO SpringCamelContext - Apache Camel 2.20.0-SNAPSHOT (CamelContext: camel-1) started in 1.645 seconds
2017-08-06 10:43:58,492 [main ] INFO DefaultLifecycleProcessor - Starting beans in phase 2147483646
2017-08-06 10:43:59,586 [2 - timer://foo] INFO cassandra-route - Query result set [Row[1, oscerd]]
2017-08-06 10:44:04,575 [2 - timer://foo] INFO cassandra-route - Query result set [Row[1, oscerd]]
2017-08-06 10:44:09,577 [2 - timer://foo] INFO cassandra-route - Query result set [Row[1, oscerd]]
....
[source,sh]
----
$ kubectl get pods
----

You can follow the log for the created Pod by running:

[source,sh]
----
$ mvn -Pkubernetes-install k8s:log
----

You should then see an output similar to this:

[source,sh]
----
[INFO] k8s: 2020-08-07 12:34:32,569 [main ] INFO GuavaCompatibility - Detected Guava >= 19 in the classpath, using modern compatibility layer
[INFO] k8s: 2020-08-07 12:34:32,834 [main ] INFO ClockFactory - Using native clock to generate timestamps.
[INFO] k8s: 2020-08-07 12:34:33,005 [main ] INFO NettyUtil - Did not find Netty's native epoll transport in the classpath, defaulting to NIO.
[INFO] k8s: 2020-08-07 12:34:34,122 [main ] INFO DCAwareRoundRobinPolicy - Using data-center name 'DC1-K8Demo' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor)
[INFO] k8s: 2020-08-07 12:34:34,124 [main ] INFO Cluster - New Cassandra host cassandra/172.17.0.7:9042 added
[INFO] k8s: 2020-08-07 12:34:34,150 [main ] INFO Cluster - New Cassandra host cassandra/172.17.0.6:9042 added
[INFO] k8s: 2020-08-07 12:34:36,780 [main ] INFO CqlPopulateBean - Cassandra was populated with sample values for test.users table
[INFO] k8s: 2020-08-07 12:34:37,372 [main ] INFO LRUCacheFactory - Detected and using LRUCacheFactory: camel-caffeine-lrucache
[INFO] k8s: 2020-08-07 12:34:38,012 [main ] INFO AbstractCamelContext - Apache Camel 3.5.0-SNAPSHOT (camel-1) is starting
[INFO] k8s: 2020-08-07 12:34:38,019 [main ] INFO AbstractCamelContext - StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
[INFO] k8s: 2020-08-07 12:34:38,019 [main ] INFO AbstractCamelContext - Using HealthCheck: camel-health
[INFO] k8s: 2020-08-07 12:34:38,664 [main ] INFO DefaultMavenCoordinates - DataStax Java driver for Apache Cassandra(R) (com.datastax.oss:java-driver-core) version 4.8.0
[INFO] k8s: 2020-08-07 12:34:39,554 [s0-admin-0 ] INFO Clock - Using native clock for microsecond precision
[INFO] k8s: 2020-08-07 12:34:41,453 [main ] INFO InternalRouteStartupManager - Route: cassandra-route started and consuming from: timer://foo
[INFO] k8s: 2020-08-07 12:34:41,454 [main ] INFO AbstractCamelContext - Total 1 routes, of which 1 are started
[INFO] k8s: 2020-08-07 12:34:41,455 [main ] INFO AbstractCamelContext - Apache Camel 3.5.0-SNAPSHOT (camel-1) started in 3.441 seconds
[INFO] k8s: 2020-08-07 12:34:41,469 [main ] INFO BaseMainSupport - Using properties from: classpath:application.properties;optional=true
[INFO] k8s: 2020-08-07 12:34:41,557 [main ] INFO DefaultRoutesCollector - No additional Camel XML routes discovered from: classpath:camel/*.xml
[INFO] k8s: 2020-08-07 12:34:41,557 [main ] INFO DefaultRoutesCollector - No additional Camel XML route templates discovered from: classpath:camel-template/*.xml
[INFO] k8s: 2020-08-07 12:34:41,559 [main ] INFO DefaultRoutesCollector - No additional Camel XML rests discovered from: classpath:camel-rest/*.xml
[INFO] k8s: 2020-08-07 12:34:42,557 [1 - timer://foo] INFO cassandra-route - Query result set [1-oscerd,2-not-a-bot]
[INFO] k8s: 2020-08-07 12:34:47,548 [1 - timer://foo] INFO cassandra-route - Query result set [1-oscerd,2-not-a-bot]
[INFO] k8s: 2020-08-07 12:34:52,661 [1 - timer://foo] INFO cassandra-route - Query result set [1-oscerd,2-not-a-bot]
----

=== Cleanup

Run following to undeploy the application and cassandra nodes

....
$ mvn -Pkubernetes-install fabric8:undeploy
$ kubectl create -f src/main/resources/fabric8/cassandra-service.yaml
$ kubectl create -f src/main/resources/fabric8/cassandra-statefulset.yaml
....
[source,sh]
----
$ mvn -Pkubernetes-install k8s:undeploy
$ kubectl delete -f src/main/resources/jkube/cassandra-service.yaml
$ kubectl delete -f src/main/resources/jkube/cassandra-statefulset.yaml
----

Make sure no pod is running

....
[source,sh]
----
$ kubectl get pods
No resources found.
....
----

=== Help and contributions

Expand Down
76 changes: 50 additions & 26 deletions examples/camel-example-cassandra-kubernetes/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
<category>Cloud</category>
<!-- dependency versions -->
<cassandra.driver.version>3.7.2</cassandra.driver.version>
<main.class>org.apache.camel.spring.Main</main.class>
</properties>

<dependencyManagement>
Expand Down Expand Up @@ -72,6 +73,12 @@
<groupId>com.datastax.cassandra</groupId>
<artifactId>cassandra-driver-core</artifactId>
<version>${cassandra.driver.version}</version>
<exclusions>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty-handler</artifactId>
</exclusion>
</exclusions>
</dependency>

<!-- used for generating random message -->
Expand All @@ -98,30 +105,55 @@
<version>${log4j-version}</version>
</dependency>
</dependencies>

<!-- only run tests if this profile is enabled -->
<build>
<plugins>
<!-- allows the route to be ran via 'mvn exec:java' -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>${exec-maven-plugin-version}</version>
<configuration>
<mainClass>${main.class}</mainClass>
</configuration>
</plugin>
<!-- Create a Fat JAR -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>${maven-assembly-plugin-version}</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<appendAssemblyId>false</appendAssemblyId>
<archive>
<manifest>
<mainClass>${main.class}</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<id>fat-jar</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>kubernetes-install</id>

<build>
<defaultGoal>install</defaultGoal>

<plugins>

<plugin>
<groupId>io.fabric8</groupId>
<artifactId>fabric8-maven-plugin</artifactId>
<version>${fabric8-maven-plugin-version}</version>
<configuration>
<generator>
<config>
<java-exec>
<mainClass>org.apache.camel.spring.Main</mainClass>
</java-exec>
</config>
</generator>
</configuration>
<groupId>org.eclipse.jkube</groupId>
<artifactId>kubernetes-maven-plugin</artifactId>
<version>${jkube-version}</version>
<executions>
<execution>
<goals>
Expand All @@ -132,15 +164,7 @@
</executions>
</plugin>

<!-- allows the route to be ran via 'mvn exec:java' -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>${exec-maven-plugin-version}</version>
<configuration>
<mainClass>org.apache.camel.spring.Main</mainClass>
</configuration>
</plugin>


</plugins>
</build>
Expand Down
14 changes: 10 additions & 4 deletions ...ample/kubernetes/fmp/CqlPopulateBean.java → ...ple/kubernetes/jkube/CqlPopulateBean.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,21 +14,27 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.camel.example.kubernetes.fmp;
package org.apache.camel.example.kubernetes.jkube;

import com.datastax.driver.core.Cluster;
import com.datastax.driver.core.Session;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class CqlPopulateBean {

private static final Logger log = LoggerFactory.getLogger(CqlPopulateBean.class);

public void populate() {
Cluster cluster = Cluster.builder().addContactPoint("cassandra").build();
Session session = cluster.connect();
session.execute("create keyspace if not exists test with replication = {'class':'SimpleStrategy', 'replication_factor':1};");
session.execute("create table if not exists test.users ( id int primary key, name text );");
session.execute("insert into test.users (id,name) values (1, 'oscerd') if not exists;");
session.execute("CREATE KEYSPACE IF NOT EXISTS test WITH REPLICATION = {'class':'SimpleStrategy', 'replication_factor':1};");
session.execute("CREATE TABLE IF NOT EXISTS test.users ( id int primary key, name text );");
session.execute("INSERT INTO test.users (id,name) VALUES (1, 'oscerd') IF NOT EXISTS;");
session.execute("INSERT INTO test.users (id,name) VALUES (2, 'not-a-bot') IF NOT EXISTS;");
session.close();
cluster.close();
log.info("Cassandra was populated with sample values for test.users table");
}

}
36 changes: 36 additions & 0 deletions ...ndra-kubernetes/src/main/java/org/apache/camel/example/kubernetes/jkube/RowProcessor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.camel.example.kubernetes.jkube;

import com.datastax.oss.driver.api.core.cql.Row;
import org.apache.camel.Exchange;
import org.apache.camel.Processor;

import java.util.List;
import java.util.stream.Collectors;

public class RowProcessor implements Processor {
@SuppressWarnings("unchecked")
@Override
public void process(Exchange exchange) {
final List<Row> rows = exchange.getIn().getBody(List.class);
exchange.getIn().setBody(rows.stream()
.map(row -> String.format("%s-%s", row.getInt("id"), row.getString("name")))
.collect(Collectors.joining(","))
);
}
}
Loading