Skip to content
Permalink
Browse files
Added a google-calendar-stream Source connector example
  • Loading branch information
oscerd committed Nov 9, 2020
1 parent 7d6b730 commit 5d09365d9f49b4b1b3fcd529177c0006351f6684
Show file tree
Hide file tree
Showing 2 changed files with 141 additions and 0 deletions.
@@ -0,0 +1,107 @@
= Camel-Kafka-connector Calendar Stream Source Example

This is an example for Camel-Kafka-connector Calendar Stream Source Example

== Standalone

=== What is needed

- A google account
- A google application authorized to deal with calendar API

=== Setting up Google Application

The step-by-step guide is a bit long, so follow it with attention:

1. Create a project
2. Note down clientSecret and clientId
2. Create an OAuth consent screen
3. Create OAuth Client credentials (OAuth for WebApp)
4. Add https://developers.google.com/oauthplayground to Authorized redirect URIs
5. Setup https://developers.google.com/oauthplayground using credentials from above
6. Authorize APIs for Calendar API v3 scope https://www.googleapis.com/auth/calendar
7. Exchange authorization code for tokens
8. Note down Access token & Refresh token
9. Create a calendar named "Test" on the account you're using
10. Note down the calendarId
11. Create an event in the calendar for the future.

For this example we created an event with "Test for ckc" as body.

=== Running Kafka

[source]
----
$KAFKA_HOME/bin/zookeeper-server-start.sh $KAFKA_HOME/config/zookeeper.properties
$KAFKA_HOME/bin/kafka-server-start.sh $KAFKA_HOME/config/server.properties
$KAFKA_HOME/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic mytopic
----

=== Download the connector package

Download the connector package zip and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/`

[source]
----
> cd /home/oscerd/connectors/
> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-google-calendar-stream-kafka-connector/0.6.0/camel-google-calendar-stream-kafka-connector-0.6.0-package.zip
> unzip camel-google-calendar-stream-kafka-connector-0.6.0-package.zip
----

=== Configuring Kafka Connect

You'll need to set up the `plugin.path` property in your kafka

Open the `$KAFKA_HOME/config/connect-standalone.properties` and set the `plugin.path` property to your choosen location:

[source]
----
...
plugin.path=/home/oscerd/connectors
...
----

=== Setup the connectors

Open the Git configuration file at `$EXAMPLES/google-calendar-stream/google-calendar-stream-source/config/CamelGoogleCalendarStreamSourceConnector.properties`

[source]
----
name=CamelGoogle-calendar-streamSourceConnector
connector.class=org.apache.camel.kafkaconnector.googlecalendarstream.CamelGooglecalendarstreamSourceConnector
tasks.max=1
key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=org.apache.kafka.connect.storage.StringConverter
topics=mytopic
camel.source.path.index=ckc
camel.source.endpoint.clientId=<client_id>
camel.source.endpoint.calendarId=<calendar_id>
camel.source.endpoint.accessToken=<access_token>
camel.source.endpoint.refreshToken=<refresh_token>
camel.source.endpoint.clientSecret=<client_secret>
camel.source.endpoint.applicationName=ckc
camel.source.endpoint.delay=60000
camel.source.endpoint.consumeFromNow=true
camel.source.endpoint.considerLastUpdate=true
----

=== Running the example

Run the kafka connect with the Google Calendar Stream Source connector:

[source]
----
$KAFKA_HOME/bin/connect-standalone.sh $KAFKA_HOME/config/connect-standalone.properties $EXAMPLES/google-calendar-stream/google-calendar-stream-source/config/CamelGoogleCalendarStreamSourceConnector.properties
----

On a different terminal run the kafkacat consumer

[source]
----
> ./kafkacat -b localhost:9092 -t mytopic
{"created":"2020-11-09T13:20:12.000Z","creator"...
...,"reminders":{"useDefault":true},"sequence":0,"start":{"dateTime":"2020-11-09T15:00:00.000+01:00"},"status":"confirmed","summary":"Test for ckc","updated":"2020-11-09T13:20:12.380Z"}
% Reached end of topic mytopic [0] at offset 1
----
@@ -0,0 +1,34 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

name=CamelGoogle-calendar-streamSourceConnector
connector.class=org.apache.camel.kafkaconnector.googlecalendarstream.CamelGooglecalendarstreamSourceConnector
key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=org.apache.kafka.connect.storage.StringConverter

topics=mytopic

camel.source.path.index=ckc
camel.source.endpoint.clientId=<client_id>
camel.source.endpoint.calendarId=<calendar_id>
camel.source.endpoint.accessToken=<access_token>
camel.source.endpoint.refreshToken=<refresh_token>
camel.source.endpoint.clientSecret=<client_secret>
camel.source.endpoint.applicationName=ckc
camel.source.endpoint.delay=60000
camel.source.endpoint.consumeFromNow=true
camel.source.endpoint.considerLastUpdate=true

0 comments on commit 5d09365

Please sign in to comment.