Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#76: Add Pulsar source Kamelet #614

Merged
merged 1 commit into from
Dec 9, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 100 additions & 0 deletions docs/modules/ROOT/assets/images/kamelets/pulsar-source.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@
* xref:ROOT:predicate-filter-action.adoc[image:kamelets/predicate-filter-action.svg[] Predicate Filter Action]
* xref:ROOT:protobuf-deserialize-action.adoc[image:kamelets/protobuf-deserialize-action.svg[] Protobuf Deserialize Action]
* xref:ROOT:protobuf-serialize-action.adoc[image:kamelets/protobuf-serialize-action.svg[] Protobuf Serialize Action]
* xref:ROOT:pulsar-source.adoc[image:kamelets/pulsar-source.svg[] Pulsar Source]
* xref:ROOT:rabbitmq-source.adoc[image:kamelets/rabbitmq-source.svg[] RabbitMQ Source]
* xref:ROOT:redis-sink.adoc[image:kamelets/redis-sink.svg[] Redis Sink]
* xref:ROOT:redis-source.adoc[image:kamelets/redis-source.svg[] Redis Source]
Expand Down
175 changes: 175 additions & 0 deletions docs/modules/ROOT/pages/pulsar-source.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
// THIS FILE IS AUTOMATICALLY GENERATED: DO NOT EDIT

= image:kamelets/pulsar-source.svg[] Pulsar Source

*Provided by: "Apache Software Foundation"*

*Support Level for this Kamelet is: "Preview"*

Receive data from Pulsar topics.

== Configuration Options

The following table summarizes the configuration options available for the `pulsar-source` Kamelet:
[width="100%",cols="2,^2,3,^2,^2,^3",options="header"]
|===
| Property| Name| Description| Type| Default| Example
| *namespaceName {empty}* *| Pulsar Namespace Name| The Pulsar Namespace Name| string| |
| *serviceUrl {empty}* *| Service URL| The Pulsar Service URL to point while creating the client from URI.| string| |
| *tenant {empty}* *| Tenant Name| The Tenant Name| string| |
| *topic {empty}* *| Topic Name| The topic name or regexp| string| |
| *topicType {empty}* *| Topic Type| The topic type. Possible values are: persistent or non-persistent| string| |
| authenticationClass| Authentication Class| The Authentication FQCN to be used while creating the client from URI.| string| |
| authenticationParams| Authentication Params| The Authentication Parameters to be used while creating the client from URI.| string| |
| consumerNamePrefix| Consumer Name Prefix| Prefix to add to consumer names when a SHARED or FAILOVER subscription is used| string| `"cons"`|
| consumerQueueSize| Consumer Queue Size| Size of the consumer queue| int| `10`|
| deadLetterTopic| Dead Letter Topic| Name of the topic where the messages which fail maxRedeliverCount times will be sent. Note: if not set, default topic name will be topicName-subscriptionName-DLQ.| int| |
| maxRedeliverCount| Maximum Redelivery Count| Maximum number of times that a message will be redelivered before being sent to the dead letter queue. If this value is not set, no Dead Letter Policy will be created.| int| |
| messageListener| Message Listener| Whether to use the messageListener interface, or to receive messages using a separate thread pool.| boolean| `true`|
| negativeAckRedeliveryDelayMicros| Negative Ack Redelivery Delay in Microseconds| Set the negative acknowledgement delay.| long| `60000000`|
| numberOfConsumerThreads| Number Of Consumer Threads| Number of threads to receive and handle messages when using a separate thread pool.| int| `1`|
| numberOfConsumers| Number Of Consumers| Number of consumers.| int| `1`|
| readCompacted| Read Compacted| Enable compacted topic reading.| boolean| `false`|
| subscriptionInitialPosition| Subscription Initial Position| Control the initial position in the topic of a newly created subscription. Default is latest message.Possible values: EARLIEST or LATEST| string| `"LATEST"`|
| subscriptionName| Subscription Name| Name of the subscription to use.| string| `"subs"`|
| subscriptionTopicsMode| Subscription Topics Mode| Determines to which topics this consumer should be subscribed to - Persistent, Non-Persistent, or both. Only used with pattern subscriptions.Possible values: PersistentOnly, NonPersistentOnly, AllTopics| string| `"PersistentOnly"`|
| subscriptionType| Subscription Type| Type of the subscription. Possible values: EXCLUSIVE, SHARED, FAILOVER, KEY_SHARED| string| `"EXCLUSIVE"`|
| topicsPattern| Topic Pattern| Whether the topic is a pattern (regular expression) that allows the consumer to subscribe to all matching topics in the namespace.| boolean| `false`|
|===

NOTE: Fields marked with an asterisk ({empty}*) are mandatory.


== Dependencies

At runtime, the `pulsar-source` Kamelet relies upon the presence of the following dependencies:

- camel:pulsar
- camel:kamelet
- camel:core

== Usage

This section describes how you can use the `pulsar-source`.

=== Knative Source

You can use the `pulsar-source` Kamelet as a Knative source by binding it to a Knative object.

.pulsar-source-binding.yaml
[source,yaml]
----
apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
name: pulsar-source-binding
spec:
source:
ref:
kind: Kamelet
apiVersion: camel.apache.org/v1alpha1
name: pulsar-source
properties:
namespaceName: "The Pulsar Namespace Name"
serviceUrl: "The Service URL"
tenant: "The Tenant Name"
topic: "The Topic Name"
topicType: "The Topic Type"
sink:
ref:
kind: Channel
apiVersion: messaging.knative.dev/v1
name: mychannel

----

==== *Prerequisite*

You have xref:{camel-k-version}@camel-k::installation/installation.adoc[Camel K installed] on the cluster.

==== *Procedure for using the cluster CLI*

. Save the `pulsar-source-binding.yaml` file to your local drive, and then edit it as needed for your configuration.

. Run the source by using the following command:
+
[source,shell]
----
kubectl apply -f pulsar-source-binding.yaml
----

==== *Procedure for using the Kamel CLI*

Configure and run the source by using the following command:

[source,shell]
----
kamel bind pulsar-source -p "source.namespaceName=The Pulsar Namespace Name" -p "source.serviceUrl=The Service URL" -p "source.tenant=The Tenant Name" -p "source.topic=The Topic Name" -p "source.topicType=The Topic Type" channel:mychannel
----

This command creates the KameletBinding in the current namespace on the cluster.

=== Kafka Source

You can use the `pulsar-source` Kamelet as a Kafka source by binding it to a Kafka topic.

.pulsar-source-binding.yaml
[source,yaml]
----
apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
name: pulsar-source-binding
spec:
source:
ref:
kind: Kamelet
apiVersion: camel.apache.org/v1alpha1
name: pulsar-source
properties:
namespaceName: "The Pulsar Namespace Name"
serviceUrl: "The Service URL"
tenant: "The Tenant Name"
topic: "The Topic Name"
topicType: "The Topic Type"
sink:
ref:
kind: KafkaTopic
apiVersion: kafka.strimzi.io/v1beta1
name: my-topic

----

==== *Prerequisites*

* You've installed https://strimzi.io/[Strimzi].
* You've created a topic named `my-topic` in the current namespace.
* You have xref:{camel-k-version}@camel-k::installation/installation.adoc[Camel K installed] on the cluster.

==== *Procedure for using the cluster CLI*

. Save the `pulsar-source-binding.yaml` file to your local drive, and then edit it as needed for your configuration.

. Run the source by using the following command:
+
[source,shell]
----
kubectl apply -f pulsar-source-binding.yaml
----

==== *Procedure for using the Kamel CLI*

Configure and run the source by using the following command:

[source,shell]
----
kamel bind pulsar-source -p "source.namespaceName=The Pulsar Namespace Name" -p "source.serviceUrl=The Service URL" -p "source.tenant=The Tenant Name" -p "source.topic=The Topic Name" -p "source.topicType=The Topic Type" kafka.strimzi.io/v1beta1:KafkaTopic:my-topic
----

This command creates the KameletBinding in the current namespace on the cluster.

== Kamelet source file

https://github.com/apache/camel-kamelets/blob/main/kamelets/pulsar-source.kamelet.yaml

// THIS FILE IS AUTOMATICALLY GENERATED: DO NOT EDIT
Loading