Skip to content

Commit

Permalink
apache#76: Add Pulsar source Kamelet
Browse files Browse the repository at this point in the history
  • Loading branch information
luigidemasi committed Dec 7, 2021
1 parent f301894 commit 65c77d1
Show file tree
Hide file tree
Showing 7 changed files with 649 additions and 0 deletions.
100 changes: 100 additions & 0 deletions docs/modules/ROOT/assets/images/kamelets/pulsar-source.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@
* xref:ROOT:predicate-filter-action.adoc[image:kamelets/predicate-filter-action.svg[] Predicate Filter Action]
* xref:ROOT:protobuf-deserialize-action.adoc[image:kamelets/protobuf-deserialize-action.svg[] Protobuf Deserialize Action]
* xref:ROOT:protobuf-serialize-action.adoc[image:kamelets/protobuf-serialize-action.svg[] Protobuf Serialize Action]
* xref:ROOT:pulsar-source.adoc[image:kamelets/pulsar-source.svg[] Pulsar Source]
* xref:ROOT:rabbitmq-source.adoc[image:kamelets/rabbitmq-source.svg[] RabbitMQ Source]
* xref:ROOT:redis-sink.adoc[image:kamelets/redis-sink.svg[] Redis Sink]
* xref:ROOT:redis-source.adoc[image:kamelets/redis-source.svg[] Redis Source]
Expand Down
175 changes: 175 additions & 0 deletions docs/modules/ROOT/pages/pulsar-source.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
// THIS FILE IS AUTOMATICALLY GENERATED: DO NOT EDIT

= image:kamelets/pulsar-source.svg[] Pulsar Source

*Provided by: "Apache Software Foundation"*

*Support Level for this Kamelet is: "Preview"*

Receive data from Pulsar topics.

== Configuration Options

The following table summarizes the configuration options available for the `pulsar-source` Kamelet:
[width="100%",cols="2,^2,3,^2,^2,^3",options="header"]
|===
| Property| Name| Description| Type| Default| Example
| *namespaceName {empty}* *| Pulsar Namespace Name| The Pulsar Namespace Name| string| |
| *serviceUrl {empty}* *| Service URL| The Pulsar Service URL to point while creating the client from URI.| string| |
| *tenant {empty}* *| Tenant Name| The Tenant Name| string| |
| *topic {empty}* *| Topic Name| The topic name or regexp| string| |
| *topicType {empty}* *| Topic Type| The topic type. Possible values are: persistent or non-persistent| string| |
| authenticationClass| Authentication Class| The Authentication FQCN to be used while creating the client from URI.| string| |
| authenticationParams| Authentication Params| The Authentication Parameters to be used while creating the client from URI.| string| |
| consumerNamePrefix| Consumer Name Prefix| Prefix to add to consumer names when a SHARED or FAILOVER subscription is used| string| `"cons"`|
| consumerQueueSize| Consumer Queue Size| Size of the consumer queue| int| `10`|
| deadLetterTopic| Dead Letter Topic| Name of the topic where the messages which fail maxRedeliverCount times will be sent. Note: if not set, default topic name will be topicName-subscriptionName-DLQ.| int| |
| maxRedeliverCount| Maximum Redelivery Count| Maximum number of times that a message will be redelivered before being sent to the dead letter queue. If this value is not set, no Dead Letter Policy will be created.| int| |
| messageListener| Message Listener| Whether to use the messageListener interface, or to receive messages using a separate thread pool.| boolean| `true`|
| negativeAckRedeliveryDelayMicros| Negative Ack Redelivery Delay in Microseconds| Set the negative acknowledgement delay.| long| `60000000`|
| numberOfConsumerThreads| Number Of Consumer Threads| Number of threads to receive and handle messages when using a separate thread pool.| int| `1`|
| numberOfConsumers| Number Of Consumers| Number of consumers.| int| `1`|
| readCompacted| Read Compacted| Enable compacted topic reading.| boolean| `false`|
| subscriptionInitialPosition| Subscription Initial Position| Control the initial position in the topic of a newly created subscription. Default is latest message.Possible values: EARLIEST or LATEST| string| `"LATEST"`|
| subscriptionName| Subscription Name| Name of the subscription to use.| string| `"subs"`|
| subscriptionTopicsMode| Subscription Topics Mode| Determines to which topics this consumer should be subscribed to - Persistent, Non-Persistent, or both. Only used with pattern subscriptions.Possible values: PersistentOnly, NonPersistentOnly, AllTopics| string| `"PersistentOnly"`|
| subscriptionType| Subscription Type| Type of the subscription. Possible values: EXCLUSIVE, SHARED, FAILOVER, KEY_SHARED| string| `"EXCLUSIVE"`|
| topicsPattern| Topic Pattern| Whether the topic is a pattern (regular expression) that allows the consumer to subscribe to all matching topics in the namespace.| boolean| `false`|
|===

NOTE: Fields marked with an asterisk ({empty}*) are mandatory.


== Dependencies

At runtime, the `pulsar-source` Kamelet relies upon the presence of the following dependencies:

- camel:pulsar
- camel:kamelet
- camel:core

== Usage

This section describes how you can use the `pulsar-source`.

=== Knative Source

You can use the `pulsar-source` Kamelet as a Knative source by binding it to a Knative object.

.pulsar-source-binding.yaml
[source,yaml]
----
apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
name: pulsar-source-binding
spec:
source:
ref:
kind: Kamelet
apiVersion: camel.apache.org/v1alpha1
name: pulsar-source
properties:
namespaceName: "The Pulsar Namespace Name"
serviceUrl: "The Service URL"
tenant: "The Tenant Name"
topic: "The Topic Name"
topicType: "The Topic Type"
sink:
ref:
kind: Channel
apiVersion: messaging.knative.dev/v1
name: mychannel
----

==== *Prerequisite*

You have xref:{camel-k-version}@camel-k::installation/installation.adoc[Camel K installed] on the cluster.

==== *Procedure for using the cluster CLI*

. Save the `pulsar-source-binding.yaml` file to your local drive, and then edit it as needed for your configuration.

. Run the source by using the following command:
+
[source,shell]
----
kubectl apply -f pulsar-source-binding.yaml
----

==== *Procedure for using the Kamel CLI*

Configure and run the source by using the following command:

[source,shell]
----
kamel bind pulsar-source -p "source.namespaceName=The Pulsar Namespace Name" -p "source.serviceUrl=The Service URL" -p "source.tenant=The Tenant Name" -p "source.topic=The Topic Name" -p "source.topicType=The Topic Type" channel:mychannel
----

This command creates the KameletBinding in the current namespace on the cluster.

=== Kafka Source

You can use the `pulsar-source` Kamelet as a Kafka source by binding it to a Kafka topic.

.pulsar-source-binding.yaml
[source,yaml]
----
apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
name: pulsar-source-binding
spec:
source:
ref:
kind: Kamelet
apiVersion: camel.apache.org/v1alpha1
name: pulsar-source
properties:
namespaceName: "The Pulsar Namespace Name"
serviceUrl: "The Service URL"
tenant: "The Tenant Name"
topic: "The Topic Name"
topicType: "The Topic Type"
sink:
ref:
kind: KafkaTopic
apiVersion: kafka.strimzi.io/v1beta1
name: my-topic
----

==== *Prerequisites*

* You've installed https://strimzi.io/[Strimzi].
* You've created a topic named `my-topic` in the current namespace.
* You have xref:{camel-k-version}@camel-k::installation/installation.adoc[Camel K installed] on the cluster.

==== *Procedure for using the cluster CLI*

. Save the `pulsar-source-binding.yaml` file to your local drive, and then edit it as needed for your configuration.

. Run the source by using the following command:
+
[source,shell]
----
kubectl apply -f pulsar-source-binding.yaml
----

==== *Procedure for using the Kamel CLI*

Configure and run the source by using the following command:

[source,shell]
----
kamel bind pulsar-source -p "source.namespaceName=The Pulsar Namespace Name" -p "source.serviceUrl=The Service URL" -p "source.tenant=The Tenant Name" -p "source.topic=The Topic Name" -p "source.topicType=The Topic Type" kafka.strimzi.io/v1beta1:KafkaTopic:my-topic
----

This command creates the KameletBinding in the current namespace on the cluster.

== Kamelet source file

https://github.com/apache/camel-kamelets/blob/main/kamelets/pulsar-source.kamelet.yaml

// THIS FILE IS AUTOMATICALLY GENERATED: DO NOT EDIT
Loading

0 comments on commit 65c77d1

Please sign in to comment.