Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS S3 Streaming Upload Kamelet: Support also StreamingUploadTimeout parameter #505

Merged
merged 5 commits into from
Sep 13, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions aws-s3-streaming-upload-sink.kamelet.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,10 @@ spec:
description: The batch size (in bytes) in streaming upload mode
type: int
default: '1000000'
streamingUploadTimeout:
title: Streaming Upload Timeout
description: While streaming upload mode is true, this option set the timeout to complete upload
type: long
namingStrategy:
title: Naming Strategy
description: The naming strategy to use in streaming upload mode. There are 2 enums and the value can be one of progressive, random
Expand Down Expand Up @@ -104,4 +108,5 @@ spec:
batchSize: "{{batchSize}}"
namingStrategy: "{{namingStrategy}}"
keyName: "{{keyName}}"
streamingUploadTimeout: "{{?streamingUploadTimeout}}"

1 change: 1 addition & 0 deletions docs/modules/ROOT/pages/aws-s3-streaming-upload-sink.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ The following table summarizes the configuration options available for the `aws-
| namingStrategy| Naming Strategy| The naming strategy to use in streaming upload mode. There are 2 enums and the value can be one of progressive, random| string| `"progressive"`|
| restartingPolicy| Restarting Policy| The restarting policy to use in streaming upload mode. There are 2 enums and the value can be one of override, lastPart| string| `"lastPart"`|
| streamingUploadMode| Streaming Upload Mode| Setting the Streaming Upload Mode| boolean| `true`|
| streamingUploadTimeout| Streaming Upload Timeout| While streaming upload mode is true, this option set the timeout to complete upload| long| |
|===

NOTE: Fields marked with ({empty}*) are mandatory.
Expand Down
1 change: 1 addition & 0 deletions docs/modules/ROOT/pages/has-header-filter-action.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ The following table summarizes the configuration options available for the `has-
|===
| Property| Name| Description| Type| Default| Example
| *name {empty}* *| Header Name| The header name to evaluate| string| | `"headerName"`
| value| Header Value| An optional header value to compare the header to| string| | `"headerValue"`
|===

NOTE: Fields marked with ({empty}*) are mandatory.
Expand Down
1 change: 1 addition & 0 deletions docs/modules/ROOT/pages/kafka-not-secured-source.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ The following table summarizes the configuration options available for the `kafk
| allowManualCommit| Allow Manual Commit| Whether to allow doing manual commits| boolean| `false`|
| autoCommitEnable| Auto Commit Enable| If true, periodically commit to ZooKeeper the offset of messages already fetched by the consumer| boolean| `true`|
| autoOffsetReset| Auto Offset Reset| What to do when there is no initial offset. There are 3 enums and the value can be one of latest, earliest, none| string| `"latest"`|
| consumerGroup| Consumer Group| A string that uniquely identifies the group of consumers to which this source belongs| string| | `"my-group-id"`
| pollOnError| Poll On Error Behavior| What to do if kafka threw an exception while polling for new messages. There are 5 enums and the value can be one of DISCARD, ERROR_HANDLER, RECONNECT, RETRY, STOP| string| `"ERROR_HANDLER"`|
|===

Expand Down
1 change: 1 addition & 0 deletions docs/modules/ROOT/pages/kafka-source.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ The following table summarizes the configuration options available for the `kafk
| allowManualCommit| Allow Manual Commit| Whether to allow doing manual commits| boolean| `false`|
| autoCommitEnable| Auto Commit Enable| If true, periodically commit to ZooKeeper the offset of messages already fetched by the consumer| boolean| `true`|
| autoOffsetReset| Auto Offset Reset| What to do when there is no initial offset. There are 3 enums and the value can be one of latest, earliest, none| string| `"latest"`|
| consumerGroup| Consumer Group| A string that uniquely identifies the group of consumers to which this source belongs| string| | `"my-group-id"`
| pollOnError| Poll On Error Behavior| What to do if kafka threw an exception while polling for new messages. There are 5 enums and the value can be one of DISCARD, ERROR_HANDLER, RECONNECT, RETRY, STOP| string| `"ERROR_HANDLER"`|
| saslMechanism| SASL Mechanism| The Simple Authentication and Security Layer (SASL) Mechanism used.| string| `"PLAIN"`|
| securityProtocol| Security Protocol| Protocol used to communicate with brokers. SASL_PLAINTEXT, PLAINTEXT, SASL_SSL and SSL are supported| string| `"SASL_SSL"`|
Expand Down
21 changes: 9 additions & 12 deletions docs/modules/ROOT/pages/rabbitmq-source.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,12 @@ The following table summarizes the configuration options available for the `rabb
[width="100%",cols="2,^2,3,^2,^2,^3",options="header"]
|===
| Property| Name| Description| Type| Default| Example
| *addresses {empty}* *| Addresses| Comma separated list of RabbitMQ broker addresses| string| |
| *addresses {empty}* *| Addresses| Comma separated list of RabbitMQ broker addresses| string| | `"localhost:5672"`
| *exchangeName {empty}* *| Exchange name| The exchange name determines the exchange the queue will be bound to| string| |
| *password {empty}* *| Password| The password to access the RabbitMQ server| string| |
| *username {empty}* *| Username| The username to access the RabbitMQ server| string| |
| portNumber| Port Number| Port of the RabbitMQ server| string| `5672`|
| password| Password| The password to access the RabbitMQ server| string| |
| queue| Queue name| The queue to receive messages from| string| |
| routingKey| Routing Key| The routing key to use when binding a consumer queue to the exchange| string| |
| username| Username| The username to access the RabbitMQ server| string| |
|===

NOTE: Fields marked with ({empty}*) are mandatory.
Expand All @@ -44,10 +45,8 @@ spec:
apiVersion: camel.apache.org/v1alpha1
name: rabbitmq-source
properties:
addresses: "The Addresses"
addresses: "localhost:5672"
exchangeName: "The Exchange name"
password: "The Password"
username: "The Username"
sink:
ref:
kind: InMemoryChannel
Expand All @@ -72,7 +71,7 @@ The procedure described above can be simplified into a single execution of the `

[source,shell]
----
kamel bind rabbitmq-source -p "source.addresses=The Addresses" -p "source.exchangeName=The Exchange name" -p "source.password=The Password" -p "source.username=The Username" channel/mychannel
kamel bind rabbitmq-source -p "source.addresses=localhost:5672" -p "source.exchangeName=The Exchange name" channel/mychannel
----

This will create the KameletBinding under the hood and apply it to the current namespace in the cluster.
Expand All @@ -95,10 +94,8 @@ spec:
apiVersion: camel.apache.org/v1alpha1
name: rabbitmq-source
properties:
addresses: "The Addresses"
addresses: "localhost:5672"
exchangeName: "The Exchange name"
password: "The Password"
username: "The Username"
sink:
ref:
kind: KafkaTopic
Expand All @@ -125,7 +122,7 @@ The procedure described above can be simplified into a single execution of the `

[source,shell]
----
kamel bind rabbitmq-source -p "source.addresses=The Addresses" -p "source.exchangeName=The Exchange name" -p "source.password=The Password" -p "source.username=The Username" kafka.strimzi.io/v1beta1:KafkaTopic:my-topic
kamel bind rabbitmq-source -p "source.addresses=localhost:5672" -p "source.exchangeName=The Exchange name" kafka.strimzi.io/v1beta1:KafkaTopic:my-topic
----

This will create the KameletBinding under the hood and apply it to the current namespace in the cluster.
Expand Down
4 changes: 4 additions & 0 deletions docs/modules/ROOT/pages/telegram-source.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ Receive all messages that people send to your Telegram bot.

To create a bot, contact the @botfather account using the Telegram app.

The source attaches the following headers to the messages:

- `chat-id` / `ce-chat-id`: the ID of the chat where the message comes from

== Configuration Options

The following table summarizes the configuration options available for the `telegram-source` Kamelet:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,10 @@ spec:
description: The batch size (in bytes) in streaming upload mode
type: int
default: '1000000'
streamingUploadTimeout:
title: Streaming Upload Timeout
description: While streaming upload mode is true, this option set the timeout to complete upload
type: long
namingStrategy:
title: Naming Strategy
description: The naming strategy to use in streaming upload mode. There are 2 enums and the value can be one of progressive, random
Expand Down Expand Up @@ -104,4 +108,5 @@ spec:
batchSize: "{{batchSize}}"
namingStrategy: "{{namingStrategy}}"
keyName: "{{keyName}}"
streamingUploadTimeout: "{{?streamingUploadTimeout}}"

4 changes: 1 addition & 3 deletions templates/bindings/camel-k/rabbitmq-source-binding.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,8 @@ spec:
apiVersion: camel.apache.org/v1alpha1
name: rabbitmq-source
properties:
addresses: "The Addresses"
addresses: "localhost:5672"
exchangeName: "The Exchange name"
password: "The Password"
username: "The Username"
sink:
ref:
kind: KafkaTopic
Expand Down
4 changes: 1 addition & 3 deletions templates/bindings/core/rabbitmq-source-binding.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,8 @@
from:
uri: "kamelet:rabbitmq-source"
parameters:
addresses: "The Addresses"
addresses: "localhost:5672"
exchangeName: "The Exchange name"
password: "The Password"
username: "The Username"
steps:
- to:
uri: "log:info"
Expand Down