Skip to content

Commit

Permalink
Include experimental Kamelets in the catalog
Browse files Browse the repository at this point in the history
  • Loading branch information
christophd authored and oscerd committed Dec 2, 2022
1 parent b45bb5d commit b1dec7f
Show file tree
Hide file tree
Showing 18 changed files with 359 additions and 49 deletions.
9 changes: 4 additions & 5 deletions .github/workflows/yaks-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,6 @@ jobs:
# Overwrite JitPack coordinates in the local Kamelets so the tests can use the utility classes in this PR
find kamelets -maxdepth 1 -name '*.kamelet.yaml' -exec sed -i "s/github:apache.camel-kamelets:camel-kamelets-utils:${BASE_REF}-SNAPSHOT/github:${HEAD_REPO/\//.}:camel-kamelets-utils:${HEAD_REF//\//'~'}-SNAPSHOT/g" {} +
find experimental -maxdepth 1 -name '*.kamelet.yaml' -exec sed -i "s/github:apache.camel-kamelets:camel-kamelets-utils:${BASE_REF}-SNAPSHOT/github:${HEAD_REPO/\//.}:camel-kamelets-utils:${HEAD_REF//\//'~'}-SNAPSHOT/g" {} +
- name: Get Camel K CLI
run: |
curl --fail -L --silent https://github.com/apache/camel-k/releases/download/v${CAMEL_K_VERSION}/camel-k-client-${CAMEL_K_VERSION}-linux-64bit.tar.gz -o kamel.tar.gz
Expand Down Expand Up @@ -110,7 +109,7 @@ jobs:
yaks install --operator-image $YAKS_IMAGE_NAME:$YAKS_VERSION
- name: YAKS Tests
run: |
echo "Running tests"
echo "Running tests for Kamelets"
yaks run test/aws-ddb-sink $YAKS_RUN_OPTIONS
yaks run test/aws-s3 $YAKS_RUN_OPTIONS
Expand All @@ -122,11 +121,11 @@ jobs:
yaks run test/earthquake-source $YAKS_RUN_OPTIONS
yaks run test/rest-openapi-sink $YAKS_RUN_OPTIONS
yaks run test/kafka $YAKS_RUN_OPTIONS
- name: YAKS Tests on experimental Kamelets
- name: YAKS Tests experimental Kamelets
run: |
echo "Running tests for experimental Kamelets"
yaks run experimental/test/aws-ddb-sink $YAKS_RUN_OPTIONS
yaks run experimental/test/aws-s3 $YAKS_RUN_OPTIONS
yaks run test/experimental/aws-ddb-sink-exp $YAKS_RUN_OPTIONS
yaks run test/experimental/aws-s3-exp $YAKS_RUN_OPTIONS
- name: YAKS Report
if: failure()
run: |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@
apiVersion: camel.apache.org/v1alpha1
kind: Kamelet
metadata:
name: aws-ddb-sink-experimental
name: aws-ddb-experimental-sink
annotations:
camel.apache.org/kamelet.support.level: "Experiemental"
camel.apache.org/kamelet.support.level: "Experimental"
camel.apache.org/catalog.version: "main-SNAPSHOT"
camel.apache.org/kamelet.icon: "data:image/svg+xml;base64,PHN2ZyBoZWlnaHQ9IjEwMCIgd2lkdGg9IjEwMCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj48cGF0aCBmaWxsPSIjMkQ3MkI4IiBkPSJNNzQuMTc0IDMxLjgwN2w3LjQzNyA1LjM2N3YtNy42MDJsLTcuNDgtOC43NjV2MTAuOTU3bC4wNDMuMDE1eiIvPjxwYXRoIGZpbGw9IiM1Mjk0Q0YiIGQ9Ik01OS44MzggODUuNjY2bDE0LjI5My03LjE0NlYyMC43OTFsLTE0LjMwMy03LjEyNHoiLz48cGF0aCBmaWxsPSIjMjA1Qjk4IiBkPSJNMzkuNDk2IDg1LjY2NkwyNS4yMDMgNzguNTJWMjAuNzkxbDE0LjMwMy03LjEyNHoiLz48cGF0aCBmaWxsPSIjMkQ3MkI4IiBkPSJNMzkuNTA2IDEzLjY2N2gyMC4zMjF2NzEuOTk5SDM5LjUwNnpNNzQuMTMxIDY3LjU2NFY3OC41Mmw3LjQ4LTguNzY0di03LjYwMmwtNy40MzcgNS4zOTd6TTc0LjEzMSA2Mi45MzZsLjA0My0uMDEgNy40MzctNHYtNy42NDlsLTcuNDguNjg4ek03NC4xNzQgMzYuNDI5bC0uMDQzLS4wMVY0Ny4zNWw3LjQ4LjY5OXYtNy42NDV6Ii8+PHBhdGggZmlsbD0iIzFBNDc2RiIgZD0iTTgxLjYxMSA0OC4wNDlsLTcuNDgtLjY5OS0xNC4zMDMtLjU3MkgzOS41MDZsLTE0LjMwMy41NzJWMzYuNDQzbC0uMDE1LjAwOC4wMTUtLjAzMiAxNC4zMDMtMy4zMTRINTkuODI4bDE0LjMwMyAzLjMxNCA1LjI1OCAyLjc5NXYtMS43OTdsMi4yMjItLjI0My03LjQ4LTUuNDEtMTQuMzAzLTQuNDMySDM5LjUwNmwtMTQuMzAzIDQuNDMyVjIwLjgwN2wtNy40OCA4Ljc2M3Y3LjY1M2wuMDU4LS4wNDIgMi4xNjQuMjM2djEuODM0bC0yLjIyMiAxLjE4OXY3LjYxNWwuMDU4LS4wMDYgMi4xNjQuMDMydjMuMTk2bC0xLjg2Ny4wMjgtLjM1NS0uMDM0djcuNjE4bDIuMjIyIDEuMTk1djEuODU1bC0yLjEyOS4yMzUtLjA5My0uMDd2Ny42NTJsNy40OCA4Ljc2NFY2Ny41NjRsMTQuMzAzIDQuNDMySDU5LjgyOGwxNC4zNDUtNC40NDUgNy40MzgtNS4zNjctMi4yMjItLjI0NXYtMS44MThsLTUuMjE2IDIuODA1LTE0LjM0NSAzLjI5NXYuMDA0SDM5LjUwNnYtLjAwNGwtMTQuMzQ4LTMuMjk1LS4wMjUtLjA1MS4wNy4wMzdWNTEuOTY1bDE0LjMwMy41N3YuMDE0SDU5LjgyOHYtLjAxNGwxNC4zMDMtLjU3IDcuNDgtLjY1Ni0yLjIyMi0uMDMydi0zLjE5NnoiLz48L3N2Zz4="
camel.apache.org/provider: "Apache Software Foundation"
Expand All @@ -29,15 +29,17 @@ metadata:
camel.apache.org/kamelet.type: "sink"
spec:
definition:
title: "AWS DynamoDB Sink"
title: "AWS DynamoDB Experimental Sink"
description: |-
Send data to Amazon DynamoDB. The sent data inserts, updates, or deletes an item on the specified AWS DynamoDB table.
The basic authentication method for the AWS DynamoDB service is to specify an access key and a secret key. These parameters are optional because the Kamelet provides a default credentials provider.
If you use the default credentials provider, the DynamoDB client loads the credentials through this provider and doesn't use the basic authentication method.
This Kamelet expects a JSON-formatted body and it must include the primary key values that define the DynamoDB item. The mapping between the JSON fields and table attribute values is done by key. For example, for '{"username":"oscerd", "city":"Rome"}' input, the Kamelet inserts or update an item in the specified AWS DynamoDB table and sets the values for the 'username' and 'city' attributes.
This Kamelet expects a JSON-formatted body and it must include the primary key values that define the DynamoDB item. The mapping between the JSON fields and table attribute values is done by key. For example, for '{"username":"oscerd", "city":"Rome"}' input, the Kamelet inserts or update an item in the specified AWS DynamoDB table and sets the values for the 'username' and 'city' attributes.
This Kamelet supports experimental input format to specify the data type that that is given to this sink. The Kamelet will do best effort to convert the provided input type to the required input for the sink.
required:
- table
- region
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
apiVersion: camel.apache.org/v1alpha1
kind: Kamelet
metadata:
name: aws-s3-source-experimental
name: aws-s3-experimental-source
annotations:
camel.apache.org/kamelet.support.level: "Experimental"
camel.apache.org/catalog.version: "main-SNAPSHOT"
Expand All @@ -12,15 +12,17 @@ metadata:
camel.apache.org/kamelet.type: "source"
spec:
definition:
title: "AWS S3 Source"
title: "AWS S3 Experimental Source"
description: |-
Receive data from an Amazon S3 Bucket.
The basic authentication method for the S3 service is to specify an access key and a secret key. These parameters are optional because the Kamelet provides a default credentials provider.
If you use the default credentials provider, the S3 client loads the credentials through this provider and doesn't use the basic authentication method.
Two headers will be duplicated with different names for clarity at sink level, CamelAwsS3Key will be duplicated into aws.s3.key and CamelAwsS3BucketName will be duplicated in aws.s3.bucket.name
Two headers will be duplicated with different names for clarity at sink level, CamelAwsS3Key will be duplicated into aws.s3.key and CamelAwsS3BucketName will be duplicated in aws.s3.bucket.name.
This Kamelet supports experimental output format to specify the data type produced by this source. Users of the Kamelet are able to choose from different output types.
required:
- bucketNameOrArn
- region
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
# ---------------------------------------------------------------------------
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ---------------------------------------------------------------------------

apiVersion: camel.apache.org/v1alpha1
kind: Kamelet
metadata:
name: aws-ddb-experimental-sink
annotations:
camel.apache.org/kamelet.support.level: "Experimental"
camel.apache.org/catalog.version: "main-SNAPSHOT"
camel.apache.org/kamelet.icon: "data:image/svg+xml;base64,PHN2ZyBoZWlnaHQ9IjEwMCIgd2lkdGg9IjEwMCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj48cGF0aCBmaWxsPSIjMkQ3MkI4IiBkPSJNNzQuMTc0IDMxLjgwN2w3LjQzNyA1LjM2N3YtNy42MDJsLTcuNDgtOC43NjV2MTAuOTU3bC4wNDMuMDE1eiIvPjxwYXRoIGZpbGw9IiM1Mjk0Q0YiIGQ9Ik01OS44MzggODUuNjY2bDE0LjI5My03LjE0NlYyMC43OTFsLTE0LjMwMy03LjEyNHoiLz48cGF0aCBmaWxsPSIjMjA1Qjk4IiBkPSJNMzkuNDk2IDg1LjY2NkwyNS4yMDMgNzguNTJWMjAuNzkxbDE0LjMwMy03LjEyNHoiLz48cGF0aCBmaWxsPSIjMkQ3MkI4IiBkPSJNMzkuNTA2IDEzLjY2N2gyMC4zMjF2NzEuOTk5SDM5LjUwNnpNNzQuMTMxIDY3LjU2NFY3OC41Mmw3LjQ4LTguNzY0di03LjYwMmwtNy40MzcgNS4zOTd6TTc0LjEzMSA2Mi45MzZsLjA0My0uMDEgNy40MzctNHYtNy42NDlsLTcuNDguNjg4ek03NC4xNzQgMzYuNDI5bC0uMDQzLS4wMVY0Ny4zNWw3LjQ4LjY5OXYtNy42NDV6Ii8+PHBhdGggZmlsbD0iIzFBNDc2RiIgZD0iTTgxLjYxMSA0OC4wNDlsLTcuNDgtLjY5OS0xNC4zMDMtLjU3MkgzOS41MDZsLTE0LjMwMy41NzJWMzYuNDQzbC0uMDE1LjAwOC4wMTUtLjAzMiAxNC4zMDMtMy4zMTRINTkuODI4bDE0LjMwMyAzLjMxNCA1LjI1OCAyLjc5NXYtMS43OTdsMi4yMjItLjI0My03LjQ4LTUuNDEtMTQuMzAzLTQuNDMySDM5LjUwNmwtMTQuMzAzIDQuNDMyVjIwLjgwN2wtNy40OCA4Ljc2M3Y3LjY1M2wuMDU4LS4wNDIgMi4xNjQuMjM2djEuODM0bC0yLjIyMiAxLjE4OXY3LjYxNWwuMDU4LS4wMDYgMi4xNjQuMDMydjMuMTk2bC0xLjg2Ny4wMjgtLjM1NS0uMDM0djcuNjE4bDIuMjIyIDEuMTk1djEuODU1bC0yLjEyOS4yMzUtLjA5My0uMDd2Ny42NTJsNy40OCA4Ljc2NFY2Ny41NjRsMTQuMzAzIDQuNDMySDU5LjgyOGwxNC4zNDUtNC40NDUgNy40MzgtNS4zNjctMi4yMjItLjI0NXYtMS44MThsLTUuMjE2IDIuODA1LTE0LjM0NSAzLjI5NXYuMDA0SDM5LjUwNnYtLjAwNGwtMTQuMzQ4LTMuMjk1LS4wMjUtLjA1MS4wNy4wMzdWNTEuOTY1bDE0LjMwMy41N3YuMDE0SDU5LjgyOHYtLjAxNGwxNC4zMDMtLjU3IDcuNDgtLjY1Ni0yLjIyMi0uMDMydi0zLjE5NnoiLz48L3N2Zz4="
camel.apache.org/provider: "Apache Software Foundation"
camel.apache.org/kamelet.group: "AWS DynamoDB Streams"
labels:
camel.apache.org/kamelet.type: "sink"
spec:
definition:
title: "AWS DynamoDB Experimental Sink"
description: |-
Send data to Amazon DynamoDB. The sent data inserts, updates, or deletes an item on the specified AWS DynamoDB table.
The basic authentication method for the AWS DynamoDB service is to specify an access key and a secret key. These parameters are optional because the Kamelet provides a default credentials provider.
If you use the default credentials provider, the DynamoDB client loads the credentials through this provider and doesn't use the basic authentication method.
This Kamelet expects a JSON-formatted body and it must include the primary key values that define the DynamoDB item. The mapping between the JSON fields and table attribute values is done by key. For example, for '{"username":"oscerd", "city":"Rome"}' input, the Kamelet inserts or update an item in the specified AWS DynamoDB table and sets the values for the 'username' and 'city' attributes.
This Kamelet supports experimental input format to specify the data type that that is given to this sink. The Kamelet will do best effort to convert the provided input type to the required input for the sink.
required:
- table
- region
type: object
properties:
table:
title: Table
description: The name of the DynamoDB table.
type: string
accessKey:
title: Access Key
description: The access key obtained from AWS.
type: string
format: password
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:password
- urn:camel:group:credentials
secretKey:
title: Secret Key
description: The secret key obtained from AWS.
type: string
format: password
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:password
- urn:camel:group:credentials
region:
title: AWS Region
description: The AWS region to access.
type: string
enum: ["ap-south-1", "eu-south-1", "us-gov-east-1", "me-central-1", "ca-central-1", "eu-central-1", "us-iso-west-1", "us-west-1", "us-west-2", "af-south-1", "eu-north-1", "eu-west-3", "eu-west-2", "eu-west-1", "ap-northeast-3", "ap-northeast-2", "ap-northeast-1", "me-south-1", "sa-east-1", "ap-east-1", "cn-north-1", "us-gov-west-1", "ap-southeast-1", "ap-southeast-2", "us-iso-east-1", "ap-southeast-3", "us-east-1", "us-east-2", "cn-northwest-1", "us-isob-east-1", "aws-global", "aws-cn-global", "aws-us-gov-global", "aws-iso-global", "aws-iso-b-global"]
operation:
title: Operation
description: "The operation to perform. The options are PutItem, UpdateItem, or DeleteItem."
type: string
default: PutItem
example: PutItem
writeCapacity:
title: Write Capacity
description: The provisioned throughput to reserve for writing resources to your table.
type: integer
default: 1
useDefaultCredentialsProvider:
title: Default Credentials Provider
description: If true, the DynamoDB client loads credentials through a default credentials provider. If false, it uses the basic authentication method (access key and secret key).
type: boolean
x-descriptors:
- 'urn:alm:descriptor:com.tectonic.ui:checkbox'
default: false
uriEndpointOverride:
title: Overwrite Endpoint URI
description: The overriding endpoint URI. To use this option, you must also select the `overrideEndpoint` option.
type: string
overrideEndpoint:
title: Endpoint Overwrite
description: Select this option to override the endpoint URI. To use this option, you must also provide a URI for the `uriEndpointOverride` option.
type: boolean
x-descriptors:
- 'urn:alm:descriptor:com.tectonic.ui:checkbox'
default: false
inputFormat:
title: Input Type
description: Specify the input type for this Kamelet. The Kamelet will automatically apply conversion logic in order to transform message content to this data type.
type: string
default: json
example: json
types:
in:
mediaType: application/json
dependencies:
- github:apache.camel-kamelets:camel-kamelets-utils:main-SNAPSHOT
- "camel:core"
- "camel:jackson"
- "camel:aws2-ddb"
- "camel:kamelet"
template:
beans:
- name: dataTypeRegistry
type: "#class:org.apache.camel.kamelets.utils.format.DefaultDataTypeRegistry"
- name: inputTypeProcessor
type: "#class:org.apache.camel.kamelets.utils.format.DataTypeProcessor"
property:
- key: scheme
value: 'aws2-ddb'
- key: format
value: '{{inputFormat}}'
- key: registry
value: '#bean:{{dataTypeRegistry}}'
from:
uri: "kamelet:source"
steps:
- set-property:
name: operation
constant: "{{operation}}"
- process:
ref: "{{inputTypeProcessor}}"
- to:
uri: "aws2-ddb:{{table}}"
parameters:
secretKey: "{{?secretKey}}"
accessKey: "{{?accessKey}}"
region: "{{region}}"
operation: "{{operation}}"
writeCapacity: "{{?writeCapacity}}"
useDefaultCredentialsProvider: "{{useDefaultCredentialsProvider}}"
uriEndpointOverride: "{{?uriEndpointOverride}}"
overrideEndpoint: "{{overrideEndpoint}}"
Loading

0 comments on commit b1dec7f

Please sign in to comment.