From 2203f1559f95548bf5e45b7333b5c1f943415a9a Mon Sep 17 00:00:00 2001 From: cremond <76732557+cremond@users.noreply.github.com> Date: Wed, 11 May 2022 16:30:50 +0200 Subject: [PATCH] Integration with Amplify Central Marketplace (#158) --- content/en/docs/install/_index.md | 48 +++++++----------- .../install/amplify-central-integration.md | 49 +++++++++++++++++++ .../docs/install/helm-parameters-reference.md | 40 ++++++++------- content/en/docs/install/upgrade.md | 2 +- .../relnotes/20220530_streams_relnotes.md | 8 ++- 5 files changed, 98 insertions(+), 49 deletions(-) create mode 100755 content/en/docs/install/amplify-central-integration.md diff --git a/content/en/docs/install/_index.md b/content/en/docs/install/_index.md index 999daeb3..4fb268e6 100644 --- a/content/en/docs/install/_index.md +++ b/content/en/docs/install/_index.md @@ -8,7 +8,7 @@ no_list: true description: Learn how to install Streams on-premise or deploy it in your private cloud, configure a helm chart, and validate the installation. --- -This section covers recommended steps to install Streams either on development environment or production environment. +This section covers recommended steps to install Streams both on development environment and production environment. ## Prerequisites @@ -62,36 +62,24 @@ kubectl create namespace "${NAMESPACE}" ## Use Amplify Platform as your Docker registry -Docker images must be hosted in a docker registry accessible from your Kubernetes (K8s) cluster. We recommend you to use the Amplify Platform repository for a custom docker registry. Alternatively, you can use [your own custom Docker registry](/docs/install/customize-install#use-a-custom-docker-registry). +Docker images must be hosted in a docker registry accessible from your Kubernetes (K8s) cluster. +We recommend that you use the Amplify Platform repository for a custom docker registry. Alternatively, you can use [your own custom Docker registry](/docs/install/customize-install#use-a-custom-docker-registry). -To use the Amplify Platform as your container registry you must first ensure the following: +Before you start using Amplify Platform as your Docker registry, ensure the following: -* You can see our images with your organization on the Amplify repository search page. -* You have administrator access to create a service account in your organization. +* Your Streams artifacts are listed on the +* You must be entitled to Streams in your Amplify Central organisation: you should find the Streams artifacts listed on the [repository](https://repository.axway.com/home) +* You must have [a service account in Amplify Central](https://docs.axway.com/bundle/platform-management/page/docs/management_guide/organizations/managing_organizations/index.html#managing-service-accounts) with the **Authentication method** set to **Client Secret**. -After you have verified that your images are loaded and that you have the correct level of access, you must create a service account, then create docker-registry secret with the information from your service account. +### Create Kubernetes secret -### Create a service account - -To create your service account, perform the following steps: - -1. Log in to the [Amplify Platform](https://platform.axway.com). -2. Select your organization, and from the left menu, click **Service Accounts** (You should see all service accounts already created). -3. Click **+ Service Account**, and fill in the mandatory fields: - * Enter a name for the service account. - * Choose `Client Secret` for the method. - * Choose `Platform-generated secret` for the credentials. -4. Click **Save** -5. Ensure to securely store the generated client secret because it will be required in further steps. - -### Create a secret - -To create a secret to use with the Amplify platform docker-registry, run the following command with the service account information: +Streams needs the credentials of your service account to pull images from the repository. +Create a secret containing these credentials: ```sh export NAMESPACE="my-namespace" -export REGISTRY_USERNAME="my-service-account-client-id" -export REGISTRY_PASSWORD="my-service-account-client-secret" +export REGISTRY_USERNAME="" +export REGISTRY_PASSWORD="" export REGISTRY_SERVER="docker.repository.axway.com" kubectl create secret docker-registry streams-docker-registry-secret --docker-server="${REGISTRY_SERVER}" --docker-username="${REGISTRY_USERNAME}" --docker-password="${REGISTRY_PASSWORD}" -n "${NAMESPACE}" @@ -377,6 +365,10 @@ TLS endpoints which Streams services connect to must have a valid TLS certificat 3. Set the [Helm parameter](/docs/install/helm-parameters-reference/) `streams.extraCertificatesSecrets` to your `$SECRET_NAME`. If you have more than one secrets, they must be separated by a comma. +## Integrate with Amplify Central + +Streams can connect to [Amplify Central](https://docs.axway.com/bundle/amplify-central/page/docs/index.html) and expose assets to leverage tools like the [Amplify Marketplace](https://docs.axway.com/bundle/amplify-central/page/docs/manage_marketplace/index.html). This integration is disabled by default. For more information on how to enable it, see [Amplify Central Integration](/docs/install/amplify-central-integration). + ## Customize your installation You can specify optional [Helm parameters](/docs/install/helm-parameters-reference/) to customize your installation. @@ -398,22 +390,20 @@ export NAMESPACE="my-namespace" export HELM_RELEASE_NAME="my-release" helm install "${HELM_RELEASE_NAME}" . \ - -f values.yaml \ -n "${NAMESPACE}" ``` -### High availability configuration (recommend for production) +### High availability configuration -The following command deploys Streams on the Kubernetes cluster in High availability. This might take a few minutes. +The following command deploys Streams on the Kubernetes cluster in high availability. The deploy might take a few minutes. -{{< alert title="Note" >}}This is recommended for production environments.{{< /alert >}} +{{< alert title="Note" >}}This configuration is recommended for production environments.{{< /alert >}} ```sh export NAMESPACE="my-namespace" export HELM_RELEASE_NAME="my-release" helm install "${HELM_RELEASE_NAME}" . \ - -f values.yaml \ -f values-ha.yaml \ -n "${NAMESPACE}" ``` diff --git a/content/en/docs/install/amplify-central-integration.md b/content/en/docs/install/amplify-central-integration.md new file mode 100755 index 00000000..8632506c --- /dev/null +++ b/content/en/docs/install/amplify-central-integration.md @@ -0,0 +1,49 @@ +--- +title: Integrate with Amplify Central Marketplace +linkTitle: Integrate with Amplify Central Marketplace +weight: 15 +date: 2022-04-26 +description: Connect Streams to Amplify Central to leverage tools like the Amplify Marketplace, where you can expose your Streams assets. +--- + +Follow this section to integrate Streams with [Amplify Central](https://docs.axway.com/bundle/amplify-central/page/docs/index.html). + +## Prerequisites + +* You must know your Amplify Central organization ID. +* You must have an environment in which you wish to publish the Streams assets. For more information on how to create a new environment, see [Connect and manage your environment](https://docs.axway.com/bundle/amplify-central/page/docs/connect_manage_environ/index.html). +* You must have [a service account in Amplify Central](https://docs.axway.com/bundle/platform-management/page/docs/management_guide/organizations/managing_organizations/index.html#managing-service-accounts) with the following configuration: + * **Org Roles**: Central Admin + * **Authentication method**: Client Certificate + +## Create Kubernetes secret + +Streams requires the certificates associated with your service account to authenticate to Amplify Central. The following is an example of how to create a secret containing those certificates: + +```sh +export NAMESPACE="my-namespace" +export PRIVATE_KEY_PATH="" +export PUBLIC_KEY_PATH="" + +kubectl -n "${NAMESPACE}" create secret generic central-auth-credentials \ + --from-file=private_key.pem="${PRIVATE_KEY_PATH}" \ + --from-file=public_key.pem="${PUBLIC_KEY_PATH}" +``` + +## Update your custom Helm values + +Add your organization ID, your environment name, and the clientID associated to your service account to your custom Helm values for the installation. For example: + +```yml +discoveryAgent: + enabled: true +central: + organizationID: "" + environment: "" + auth: + clientID: "" +``` + +You Streams installation is now connected to Amplify Central. + +You can [proceed with your Streams installation](/docs/install/#amplify-central-integration), or if you have already installed Streams without enabling this integration, you can perform a Helm upgrade instead. If upgrading the Helm chart, ensure to provide the same custom values you used for your original installation and that your Streams Helm chart contains the ``discoveryAgent`` section in its `values.yaml` file. diff --git a/content/en/docs/install/helm-parameters-reference.md b/content/en/docs/install/helm-parameters-reference.md index 3208b690..4c041cc7 100644 --- a/content/en/docs/install/helm-parameters-reference.md +++ b/content/en/docs/install/helm-parameters-reference.md @@ -3,10 +3,10 @@ title: Helm parameters reference linkTitle: Helm parameters reference weight: 20 date: 2021-02-18 -description: Use the following parameters to edit your configuration. +description: Use the following parameters to edit your Streams configuration. --- -This page describe the Helm parameters that you can change to customize your configuration. +Description of the Helm parameters that you can change to customize your Streams configuration. ## MariaDB parameters @@ -23,8 +23,8 @@ This page describe the Helm parameters that you can change to customize your con | `externalizedMariadb.db.name` | Name of the MySQL database used for Streams (Only used when `embeddedMariadb.enabled` set to false) | no | streams | | `externalizedMariadb.db.user` | Username of the externalized MariaDB used by Streams (Only used when `embeddedMariadb.enabled` set to false) | no | streams | | `externalizedMariadb.rootUsername` | Root username of the externalized MariaDB used by Streams (Only used when `embeddedMariadb.enabled` set to false) | no | root | -| `externalizedMariadb.tls.enabled` | Externalized MariaDB tls enabled (Only used when `embeddedMariadb.enabled` set to false) | no | true | -| `externalizedMariadb.tls.twoWay` | Externalized MariaDB Two-Way tls enabled (only used when `embeddedMariadb.enabled` set to false) | no | true | +| `externalizedMariadb.tls.enabled` | Externalized MariaDB TLS enabled (Only used when `embeddedMariadb.enabled` set to false) | no | true | +| `externalizedMariadb.tls.twoWay` | Externalized MariaDB Two-Way TLS enabled (only used when `embeddedMariadb.enabled` set to false) | no | true | ## Kafka parameters @@ -32,11 +32,11 @@ This page describe the Helm parameters that you can change to customize your con | --------------------------------------- | ----------------------------------- | --------- | ------------- | | `embeddedKafka.enabled` | Kafka installed in K8s with the Helm chart. If set to false, the `externalizedKafka` parameter will be used | no | true | | `embeddedKafka.auth.clientProtocol` | Authentication protocol used by Kafka client (must be "sasl_tls" or "plaintext") | no | sasl_tls | -| `embeddedKafka.auth.interBrokerProtocol` | Authentication protocol internaly used by Kafka broker (must be "sasl_tls" or "plaintext") | no | sasl_tls | +| `embeddedKafka.auth.interBrokerProtocol` | Authentication protocol internally used by Kafka broker (must be "sasl_tls" or "plaintext") | no | sasl_tls | | `embeddedKafka.metrics.jmx.enabled` | Activate metrics endpoint for Kafka | no | false | | `externalizedKafka.bootstrapServers` | List of externalized Kafka bootstrap servers used by Streams (only used when `embeddedKafka.enabled` set to false) | no | my.broker.1:port,my.broker.2:port | -| `externalizedKafka.auth.clientUsername` | Username of the externalized Kafka used by Streams (only used when `embeddedKafka.enabled` set to false) | no | streams | -| `externalizedKafka.auth.clientProtocol` | Authentication protocol used by Kafka client (must be "sasl_tls" or "plaintext" ; only used when `embeddedKafka.enabled` set to false)) | no | sasl_tls | +| `externalizedKafka.auth.clientUsername` | Username of the externalized Kafka used by Streams. Only used when `embeddedKafka.enabled` is set to false) | no | streams | +| `externalizedKafka.auth.clientProtocol` | Authentication protocol used by Kafka client (must be "sasl_tls" or "plaintext". Only used when `embeddedKafka.enabled` is set to false)) | no | sasl_tls | ## Zookeeper parameters @@ -77,24 +77,24 @@ If you are setting those parameters through `--set` on the command line, you mus | `images.repository` | Streams Images repository | yes | docker.repository.axway.com/axwaystreams-docker-prod-ptx/2.0 | | `imagePullSecrets[0].name` | Image registry keys | no | streams-docker-registry-secret | | `hub.replicaCount` | Hub replica count | no | 1 (2 HA) | -| `hub.service.port` | Http port to reach the Streams Topics API | no | 8080 | +| `hub.service.port` | HTTP port to reach the Streams Topics API | no | 8080 | | `subscriberSse.enabled` | Enable/Disable Subscriber SSE | no | true | | `subscriberSse.replicaCount` | Subscriber SSE replica count | no | 1 (2 HA) | -| `subscriberSse.service.port` | Http port to subscribe to a topic | no | 8080 | +| `subscriberSse.service.port` | HTTP port to subscribe to a topic | no | 8080 | | `subscriberWebhook.enabled` | Enable/Disable Subscriber Webhook | no | true | | `subscriberWebhook.replicaCount` | Subscriber Webhook replica count | no | 1 (2 HA) | -| `subscriberWebhook.service.port` | Http port to subscribe to a topic | no | 8080 | +| `subscriberWebhook.service.port` | HTTP port to subscribe to a topic | no | 8080 | | `subscriberWebSocket.enabled` | Enable/Disable Subscriber WebSocket | no | false | | `subscriberWebSocket.replicaCount` | Subscriber WebSocket replica count | no | 1 (2 HA) | -| `subscriberWebSocket.service.port` | Http port to subscribe to a topic | no | 8080 | +| `subscriberWebSocket.service.port` | HTTP port to subscribe to a topic | no | 8080 | | `subscriberKafka.enabled` | Enable/Disable Subscriber Kafka | no | false | | `subscriberKafka.replicaCount` | Subscriber Kafka replica count | no | 1 (2 HA) | -| `subscriberKafka.service.port` | Http port to subscribe to a topic | no | 8080 | +| `subscriberKafka.service.port` | HTTP port to subscribe to a topic | no | 8080 | | `publisherHttpPoller.enabled` | Enable/Disable Publisher HTTP Poller | no | true | | `publisherHttpPoller.replicaCount` | Publisher HTTP Poller replica count | no | 1 (2 HA) | | `publisherHttpPost.enabled` | Enable/Disable Publisher HTTP Post | no | true | | `publisherHttpPost.replicaCount` | Publisher HTTP Post replica count | no | 1 (2 HA) | -| `publisherHttpPost.service.port` | Http port to publish to a topic | no | 8080 | +| `publisherHttpPost.service.port` | HTTP port to publish to a topic | no | 8080 | | `publisherKafka.enabled` | Enable/Disable Publisher Kafka | no | false | | `publisherKafka.replicaCount` | Publisher Kafka replica count | no | 1 (2 HA) | | `publisherSfdc.enabled` | Enable/Disable Publisher SFDC | no | false | @@ -102,6 +102,12 @@ If you are setting those parameters through `--set` on the command line, you mus | `streams.extraCertificatesSecrets` | List of secrets containing TLS certs to add as trusted by Streams | no | [] | | `actuator.prometheus.enabled` | Activate metrics endpoints for Streams services | no | false | | `streams.serviceArgs.spring.datasource.hikari.maxLifetime` | Maximum lifetime in milliseconds for a Streams database connection | no | 280000 | +| `discoveryAgent.enabled` | Activate integration with Amplify Central | yes | false | +| `central.organizationID` | Your Amplify Central organization ID | no | N/A | +| `central.environment` | Your Amplify Central environment, as seen in topology | no | N/A | +| `central.url` | Amplify Central URL | no | [https://apicentral.axway.com](https://apicentral.axway.com) | +| `central.auth.clientID` | Client ID in the service account associated with your key pair | no | N/A | +| `central.auth.url` | Amplify Central authentication URL | no | [https://login.axway.com/auth](https://login.axway.com/auth) | ## Monitoring parameters @@ -113,12 +119,10 @@ If you are setting those parameters through `--set` on the command line, you mus | `nginx-ingress-controller.metrics.enabled` | Activate metrics endpoint for Ingress controller | no | false | | `actuator.prometheus.enabled` | Activate metrics endpoints for Streams services | no | false | -{{< alert title="Note" >}} -To configure a parameter from a dependency chart ([MariaDB](https://github.com/bitnami/charts/tree/master/bitnami/mariadb), [Kafka](https://github.com/bitnami/charts/tree/master/bitnami/kafka), [Zookeeper](https://github.com/bitnami/charts/tree/master/bitnami/zookeeper), or [Nginx](https://github.com/bitnami/charts/tree/master/bitnami/nginx-ingress-controller)), you must add the chart prefix name to the command line argument. For example: +## Configure parameters from a dependency chart + +To configure a parameter from a dependency chart, for example, [MariaDB](https://github.com/bitnami/charts/tree/master/bitnami/mariadb), [Kafka](https://github.com/bitnami/charts/tree/master/bitnami/kafka), [Zookeeper](https://github.com/bitnami/charts/tree/master/bitnami/zookeeper), or [Nginx](https://github.com/bitnami/charts/tree/master/bitnami/nginx-ingress-controller), you must add the chart prefix name to the command line argument. For example: ``` --set embeddedMariadb.image.tag=latest --set embeddedKafka.replicaCount=2 ``` - -For more information on the list of parameters, see the dependency chart's documentation. -{{< /alert >}} diff --git a/content/en/docs/install/upgrade.md b/content/en/docs/install/upgrade.md index 8d822e15..77340c50 100644 --- a/content/en/docs/install/upgrade.md +++ b/content/en/docs/install/upgrade.md @@ -23,7 +23,7 @@ Run the following command to upgrade your Streams installation: export NAMESPACE="my-namespace" export HELM_RELEASE_NAME="my-release" -helm upgrade "${HELM_RELEASE_NAME}" . -f values.yaml [-f values-ha.yaml] -f my-values.yaml -n "${NAMESPACE}" +helm upgrade "${HELM_RELEASE_NAME}" . [-f values-ha.yaml] -f my-values.yaml -n "${NAMESPACE}" ``` {{< alert title="Caution" color="warning">}}Any difference in any of the `values.yaml` files or in the `--set` parameter from the initial installation will also be upgraded. Therefore, if you initially installed Streams with `-f values.yaml` or `-f values-ha.yaml`, you must specify the same parameters for the upgrade. diff --git a/content/en/docs/relnotes/20220530_streams_relnotes.md b/content/en/docs/relnotes/20220530_streams_relnotes.md index cccf16cd..c0fa3837 100644 --- a/content/en/docs/relnotes/20220530_streams_relnotes.md +++ b/content/en/docs/relnotes/20220530_streams_relnotes.md @@ -11,7 +11,13 @@ Streams is available as a set of Docker containers deployable in Kubernetes by u ## New features and enhancements -The following new features and enhancements are available in this update: +The following new features and enhancements are available in this update. + +### Connect to Amplify Central to use Amplify Marketplace + +Now you can connect Streams to [Amplify Central](https://docs.axway.com/bundle/amplify-central/page/docs/index.html) to leverage tools like the [Amplify Marketplace](https://docs.axway.com/bundle/amplify-central/page/docs/manage_marketplace/index.html), where you can expose your Streams assets. + +For more information on how to set up this integration, see [Integrate with Amplify Central Marketplace](/docs/install/amplify-central-integration). ### Kafka subscription configured with SASL and SSL