Skip to content

Commit

Permalink
Integration with Amplify Central Marketplace (#158)
Browse files Browse the repository at this point in the history
  • Loading branch information
cremond authored and epiasentin-axway committed Jul 11, 2022
1 parent 80d2f11 commit 2203f15
Show file tree
Hide file tree
Showing 5 changed files with 98 additions and 49 deletions.
48 changes: 19 additions & 29 deletions content/en/docs/install/_index.md
Expand Up @@ -8,7 +8,7 @@ no_list: true
description: Learn how to install Streams on-premise or deploy it in your private cloud, configure a helm chart, and validate the installation.
---

This section covers recommended steps to install Streams either on development environment or production environment.
This section covers recommended steps to install Streams both on development environment and production environment.

## Prerequisites

Expand Down Expand Up @@ -62,36 +62,24 @@ kubectl create namespace "${NAMESPACE}"

## Use Amplify Platform as your Docker registry

Docker images must be hosted in a docker registry accessible from your Kubernetes (K8s) cluster. We recommend you to use the Amplify Platform repository for a custom docker registry. Alternatively, you can use [your own custom Docker registry](/docs/install/customize-install#use-a-custom-docker-registry).
Docker images must be hosted in a docker registry accessible from your Kubernetes (K8s) cluster.
We recommend that you use the Amplify Platform repository for a custom docker registry. Alternatively, you can use [your own custom Docker registry](/docs/install/customize-install#use-a-custom-docker-registry).

To use the Amplify Platform as your container registry you must first ensure the following:
Before you start using Amplify Platform as your Docker registry, ensure the following:

* You can see our images with your organization on the Amplify repository search page.
* You have administrator access to create a service account in your organization.
* Your Streams artifacts are listed on the
* You must be entitled to Streams in your Amplify Central organisation: you should find the Streams artifacts listed on the [repository](https://repository.axway.com/home)
* You must have [a service account in Amplify Central](https://docs.axway.com/bundle/platform-management/page/docs/management_guide/organizations/managing_organizations/index.html#managing-service-accounts) with the **Authentication method** set to **Client Secret**.

After you have verified that your images are loaded and that you have the correct level of access, you must create a service account, then create docker-registry secret with the information from your service account.
### Create Kubernetes secret

### Create a service account

To create your service account, perform the following steps:

1. Log in to the [Amplify Platform](https://platform.axway.com).
2. Select your organization, and from the left menu, click **Service Accounts** (You should see all service accounts already created).
3. Click **+ Service Account**, and fill in the mandatory fields:
* Enter a name for the service account.
* Choose `Client Secret` for the method.
* Choose `Platform-generated secret` for the credentials.
4. Click **Save**
5. Ensure to securely store the generated client secret because it will be required in further steps.

### Create a secret

To create a secret to use with the Amplify platform docker-registry, run the following command with the service account information:
Streams needs the credentials of your service account to pull images from the repository.
Create a secret containing these credentials:

```sh
export NAMESPACE="my-namespace"
export REGISTRY_USERNAME="my-service-account-client-id"
export REGISTRY_PASSWORD="my-service-account-client-secret"
export REGISTRY_USERNAME="<my-service-account-client-id>"
export REGISTRY_PASSWORD="<my-service-account-client-secret>"
export REGISTRY_SERVER="docker.repository.axway.com"

kubectl create secret docker-registry streams-docker-registry-secret --docker-server="${REGISTRY_SERVER}" --docker-username="${REGISTRY_USERNAME}" --docker-password="${REGISTRY_PASSWORD}" -n "${NAMESPACE}"
Expand Down Expand Up @@ -377,6 +365,10 @@ TLS endpoints which Streams services connect to must have a valid TLS certificat

3. Set the [Helm parameter](/docs/install/helm-parameters-reference/) `streams.extraCertificatesSecrets` to your `$SECRET_NAME`. If you have more than one secrets, they must be separated by a comma.

## Integrate with Amplify Central

Streams can connect to [Amplify Central](https://docs.axway.com/bundle/amplify-central/page/docs/index.html) and expose assets to leverage tools like the [Amplify Marketplace](https://docs.axway.com/bundle/amplify-central/page/docs/manage_marketplace/index.html). This integration is disabled by default. For more information on how to enable it, see [Amplify Central Integration](/docs/install/amplify-central-integration).

## Customize your installation

You can specify optional [Helm parameters](/docs/install/helm-parameters-reference/) to customize your installation.
Expand All @@ -398,22 +390,20 @@ export NAMESPACE="my-namespace"
export HELM_RELEASE_NAME="my-release"

helm install "${HELM_RELEASE_NAME}" . \
-f values.yaml \
-n "${NAMESPACE}"
```

### High availability configuration (recommend for production)
### High availability configuration

The following command deploys Streams on the Kubernetes cluster in High availability. This might take a few minutes.
The following command deploys Streams on the Kubernetes cluster in high availability. The deploy might take a few minutes.

{{< alert title="Note" >}}This is recommended for production environments.{{< /alert >}}
{{< alert title="Note" >}}This configuration is recommended for production environments.{{< /alert >}}

```sh
export NAMESPACE="my-namespace"
export HELM_RELEASE_NAME="my-release"

helm install "${HELM_RELEASE_NAME}" . \
-f values.yaml \
-f values-ha.yaml \
-n "${NAMESPACE}"
```
Expand Down
49 changes: 49 additions & 0 deletions content/en/docs/install/amplify-central-integration.md
@@ -0,0 +1,49 @@
---
title: Integrate with Amplify Central Marketplace
linkTitle: Integrate with Amplify Central Marketplace
weight: 15
date: 2022-04-26
description: Connect Streams to Amplify Central to leverage tools like the Amplify Marketplace, where you can expose your Streams assets.
---

Follow this section to integrate Streams with [Amplify Central](https://docs.axway.com/bundle/amplify-central/page/docs/index.html).

## Prerequisites

* You must know your Amplify Central organization ID.
* You must have an environment in which you wish to publish the Streams assets. For more information on how to create a new environment, see [Connect and manage your environment](https://docs.axway.com/bundle/amplify-central/page/docs/connect_manage_environ/index.html).
* You must have [a service account in Amplify Central](https://docs.axway.com/bundle/platform-management/page/docs/management_guide/organizations/managing_organizations/index.html#managing-service-accounts) with the following configuration:
* **Org Roles**: Central Admin
* **Authentication method**: Client Certificate

## Create Kubernetes secret

Streams requires the certificates associated with your service account to authenticate to Amplify Central. The following is an example of how to create a secret containing those certificates:

```sh
export NAMESPACE="my-namespace"
export PRIVATE_KEY_PATH=""
export PUBLIC_KEY_PATH=""

kubectl -n "${NAMESPACE}" create secret generic central-auth-credentials \
--from-file=private_key.pem="${PRIVATE_KEY_PATH}" \
--from-file=public_key.pem="${PUBLIC_KEY_PATH}"
```

## Update your custom Helm values

Add your organization ID, your environment name, and the clientID associated to your service account to your custom Helm values for the installation. For example:

```yml
discoveryAgent:
enabled: true
central:
organizationID: ""
environment: ""
auth:
clientID: ""
```

You Streams installation is now connected to Amplify Central.

You can [proceed with your Streams installation](/docs/install/#amplify-central-integration), or if you have already installed Streams without enabling this integration, you can perform a Helm upgrade instead. If upgrading the Helm chart, ensure to provide the same custom values you used for your original installation and that your Streams Helm chart contains the ``discoveryAgent`` section in its `values.yaml` file.
40 changes: 22 additions & 18 deletions content/en/docs/install/helm-parameters-reference.md
Expand Up @@ -3,10 +3,10 @@ title: Helm parameters reference
linkTitle: Helm parameters reference
weight: 20
date: 2021-02-18
description: Use the following parameters to edit your configuration.
description: Use the following parameters to edit your Streams configuration.
---

This page describe the Helm parameters that you can change to customize your configuration.
Description of the Helm parameters that you can change to customize your Streams configuration.

## MariaDB parameters

Expand All @@ -23,20 +23,20 @@ This page describe the Helm parameters that you can change to customize your con
| `externalizedMariadb.db.name` | Name of the MySQL database used for Streams (Only used when `embeddedMariadb.enabled` set to false) | no | streams |
| `externalizedMariadb.db.user` | Username of the externalized MariaDB used by Streams (Only used when `embeddedMariadb.enabled` set to false) | no | streams |
| `externalizedMariadb.rootUsername` | Root username of the externalized MariaDB used by Streams (Only used when `embeddedMariadb.enabled` set to false) | no | root |
| `externalizedMariadb.tls.enabled` | Externalized MariaDB tls enabled (Only used when `embeddedMariadb.enabled` set to false) | no | true |
| `externalizedMariadb.tls.twoWay` | Externalized MariaDB Two-Way tls enabled (only used when `embeddedMariadb.enabled` set to false) | no | true |
| `externalizedMariadb.tls.enabled` | Externalized MariaDB TLS enabled (Only used when `embeddedMariadb.enabled` set to false) | no | true |
| `externalizedMariadb.tls.twoWay` | Externalized MariaDB Two-Way TLS enabled (only used when `embeddedMariadb.enabled` set to false) | no | true |

## Kafka parameters

| Parameter | Description | Mandatory | Default value |
| --------------------------------------- | ----------------------------------- | --------- | ------------- |
| `embeddedKafka.enabled` | Kafka installed in K8s with the Helm chart. If set to false, the `externalizedKafka` parameter will be used | no | true |
| `embeddedKafka.auth.clientProtocol` | Authentication protocol used by Kafka client (must be "sasl_tls" or "plaintext") | no | sasl_tls |
| `embeddedKafka.auth.interBrokerProtocol` | Authentication protocol internaly used by Kafka broker (must be "sasl_tls" or "plaintext") | no | sasl_tls |
| `embeddedKafka.auth.interBrokerProtocol` | Authentication protocol internally used by Kafka broker (must be "sasl_tls" or "plaintext") | no | sasl_tls |
| `embeddedKafka.metrics.jmx.enabled` | Activate metrics endpoint for Kafka | no | false |
| `externalizedKafka.bootstrapServers` | List of externalized Kafka bootstrap servers used by Streams (only used when `embeddedKafka.enabled` set to false) | no | my.broker.1:port,my.broker.2:port |
| `externalizedKafka.auth.clientUsername` | Username of the externalized Kafka used by Streams (only used when `embeddedKafka.enabled` set to false) | no | streams |
| `externalizedKafka.auth.clientProtocol` | Authentication protocol used by Kafka client (must be "sasl_tls" or "plaintext" ; only used when `embeddedKafka.enabled` set to false)) | no | sasl_tls |
| `externalizedKafka.auth.clientUsername` | Username of the externalized Kafka used by Streams. Only used when `embeddedKafka.enabled` is set to false) | no | streams |
| `externalizedKafka.auth.clientProtocol` | Authentication protocol used by Kafka client (must be "sasl_tls" or "plaintext". Only used when `embeddedKafka.enabled` is set to false)) | no | sasl_tls |

## Zookeeper parameters

Expand Down Expand Up @@ -77,31 +77,37 @@ If you are setting those parameters through `--set` on the command line, you mus
| `images.repository` | Streams Images repository | yes | docker.repository.axway.com/axwaystreams-docker-prod-ptx/2.0 |
| `imagePullSecrets[0].name` | Image registry keys | no | streams-docker-registry-secret |
| `hub.replicaCount` | Hub replica count | no | 1 (2 HA) |
| `hub.service.port` | Http port to reach the Streams Topics API | no | 8080 |
| `hub.service.port` | HTTP port to reach the Streams Topics API | no | 8080 |
| `subscriberSse.enabled` | Enable/Disable Subscriber SSE | no | true |
| `subscriberSse.replicaCount` | Subscriber SSE replica count | no | 1 (2 HA) |
| `subscriberSse.service.port` | Http port to subscribe to a topic | no | 8080 |
| `subscriberSse.service.port` | HTTP port to subscribe to a topic | no | 8080 |
| `subscriberWebhook.enabled` | Enable/Disable Subscriber Webhook | no | true |
| `subscriberWebhook.replicaCount` | Subscriber Webhook replica count | no | 1 (2 HA) |
| `subscriberWebhook.service.port` | Http port to subscribe to a topic | no | 8080 |
| `subscriberWebhook.service.port` | HTTP port to subscribe to a topic | no | 8080 |
| `subscriberWebSocket.enabled` | Enable/Disable Subscriber WebSocket | no | false |
| `subscriberWebSocket.replicaCount` | Subscriber WebSocket replica count | no | 1 (2 HA) |
| `subscriberWebSocket.service.port` | Http port to subscribe to a topic | no | 8080 |
| `subscriberWebSocket.service.port` | HTTP port to subscribe to a topic | no | 8080 |
| `subscriberKafka.enabled` | Enable/Disable Subscriber Kafka | no | false |
| `subscriberKafka.replicaCount` | Subscriber Kafka replica count | no | 1 (2 HA) |
| `subscriberKafka.service.port` | Http port to subscribe to a topic | no | 8080 |
| `subscriberKafka.service.port` | HTTP port to subscribe to a topic | no | 8080 |
| `publisherHttpPoller.enabled` | Enable/Disable Publisher HTTP Poller | no | true |
| `publisherHttpPoller.replicaCount` | Publisher HTTP Poller replica count | no | 1 (2 HA) |
| `publisherHttpPost.enabled` | Enable/Disable Publisher HTTP Post | no | true |
| `publisherHttpPost.replicaCount` | Publisher HTTP Post replica count | no | 1 (2 HA) |
| `publisherHttpPost.service.port` | Http port to publish to a topic | no | 8080 |
| `publisherHttpPost.service.port` | HTTP port to publish to a topic | no | 8080 |
| `publisherKafka.enabled` | Enable/Disable Publisher Kafka | no | false |
| `publisherKafka.replicaCount` | Publisher Kafka replica count | no | 1 (2 HA) |
| `publisherSfdc.enabled` | Enable/Disable Publisher SFDC | no | false |
| `publisherSfdc.replicaCount` | Publisher SFDC replica count | no | 1 (2 HA) |
| `streams.extraCertificatesSecrets` | List of secrets containing TLS certs to add as trusted by Streams | no | [] |
| `actuator.prometheus.enabled` | Activate metrics endpoints for Streams services | no | false |
| `streams.serviceArgs.spring.datasource.hikari.maxLifetime` | Maximum lifetime in milliseconds for a Streams database connection | no | 280000 |
| `discoveryAgent.enabled` | Activate integration with Amplify Central | yes | false |
| `central.organizationID` | Your Amplify Central organization ID | no | N/A |
| `central.environment` | Your Amplify Central environment, as seen in topology | no | N/A |
| `central.url` | Amplify Central URL | no | [https://apicentral.axway.com](https://apicentral.axway.com) |
| `central.auth.clientID` | Client ID in the service account associated with your key pair | no | N/A |
| `central.auth.url` | Amplify Central authentication URL | no | [https://login.axway.com/auth](https://login.axway.com/auth) |

## Monitoring parameters

Expand All @@ -113,12 +119,10 @@ If you are setting those parameters through `--set` on the command line, you mus
| `nginx-ingress-controller.metrics.enabled` | Activate metrics endpoint for Ingress controller | no | false |
| `actuator.prometheus.enabled` | Activate metrics endpoints for Streams services | no | false |

{{< alert title="Note" >}}
To configure a parameter from a dependency chart ([MariaDB](https://github.com/bitnami/charts/tree/master/bitnami/mariadb), [Kafka](https://github.com/bitnami/charts/tree/master/bitnami/kafka), [Zookeeper](https://github.com/bitnami/charts/tree/master/bitnami/zookeeper), or [Nginx](https://github.com/bitnami/charts/tree/master/bitnami/nginx-ingress-controller)), you must add the chart prefix name to the command line argument. For example:
## Configure parameters from a dependency chart

To configure a parameter from a dependency chart, for example, [MariaDB](https://github.com/bitnami/charts/tree/master/bitnami/mariadb), [Kafka](https://github.com/bitnami/charts/tree/master/bitnami/kafka), [Zookeeper](https://github.com/bitnami/charts/tree/master/bitnami/zookeeper), or [Nginx](https://github.com/bitnami/charts/tree/master/bitnami/nginx-ingress-controller), you must add the chart prefix name to the command line argument. For example:

```
--set embeddedMariadb.image.tag=latest --set embeddedKafka.replicaCount=2
```

For more information on the list of parameters, see the dependency chart's documentation.
{{< /alert >}}
2 changes: 1 addition & 1 deletion content/en/docs/install/upgrade.md
Expand Up @@ -23,7 +23,7 @@ Run the following command to upgrade your Streams installation:
export NAMESPACE="my-namespace"
export HELM_RELEASE_NAME="my-release"

helm upgrade "${HELM_RELEASE_NAME}" . -f values.yaml [-f values-ha.yaml] -f my-values.yaml -n "${NAMESPACE}"
helm upgrade "${HELM_RELEASE_NAME}" . [-f values-ha.yaml] -f my-values.yaml -n "${NAMESPACE}"
```

{{< alert title="Caution" color="warning">}}Any difference in any of the `values.yaml` files or in the `--set` parameter from the initial installation will also be upgraded. Therefore, if you initially installed Streams with `-f values.yaml` or `-f values-ha.yaml`, you must specify the same parameters for the upgrade.
Expand Down
8 changes: 7 additions & 1 deletion content/en/docs/relnotes/20220530_streams_relnotes.md
Expand Up @@ -11,7 +11,13 @@ Streams is available as a set of Docker containers deployable in Kubernetes by u

## New features and enhancements

The following new features and enhancements are available in this update:
The following new features and enhancements are available in this update.

### Connect to Amplify Central to use Amplify Marketplace

Now you can connect Streams to [Amplify Central](https://docs.axway.com/bundle/amplify-central/page/docs/index.html) to leverage tools like the [Amplify Marketplace](https://docs.axway.com/bundle/amplify-central/page/docs/manage_marketplace/index.html), where you can expose your Streams assets.

For more information on how to set up this integration, see [Integrate with Amplify Central Marketplace](/docs/install/amplify-central-integration).

### Kafka subscription configured with SASL and SSL

Expand Down

0 comments on commit 2203f15

Please sign in to comment.