From b922d80d623adad72954313a18799c639922dd24 Mon Sep 17 00:00:00 2001 From: dervoeti Date: Fri, 1 Sep 2023 10:21:20 +0200 Subject: [PATCH 01/52] Verify all Stackable images instead of just 23.7 operator images --- .../examples/verify-signatures/kyverno-policy.yaml | 6 +++--- .../enabling_verification_of_image_signatures.adoc | 10 ++++++---- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/modules/tutorials/examples/verify-signatures/kyverno-policy.yaml b/modules/tutorials/examples/verify-signatures/kyverno-policy.yaml index 808371e45..4c00ca261 100644 --- a/modules/tutorials/examples/verify-signatures/kyverno-policy.yaml +++ b/modules/tutorials/examples/verify-signatures/kyverno-policy.yaml @@ -16,11 +16,11 @@ spec: - Pod verifyImages: - imageReferences: - - docker.stackable.tech/*-operator:23.7.* + - docker.stackable.tech/* attestors: - entries: - keyless: issuer: "https://token.actions.githubusercontent.com" - subject: "https://github.com/stackabletech/*-operator/.github/workflows/build.yml@refs/tags/23.7.*" + subject: "https://github.com/stackabletech/*/.github/workflows/build.yml@refs/*" rekor: - url: https://rekor.sigstore.dev \ No newline at end of file + url: https://rekor.sigstore.dev diff --git a/modules/tutorials/pages/enabling_verification_of_image_signatures.adoc b/modules/tutorials/pages/enabling_verification_of_image_signatures.adoc index 9bf93ae2a..695128865 100644 --- a/modules/tutorials/pages/enabling_verification_of_image_signatures.adoc +++ b/modules/tutorials/pages/enabling_verification_of_image_signatures.adoc @@ -1,8 +1,10 @@ = Enabling verification of image signatures -Image signing is a security measure that helps ensure the authenticity and integrity of container images. Starting with SDP 23.7, all our operator images are signed https://docs.sigstore.dev/cosign/openid_signing/["keyless"] (signing of product images and Helm charts will follow). By verifying these signatures, cluster administrators can ensure that the operator images pulled from Stackable's container registry are authentic and have not been tampered with. +Image signing is a security measure that helps ensure the authenticity and integrity of container images. Starting with SDP 23.7, all our images are signed https://docs.sigstore.dev/cosign/openid_signing/["keyless"]. By verifying these signatures, cluster administrators can ensure that the images pulled from Stackable's container registry are authentic and have not been tampered with. Since Kubernetes does not have native support for verifying image signatures yet, we will use a tool called https://kyverno.io/[Kyverno] in this tutorial. +IMPORTANT: Releases prior to SDP 23.7 do not have signed images. If you are using an older release and enforce image signature verification, Pods with Stackable images will be prevented from starting. + == Installing Kyverno Kyverno can be easily installed via Helm: @@ -17,7 +19,7 @@ Other installation methods and options to run Kyverno in a highly-available fash == Creating a policy to verify image signatures -Now that Kyverno is installed, we can create a policy that verifies that all operator images that are part of the SDP 23.7 releases are signed by Stackable's CI pipeline (Github Actions): +Now that Kyverno is installed, we can create a policy that verifies that all images provided by Stackable are signed by Stackable's CI pipeline (Github Actions): [source,yaml] include::example$verify-signatures/kyverno-policy.yaml[] @@ -28,6 +30,6 @@ Apply this policy to the cluster by saving it as `kyverno-policy.yaml` and runni kubectl apply -f kyverno-policy.yaml ---- -The policy will be applied to all namespaces in the cluster. It checks all newly created Pods that run any image matching the expression `docker.stackable.tech/+++*+++-operator:23.7.+++*+++` (all Stackable operators version 23.7.+++*+++) and ensures that these images have been signed by a Stackable Github Action from the release 23.7 (`https://github.com/stackabletech/+++*+++-operator/.github/workflows/build.yml@refs/tags/23.7.+++*+++`). If the signature of an operator image is invalid or missing, the policy will deny the pod creation. +The policy will be applied to all namespaces in the cluster. It checks all newly created Pods that run any image matching the expression `docker.stackable.tech/+++*+++` (all images provided by Stackable) and ensures that these images have been signed by a Stackable Github Action (`https://github.com/stackabletech/+++*+++/.github/workflows/build.yml@refs/+++*+++`). If the signature of an image is invalid or missing, the policy will deny the pod creation. For a more detailed explanation of the policy options, please refer to the https://kyverno.io/docs/writing-policies/verify-images/sigstore/#keyless-signing-and-verification[Kyverno documentation]. -If the `subject` field in the policy is changed to something like `https://github.com/test/+++*+++`, the policy will deny the creation of operator pods because the signature is no longer valid. +If the `subject` field in the policy is changed to something like `https://github.com/test/+++*+++`, the policy will deny the creation of pods with Stackable images because the signature is no longer valid. From e07dc0c240947cbe9d7b6b66849e78a93bdcab8b Mon Sep 17 00:00:00 2001 From: dervoeti Date: Mon, 4 Sep 2023 15:57:29 +0200 Subject: [PATCH 02/52] Upcoming changes regarding Superset --- modules/ROOT/pages/release_notes.adoc | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index d259d6746..1c5688fb7 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -3,6 +3,21 @@ The Stackable platform consists of multiple operators that work together. Periodically a platform release is made, including all components of the platform at a specific version. +== Release 23.11 + +... + +=== Breaking changes + +The `loadExamplesOnInit` option was removed from the `SupersetCluster` CRD. Loading examples is still supported, the process is now described in the https://docs.stackable.tech/home/stable/superset/getting_started/first_steps[documentation]. + +==== Stackable Operator for Apache Superset + +* https://github.com/stackabletech/superset-operator/pull/396[Remove SupersetDB] + +`.spec.clusterConfig.loadExamplesOnInit` was removed from the CRD. Already loaded examples in Superset will not be removed by this change. +Additionally, the `SupersetDB` custom resource is not used anymore. Those CRs should be deleted. + == Release 23.7 This release introduces the specification of resource quotas and pod overrides and updates the product versions supported by SDP. From e20baa5fb5a6edbf7693aebae8792c8e508d0002 Mon Sep 17 00:00:00 2001 From: dervoeti Date: Mon, 25 Sep 2023 13:41:05 +0200 Subject: [PATCH 03/52] Upcoming changes regarding Airflow --- modules/ROOT/pages/release_notes.adoc | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index 1c5688fb7..9a4ef4132 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -9,7 +9,15 @@ Periodically a platform release is made, including all components of the platfor === Breaking changes -The `loadExamplesOnInit` option was removed from the `SupersetCluster` CRD. Loading examples is still supported, the process is now described in the https://docs.stackable.tech/home/stable/superset/getting_started/first_steps[documentation]. +The `.spec.clusterConfig.loadExamplesOnInit` option was removed from the `SupersetCluster` CRD. Loading examples is still supported, the process is now described in the https://docs.stackable.tech/home/stable/superset/getting_started/first_steps[documentation]. +The `.spec.clusterConfig.databaseInitialization` option was removed from the `AirflowCluster` CRD. + +==== Stackable Operator for Apache Airflow + +* https://github.com/stackabletech/airflow-operator/pull/322[Removed AirflowDB] + +`.spec.clusterConfig.databaseInitialization` was removed from the CRD. This allowed configuring logging for the database initialization job, which does not exist anymore. Instead, database initialization is done by the scheduler pod, which has its own logging configuration. This was necessary to get rid of the `AirflowDB` Custom Resource, which caused problems when upgrading or reinstalling Airflow clusters. +Additionally, the `AirflowDB` custom resource is not used anymore. Those CRs should be deleted. ==== Stackable Operator for Apache Superset From 1ecb697d2c38bf2079e31fb7dd39ac117f5451fa Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Fri, 10 Nov 2023 11:51:58 +0100 Subject: [PATCH 04/52] updated ldap tutorial docs --- .../ldap-auth/superset-auth-snippet.yaml | 7 ++- .../ldap-auth/trino-auth-snippet.yaml | 7 +-- .../ldap-auth/trino-opa-bundle-snippet.yaml | 22 ++++++- .../pages/authentication_with_openldap.adoc | 57 +++++++++---------- 4 files changed, 55 insertions(+), 38 deletions(-) diff --git a/modules/tutorials/examples/ldap-auth/superset-auth-snippet.yaml b/modules/tutorials/examples/ldap-auth/superset-auth-snippet.yaml index 9d805666e..6f3c0913e 100644 --- a/modules/tutorials/examples/ldap-auth/superset-auth-snippet.yaml +++ b/modules/tutorials/examples/ldap-auth/superset-auth-snippet.yaml @@ -1,7 +1,8 @@ --- # tag::snippet[] spec: - authenticationConfig: # <1> - authenticationClass: openldap # <2> - userRegistrationRole: Admin # <3> + clusterConfig: + authentication: # <1> + - authenticationClass: openldap # <2> + userRegistrationRole: Admin # <3> # end::snippet[] \ No newline at end of file diff --git a/modules/tutorials/examples/ldap-auth/trino-auth-snippet.yaml b/modules/tutorials/examples/ldap-auth/trino-auth-snippet.yaml index bf5604915..d6e454d87 100644 --- a/modules/tutorials/examples/ldap-auth/trino-auth-snippet.yaml +++ b/modules/tutorials/examples/ldap-auth/trino-auth-snippet.yaml @@ -1,8 +1,7 @@ --- # tag::snippet[] spec: - authentication: - method: - ldap: # <1> - authenticationClass: openldap # <2> + clusterConfig: + authentication: + - authenticationClass: openldap # <1> # end::snippet[] diff --git a/modules/tutorials/examples/ldap-auth/trino-opa-bundle-snippet.yaml b/modules/tutorials/examples/ldap-auth/trino-opa-bundle-snippet.yaml index 4cbcb2dd1..65f266ecc 100644 --- a/modules/tutorials/examples/ldap-auth/trino-opa-bundle-snippet.yaml +++ b/modules/tutorials/examples/ldap-auth/trino-opa-bundle-snippet.yaml @@ -2,12 +2,32 @@ data: trino.rego: | package trino + import future.keywords.in + default allow = false allow { + is_alice + } + extended[i] { + some i + input.action.filterResources[i] + is_alice + } + + is_alice() { input.context.identity.user == "alice" } allow { + is_bob + } + extended[i] { + some i + input.action.filterResources[i] + is_bob + } + + is_bob() { input.context.identity.user == "bob" - } \ No newline at end of file + } diff --git a/modules/tutorials/pages/authentication_with_openldap.adoc b/modules/tutorials/pages/authentication_with_openldap.adoc index 60cd13c59..075387606 100644 --- a/modules/tutorials/pages/authentication_with_openldap.adoc +++ b/modules/tutorials/pages/authentication_with_openldap.adoc @@ -136,8 +136,8 @@ metadata: ... spec: version: ... - statsdExporterVersion: ... - credentialsSecret: superset-credentials + clusterConfig: + credentialsSecret: superset-credentials nodes: roleGroups: default: @@ -156,7 +156,7 @@ Modify your `superset.yaml` to include this new `authenticationConfig` property [source,yaml] include::example$ldap-auth/superset-auth-snippet.yaml[tag=snippet] -<1> The new `authenticationConfig` section which configures how Superset is authenticating users +<1> The new `authentication` section which configures how Superset is authenticating users <2> The `authenticationClass` property is referencing the AuthenticationClass `openldap` you created earlier <3> The default Superset role that users should be assigned to when they log in. Any user will be an Admin @@ -172,12 +172,11 @@ metadata: name: superset ... spec: - version: ... - statsdExporterVersion: ... - credentialsSecret: superset-credentials - authenticationConfig: - authenticationClass: openldap - userRegistrationRole: Admin + clusterConfig: + credentialsSecret: superset-credentials + authentication: + - authenticationClass: openldap + userRegistrationRole: Admin nodes: roleGroups: default: @@ -199,7 +198,7 @@ Connect to superset as before, and try logging in again with username _admin_ an Trino is configured very similarly to Superset. -Fetch the existing TrinoCluster defintion from the Kubernetes API server and save it into a `trino.yaml` file: +Fetch the existing TrinoCluster definition from the Kubernetes API server and save it into a `trino.yaml` file: [source,bash] include::example$ldap-auth/60-modify-trino.sh[tag=get-yaml] @@ -216,16 +215,17 @@ metadata: name: trino ... spec: - version: 396-stackable0.1.0 - authentication: - method: - multiUser: + clusterConfig: + authentication: + - authenticationClass: trino-users + authorization: + opa: + configMapName: opa + package: trino + catalogLabelSelector: + matchLabels: + trino: trino ... - opa: - configMapName: opa - package: trino - catalogLabelSelector: - ... workers: ... coordinators: @@ -244,8 +244,7 @@ Replace the `multiUser` authentication method in your `trino.yaml` with an `ldap [source,yaml] include::example$ldap-auth/trino-auth-snippet.yaml[tag=snippet] -<1> The new `ldap` authentication method replaces the previous `multiUser` authentication method -<2> The `authenticationClass` property is referencing the AuthenticationClass `openldap` you created earlier +<1> The `authenticationClass` property is referencing the AuthenticationClass `openldap` you created earlier .Your `trino.yaml` should now look similar to this [%collapsible] @@ -259,15 +258,13 @@ metadata: name: trino ... spec: - version: 396-stackable0.1.0 - authentication: - method: - ldap: - authenticationClass: openldap - opa: - configMapName: opa - package: trino - catalogLabelSelector: + clusterConfig: + authentication: + - authenticationClass: openldap + authorization: + opa: + configMapName: opa + package: trino ... workers: ... From 8beff9ca19e96ed66c3bb1e735a7ce8811b7b0ac Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Tue, 14 Nov 2023 12:28:26 +0100 Subject: [PATCH 05/52] wip: 23.11 release notes --- modules/ROOT/pages/release-notes.adoc | 165 +++++++++++++++++++++++++- 1 file changed, 161 insertions(+), 4 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index f26bd0a2b..0cfe43f61 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -6,26 +6,183 @@ including all components of the platform at a specific version. == Release 23.11 +This release introduces two further elements of platform stability - pod disruption budgets and graceful shutdown specifications - as well as a new management tool and updated product version support. + +=== New / extended platform features + +The following new major platform features were added: + +PodDisruptionBudgets:: +Kubernetes has mechanisms to ensure minimal planned downtime. Our product operators deploy so-called PodDisruptionBudget (PDB) resources alongside the products. For every role that you specify (e.g. HDFS namenodes or Trino workers) a PDB is created. This will determine the extent to which roles for a given application may be inactive at any given time. See xref:concepts:operations/pod_disruptions.adoc[the +documentation] for more details. + +Graceful shutdown:: +Graceful shutdown refers to the managed, controlled shutdown of service instances in the manner intended by the software authors. Typically, an instance will receive a signal indicating the intent for the server to shut down, and it will initiate a controlled shutdown. Our operators configure a sensible amount of time Pods are granted to properly shut down without disrupting the availability of the product. See xref:concepts:operations/graceful_shutdown.adoc[the +documentation] for more details. + +Stackable Cockpit:: +This release includes a preview version of Stackable Cockpit, a browser-based management tool which interacts with the Stackable data platform to display e.g. deployed stacklets and their status. + +stackablectl 1.0.0?:: +... + +Listener operator?:: +... + +New Versions:: + +The following new product versions are now supported: + +* https://github.com/stackabletech/airflow-operator/pull/334[Airflow: 2.6.3, 2.7.2] +* https://github.com/stackabletech/druid-operator/pull/480[Druid: 27.0.0] +* https://github.com/stackabletech/hbase-operator/pull/403[HBase: 2.4.17] +* https://github.com/stackabletech/hdfs-operator/pull/409[HDFS: 3.2.4, 3.3.6] +* https://github.com/stackabletech/kafka-operator/pull/627[Kafka: 2.8.2, 3.4.1, 3.5.1] +* https://github.com/stackabletech/nifi-operator/pull/513[Nifi: 1.23.2] +* https://github.com/stackabletech/opa-operator/pull/482[Opa: 0.57] +* https://github.com/stackabletech/spark-k8s-operator/pull/291[Spark: 3.4.1, 3.5.0] +* https://github.com/stackabletech/superset-operator/pull/415[Superset: 2.1.1, 3.0.1] +* https://github.com/stackabletech/trino-operator/pull/491[Trino: 428] +* https://github.com/stackabletech/zookeeper-operator/pull/732[ZooKeeper: 3.8.3] + +Deprecated Versions:: + +The following product versions are deprecated and will be removed in a later release: + +* Airflow: 2.6.1 +* HBase: 2.4.12 +* HDFS: 3.2.4, 3.3.4 +* Kafka: 2.8.2, 3.4.0 +* Opa: 0.51.0 +* Spark: 3.4.0 +* Superset: 2.1.0 +* Zookeeper: 381 + +N.B. in some cases a newly supported version is also immediately marked as deprecated. This is done to allow an update path from the latest patch of a minor version (e.g. Kafka 2.8.2 --> 3.4.1). + +Removed Versions:: + +The following product versions are no longer supported (although images for released product versions remain available https://repo.stackable.tech/#browse/browse:docker:v2%2Fstackable[here]): + +* Airflow: 2.2.3, 2.2.4, 2.2.5, 2.4.1 +* Druid: 0.23.0, 24.0.0 +* HBase: 2.4.6, 2.4.8, 2.4.9, 2.4.11 +* HDFS: 3.3.1, 3.3.3 +* Hive: 2.3.9 +* Kafka: 2.7.1, 3.1.0, 3.2.0, 3.3.1 +* Nifi: 1.15.x, 1.16.x, 1.18.0, 1.20.0 +* Opa: 0.27.1, 0.28.0, 0.37.2, 0.41.0, 0.45.0 +* Spark: 3.2.1, 3.3.0 +* Superset: 1.3.2, 1.4.1, 1.4.2, 1.5.1, 1.5.3, 2.0.1 +* Trino: 377, 387, 395, 396, 403 +* Zookeeper: 3.5.8, 3.6.3, 3.7.0, 3.8.0 + +Product features:: + +Additionally, there are some individual product features that are noteworthy: + +* https://github.com/stackabletech/airflow-operator/issues/311[Airflow: implement KubernetesExecutor] +* https://github.com/stackabletech/trino-operator/issues/491[Trino: support for the new OPA authorizer] +* https://github.com/stackabletech/docker-images/pull/494[HBase: support for Hadoop native compression] +* https://github.com/stackabletech/docker-images/pull/400[HDFS: support for FUSE] + +=== stackablectl + +The following demo has been added to `stackablectl`: + +==== Signal processing demo + +This demonstrates the ingestion of streamed data into a Timescale time-series database, augmented by a moving window of anomaly detection measurements that are plotted alongside the raw data in Grafana. + +This link lists the xref:demos:index.adoc[available demos]. + +=== Supported Kubernetes versions + +This release supports the following Kubernetes versions: + +* `1.28` _*CHECK*_ +* `1.27` +* `1.26` + +This Kubernetes version is no longer supported: + +* `1.25` +* `1.24` + +=== Supported OpenShift versions + +This release supports the following OpenShift versions: + +* `4.xx` _*TODO*_ + ... === Breaking changes -The `.spec.clusterConfig.loadExamplesOnInit` option was removed from the `SupersetCluster` CRD. Loading examples is still supported, the process is now described in the https://docs.stackable.tech/home/stable/superset/getting_started/first_steps[documentation]. -The `.spec.clusterConfig.databaseInitialization` option was removed from the `AirflowCluster` CRD. +You will need to adapt your existing CRDs due to the following breaking changes detailed below. ==== Stackable Operator for Apache Airflow * https://github.com/stackabletech/airflow-operator/pull/322[Removed AirflowDB] -`.spec.clusterConfig.databaseInitialization` was removed from the CRD. This allowed configuring logging for the database initialization job, which does not exist anymore. Instead, database initialization is done by the scheduler pod, which has its own logging configuration. This was necessary to get rid of the `AirflowDB` Custom Resource, which caused problems when upgrading or reinstalling Airflow clusters. +`.spec.clusterConfig.databaseInitialization` was removed from the CRD. This allowed configuring logging for the database initialization job, which does not exist anymore. Instead, database initialization is done by the scheduler pod, which has its own logging configuration. This was necessary to remove the `AirflowDB` Custom Resource, which caused problems when upgrading or reinstalling Airflow clusters. Additionally, the `AirflowDB` custom resource is not used anymore. Those CRs should be deleted. +* https://github.com/stackabletech/airflow-operator/pull/311[Implement KubernetesExecutor] + +As an alternative to the CeleryExecutor we now also support the KubernetesExecutor, whereby pods are spun up for job tasks and are terminated afterwards. This removes the need a Redis job queue and offers an alternative approach to resource management. + +* https://github.com/stackabletech/airflow-operator/pull/316[Rename service port name] + +The service port name has been renamed from airflow to http for consistency reasons. This change should normally not be breaking as only name has been changed, not the port. However, there might be some e.g. Ingresses that rely on the port name and need to be updated + +* https://github.com/stackabletech/airflow-operator/pull/303[AuthenticationClass references] +`spec.clusterConfig.authenticationConfig` has been consolidated to `spec.clusterConfig.authentication` which takes a vector of AuthenticationClass references + ==== Stackable Operator for Apache Superset * https://github.com/stackabletech/superset-operator/pull/396[Remove SupersetDB] `.spec.clusterConfig.loadExamplesOnInit` was removed from the CRD. Already loaded examples in Superset will not be removed by this change. -Additionally, the `SupersetDB` custom resource is not used anymore. Those CRs should be deleted. +Additionally, the `SupersetDB` custom resource is not used anymore. Those CRs should be deleted. Loading examples is still supported, the process is now described in the https://docs.stackable.tech/home/stable/superset/getting_started/first_steps[documentation]. + +Rename Service port name from superset to http for consistency reasons. This change should normally not be breaking, as we only change the name, not the port. However, there might be some e.g. Ingresses that rely on the port name and need to be updated +https://github.com/stackabletech/superset-operator/pull/394 + + +Kafka: +Let secret-operator handle certificate conversion. Doing so we were able to remove the prepare init container with the effect, that you can't configure the log level for this container anymore. You need to remove the field spec.brokers.config.logging.container.prepare in case you have specified it +https://github.com/stackabletech/kafka-operator/pull/621 + +HDFS: +Removed field autoFormatFs, which was never read +https://github.com/stackabletech/hdfs-operator/pull/422 + +Nifi: +Consolidated authentication config to a list of AuthenticationClasses +https://github.com/stackabletech/nifi-operator/pull/498 + +Removed crd support for the auto generation of admin credentials +https://github.com/stackabletech/nifi-operator/pull/498 + +Removed crd support for nifi.security.allow.anonymous.authentication that was never actually used +https://github.com/stackabletech/nifi-operator/pull/498 + +Spark: +use product image selection instead of version +https://github.com/stackabletech/spark-k8s-operator/pull/275 + +refactored application roles to use CommonConfiguration structures from the operator framework +https://github.com/stackabletech/spark-k8s-operator/pull/277 + +Trino: +Version 428 uses the new OPA authorizer from https://github.com/bloomberg/trino/tree/add-open-policy-agent which requires existing rego rules to be changed + + + + +=== Upgrade from 23.7 +_*TODO*_ == Release 23.7 From 36c4f2d3f8f34a5439e641b56040be4d107a09b2 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Tue, 14 Nov 2023 13:43:29 +0100 Subject: [PATCH 06/52] documented breaking changes --- modules/ROOT/pages/release-notes.adoc | 55 ++++++++++----------------- 1 file changed, 21 insertions(+), 34 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 0cfe43f61..60fc5c2c6 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -123,62 +123,49 @@ You will need to adapt your existing CRDs due to the following breaking changes ==== Stackable Operator for Apache Airflow -* https://github.com/stackabletech/airflow-operator/pull/322[Removed AirflowDB] - -`.spec.clusterConfig.databaseInitialization` was removed from the CRD. This allowed configuring logging for the database initialization job, which does not exist anymore. Instead, database initialization is done by the scheduler pod, which has its own logging configuration. This was necessary to remove the `AirflowDB` Custom Resource, which caused problems when upgrading or reinstalling Airflow clusters. +* https://github.com/stackabletech/airflow-operator/pull/322[Removed AirflowDB] `.spec.clusterConfig.databaseInitialization` was removed from the CRD. This allowed configuring logging for the database initialization job, which does not exist anymore. Instead, database initialization is done by the scheduler pod, which has its own logging configuration. This was necessary to remove the `AirflowDB` Custom Resource, which caused problems when upgrading or reinstalling Airflow clusters. Additionally, the `AirflowDB` custom resource is not used anymore. Those CRs should be deleted. -* https://github.com/stackabletech/airflow-operator/pull/311[Implement KubernetesExecutor] - -As an alternative to the CeleryExecutor we now also support the KubernetesExecutor, whereby pods are spun up for job tasks and are terminated afterwards. This removes the need a Redis job queue and offers an alternative approach to resource management. +* https://github.com/stackabletech/airflow-operator/pull/311[Implement KubernetesExecutor] As an alternative to the CeleryExecutor we now also support the KubernetesExecutor, whereby pods are spun up for job tasks and are terminated afterwards. This removes the need a Redis job queue and offers an alternative approach to resource management. -* https://github.com/stackabletech/airflow-operator/pull/316[Rename service port name] - -The service port name has been renamed from airflow to http for consistency reasons. This change should normally not be breaking as only name has been changed, not the port. However, there might be some e.g. Ingresses that rely on the port name and need to be updated +* https://github.com/stackabletech/airflow-operator/pull/316[Rename service port name] The service port name has been renamed from airflow to http for consistency reasons. This change should normally not be breaking as only name has been changed, not the port. However, there might be some e.g. Ingresses that rely on the port name and need to be updated * https://github.com/stackabletech/airflow-operator/pull/303[AuthenticationClass references] `spec.clusterConfig.authenticationConfig` has been consolidated to `spec.clusterConfig.authentication` which takes a vector of AuthenticationClass references ==== Stackable Operator for Apache Superset -* https://github.com/stackabletech/superset-operator/pull/396[Remove SupersetDB] +* https://github.com/stackabletech/superset-operator/pull/396[Remove SupersetDB] `.spec.clusterConfig.loadExamplesOnInit` was removed from the CRD. Already loaded examples in Superset will not be removed by this change. Additionally, the `SupersetDB` custom resource is not used anymore. Those CRs should be deleted. Loading examples is still supported, the process is now described in the https://docs.stackable.tech/home/stable/superset/getting_started/first_steps[documentation]. + +* https://github.com/stackabletech/superset-operator/pull/394[Rename service port name] + The service port name has been renamed from superset to http for consistency reasons. This change should normally not be breaking, as we only change the name, not the port. However, there might be some e.g. Ingresses that rely on the port name and need to be updated. -`.spec.clusterConfig.loadExamplesOnInit` was removed from the CRD. Already loaded examples in Superset will not be removed by this change. -Additionally, the `SupersetDB` custom resource is not used anymore. Those CRs should be deleted. Loading examples is still supported, the process is now described in the https://docs.stackable.tech/home/stable/superset/getting_started/first_steps[documentation]. +==== Stackable Operator for Apache Kafka -Rename Service port name from superset to http for consistency reasons. This change should normally not be breaking, as we only change the name, not the port. However, there might be some e.g. Ingresses that rely on the port name and need to be updated -https://github.com/stackabletech/superset-operator/pull/394 +* https://github.com/stackabletech/kafka-operator/pull/621[Certificate conversion] The secret-operator now handles certificate conversion. This allows for the removal of the prepare init container but means that you can't configure the log level for this container anymore. You will need to remove the field `spec.brokers.config.logging.container.prepare` in case it is specified. +==== Stackable Operator for Apache HDFS -Kafka: -Let secret-operator handle certificate conversion. Doing so we were able to remove the prepare init container with the effect, that you can't configure the log level for this container anymore. You need to remove the field spec.brokers.config.logging.container.prepare in case you have specified it -https://github.com/stackabletech/kafka-operator/pull/621 +* https://github.com/stackabletech/hdfs-operator/pull/422[Removed field autoFormatFs] This field was not used. -HDFS: -Removed field autoFormatFs, which was never read -https://github.com/stackabletech/hdfs-operator/pull/422 +==== Stackable Operator for Apache Nifi -Nifi: -Consolidated authentication config to a list of AuthenticationClasses -https://github.com/stackabletech/nifi-operator/pull/498 +* https://github.com/stackabletech/nifi-operator/pull/498[AuthenticationClass references] Consolidated authentication config to a list of AuthenticationClasses -Removed crd support for the auto generation of admin credentials -https://github.com/stackabletech/nifi-operator/pull/498 +* https://github.com/stackabletech/nifi-operator/pull/498[Remove credential generation] Removed crd support for the auto generation of admin credentials -Removed crd support for nifi.security.allow.anonymous.authentication that was never actually used -https://github.com/stackabletech/nifi-operator/pull/498 +* https://github.com/stackabletech/nifi-operator/pull/498[Remove redundant authenticaiton field] Removed crd support for `nifi.security.allow.anonymous.authentication` that was never used. -Spark: -use product image selection instead of version -https://github.com/stackabletech/spark-k8s-operator/pull/275 +==== Stackable Operator for Apache Spark -refactored application roles to use CommonConfiguration structures from the operator framework -https://github.com/stackabletech/spark-k8s-operator/pull/277 +* https://github.com/stackabletech/spark-k8s-operator/pull/275[Image specification] Use product image selection instead of version. -Trino: -Version 428 uses the new OPA authorizer from https://github.com/bloomberg/trino/tree/add-open-policy-agent which requires existing rego rules to be changed +* https://github.com/stackabletech/spark-k8s-operator/pull/277[Configuration structure] +Refactored application roles to use CommonConfiguration structures from the operator framework. +==== Stackable Operator for Trino +* https://github.com/stackabletech/trino-operator/pull/491[New OPA Authorizer] Version 428 uses the https://github.com/bloomberg/trino/tree/add-open-policy-agent[new OPA authorizer] which requires changes to existing rego rules. === Upgrade from 23.7 From 0342c77e9cc15571bc14755c447d2898851ddf65 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Tue, 14 Nov 2023 14:07:40 +0100 Subject: [PATCH 07/52] draft (i.e. untested) upgrade notes --- modules/ROOT/pages/release-notes.adoc | 141 +++++++++++++++++++++++++- 1 file changed, 140 insertions(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 60fc5c2c6..72969881f 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -169,7 +169,146 @@ Refactored application roles to use CommonConfiguration structures from the oper === Upgrade from 23.7 -_*TODO*_ + +==== Using stackablectl + +To uninstall the `23.7` release run + +[source,console] +---- +$ stackablectl release uninstall 23.7 +[INFO ] Uninstalling release 23.7 +[INFO ] Uninstalling airflow operator +[INFO ] Uninstalling commons operator +# ... +---- + +Afterwards you will need to upgrade the CustomResourceDefinitions (CRDs) installed by the Stackable Platform. +The reason for this is that helm will uninstall the operators but not the CRDs. This can be done using `kubectl replace`: + +[source] +---- +kubectl replace -f https://raw.githubusercontent.com/stackabletech/airflow-operator/23.11.0/deploy/helm/airflow-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/commons-operator/23.11.0/deploy/helm/commons-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/druid-operator/23.11.0/deploy/helm/druid-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/hbase-operator/23.11.0/deploy/helm/hbase-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/hdfs-operator/23.11.0/deploy/helm/hdfs-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/hello-world-operator/23.11.0/deploy/helm/hello-world-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/hive-operator/23.11.0/deploy/helm/hive-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/kafka-operator/23.11.0/deploy/helm/kafka-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/listener-operator/23.11.0/deploy/helm/listener-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/nifi-operator/23.11.0/deploy/helm/nifi-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/opa-operator/23.11.0/deploy/helm/opa-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/secret-operator/23.11.0/deploy/helm/secret-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/spark-k8s-operator/23.11.0/deploy/helm/spark-k8s-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/superset-operator/23.11.0/deploy/helm/superset-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/trino-operator/23.11.0/deploy/helm/trino-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/zookeeper-operator/23.11.0/deploy/helm/zookeeper-operator/crds/crds.yaml +---- + +[source,console] +---- +customresourcedefinition.apiextensions.k8s.io "airflowclusters.airflow.stackable.tech" replaced +customresourcedefinition.apiextensions.k8s.io "airflowdbs.airflow.stackable.tech" replaced +customresourcedefinition.apiextensions.k8s.io "authenticationclasses.authentication.stackable.tech" replaced +customresourcedefinition.apiextensions.k8s.io "s3connections.s3.stackable.tech" replaced +... +---- + +To install the `23.11` release run + +[source,console] +---- +$ stackablectl release install 23.11 +[INFO ] Installing release 23.11 +[INFO ] Installing airflow operator in version 23.11.0 +[INFO ] Installing commons operator in version 23.11.0 +[INFO ] Installing druid operator in version 23.11.0 +[INFO ] Installing hbase operator in version 23.11.0 +[INFO ] Installing hdfs operator in version 23.11.0 +[INFO ] Installing hive operator in version 23.11.0 +[INFO ] Installing kafka operator in version 23.11.0 +[INFO ] Installing listener operator in version 23.11.0 +[INFO ] Installing hello-world operator in version 23.11.0 +[INFO ] Installing nifi operator in version 23.11.0 +[INFO ] Installing opa operator in version 23.11.0 +[INFO ] Installing secret operator in version 23.11.0 +[INFO ] Installing spark-k8s operator in version 23.11.0 +[INFO ] Installing superset operator in version 23.11.0 +[INFO ] Installing trino operator in version 23.11.0 +[INFO ] Installing zookeeper operator in version 23.11.0 +---- + +==== Using helm +Use `helm list` to list the currently installed operators. + +You can use the following command to uninstall all operators that are part of the `23.7` release: + +[source,console] +---- +$ helm uninstall airflow-operator commons-operator druid-operator hbase-operator hdfs-operator hive-operator kafka-operator listener-operator nifi-operator opa-operator secret-operator spark-k8s-operator superset-operator trino-operator zookeeper-operator +release "airflow-operator" uninstalled +release "commons-operator" uninstalled +# ... +---- + +Afterwards you will need to upgrade the CustomResourceDefinitions (CRDs) installed by the Stackable Platform. +The reason for this is that helm will uninstall the operators but not the CRDs. This can be done using `kubectl replace`: + +[source,console] +---- +kubectl replace -f https://raw.githubusercontent.com/stackabletech/airflow-operator/23.11.0/deploy/helm/airflow-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/commons-operator/23.11.0/deploy/helm/commons-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/druid-operator/23.11.0/deploy/helm/druid-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/hbase-operator/23.11.0/deploy/helm/hbase-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/hdfs-operator/23.11.0/deploy/helm/hdfs-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/hello-world-operator/23.11.0/deploy/helm/hello-world-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/hive-operator/23.11.0/deploy/helm/hive-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/kafka-operator/23.11.0/deploy/helm/kafka-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/listener-operator/23.11.0/deploy/helm/listener-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/nifi-operator/23.11.0/deploy/helm/nifi-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/opa-operator/23.11.0/deploy/helm/opa-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/secret-operator/23.11.0/deploy/helm/secret-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/spark-k8s-operator/23.11.0/deploy/helm/spark-k8s-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/superset-operator/23.11.0/deploy/helm/superset-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/trino-operator/23.11.0/deploy/helm/trino-operator/crds/crds.yaml +kubectl replace -f https://raw.githubusercontent.com/stackabletech/zookeeper-operator/23.11.0/deploy/helm/zookeeper-operator/crds/crds.yaml +---- + +To install the `23.11` release run + +[source,console] +---- +helm repo add stackable-stable https://repo.stackable.tech/repository/helm-stable/ +helm repo update stackable-stable +helm install --wait airflow-operator stackable-stable/airflow-operator --version 23.11.0 +helm install --wait commons-operator stackable-stable/commons-operator --version 23.11.0 +helm install --wait druid-operator stackable-stable/druid-operator --version 23.11.0 +helm install --wait hbase-operator stackable-stable/hbase-operator --version 23.11.0 +helm install --wait hdfs-operator stackable-stable/hdfs-operator --version 23.11.0 +helm install --wait hive-operator stackable-stable/hive-operator --version 23.11.0 +helm install --wait kafka-operator stackable-stable/kafka-operator --version 23.11.0 +helm install --wait listener-operator stackable-stable/listener-operator --version 23.11.0 +helm install --wait hello-world-operator stackable-stable/hello-world-operator --version 23.11.0 +helm install --wait nifi-operator stackable-stable/nifi-operator --version 23.11.0 +helm install --wait opa-operator stackable-stable/opa-operator --version 23.11.0 +helm install --wait secret-operator stackable-stable/secret-operator --version 23.11.0 +helm install --wait spark-k8s-operator stackable-stable/spark-k8s-operator --version 23.11.0 +helm install --wait superset-operator stackable-stable/superset-operator --version 23.11.0 +helm install --wait trino-operator stackable-stable/trino-operator --version 23.11.0 +helm install --wait zookeeper-operator stackable-stable/zookeeper-operator --version 23.11.0 +---- + +==== Known upgrade issues + +In the case of the breaking changes detailed above it will be necessary to update the custom resources and re-apply them. + +Additionally, please note the following: + +===== All operators + +* If the default PVC size has been changed, then the StatefulSet must be deleted: it is not possible to change the PVC +in the StatefulSet specification. == Release 23.7 From 42174dde4869f415384cae2cbda6f1dfaf9d96f1 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Tue, 14 Nov 2023 18:05:11 +0100 Subject: [PATCH 08/52] added product signage and JVM-settings to highlights --- modules/ROOT/pages/release-notes.adoc | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 72969881f..2334738c8 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -20,6 +20,12 @@ Graceful shutdown:: Graceful shutdown refers to the managed, controlled shutdown of service instances in the manner intended by the software authors. Typically, an instance will receive a signal indicating the intent for the server to shut down, and it will initiate a controlled shutdown. Our operators configure a sensible amount of time Pods are granted to properly shut down without disrupting the availability of the product. See xref:concepts:operations/graceful_shutdown.adoc[the documentation] for more details. +Signed SDP product images:: +As of this release all Stackable product images are signed (the signing of operator images was delivered in SDP 23.7). + +Overridable Java security settings:: +For JVM-based products (i.e. Druid, HBase, HDFS, Hive, Kafka, Nifi, Spark, Trno and Zookeeper) it is now possible to provide custom security settings that override the default values. This allows the user to control things such as DNS lookup caches. + Stackable Cockpit:: This release includes a preview version of Stackable Cockpit, a browser-based management tool which interacts with the Stackable data platform to display e.g. deployed stacklets and their status. From bdd5d244debcf280966c6d1204ae9f911c126f4b Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Wed, 15 Nov 2023 11:58:43 +0100 Subject: [PATCH 09/52] re-worked highlight list and product features --- modules/ROOT/pages/release-notes.adoc | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 2334738c8..0e11c67e0 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -23,16 +23,21 @@ documentation] for more details. Signed SDP product images:: As of this release all Stackable product images are signed (the signing of operator images was delivered in SDP 23.7). +Airflow KubernetesExecutor:: +Airflow clusters can now be configured to use Kubernetes executors, whereby pods are spun up for job tasks and terminated when complete, thus offering an alternative way to use resources without the need for job queuing. + Overridable Java security settings:: -For JVM-based products (i.e. Druid, HBase, HDFS, Hive, Kafka, Nifi, Spark, Trno and Zookeeper) it is now possible to provide custom security settings that override the default values. This allows the user to control things such as DNS lookup caches. +For JVM-based products (i.e. Druid, HBase, HDFS, Hive, Kafka, Nifi, Spark, Trino and Zookeeper) it is now possible to provide custom security settings that override the default values. This allows the user to control things such as DNS lookup caches. Stackable Cockpit:: -This release includes a preview version of Stackable Cockpit, a browser-based management tool which interacts with the Stackable data platform to display e.g. deployed stacklets and their status. +This release includes a very early preview version of Stackable Cockpit, a browser-based management tool which interacts with the Stackable data platform to display e.g. deployed stacklets and their status. -stackablectl 1.0.0?:: -... +stackablectl:: +Our command line tool has been re-worked as it is now the backbone for Stackable Cockpit: you can find out about the recent enhancements by visiting the online https://docs.stackable.tech/management/stable/stackablectl/[documentation] (TODO check link). Listener operator?:: + +SBOMs?:: ... New Versions:: @@ -85,12 +90,15 @@ The following product versions are no longer supported (although images for rele Product features:: -Additionally, there are some individual product features that are noteworthy: +Additionally, there are some other individual product features that are noteworthy: -* https://github.com/stackabletech/airflow-operator/issues/311[Airflow: implement KubernetesExecutor] -* https://github.com/stackabletech/trino-operator/issues/491[Trino: support for the new OPA authorizer] * https://github.com/stackabletech/docker-images/pull/494[HBase: support for Hadoop native compression] +* https://github.com/stackabletech/docker-images/pull/497[HBase: add operator tools to product image] * https://github.com/stackabletech/docker-images/pull/400[HDFS: support for FUSE] +* https://github.com/stackabletech/docker-images/pull/505[Hive: update postgresql driver to support SCRAM authentication] +* https://github.com/stackabletech/docker-images/pull/436[Spark: all product images contain pyspark] +* https://github.com/stackabletech/trino-operator/issues/491[Trino: support for the new OPA authorizer] +* https://github.com/stackabletech/issues/issues/444[Vector: upgrade to 0.33] === stackablectl From 434f5f7fdb7faef72eb123482e0b1a81dacfe5d6 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Wed, 15 Nov 2023 11:59:54 +0100 Subject: [PATCH 10/52] Update modules/ROOT/pages/release-notes.adoc Co-authored-by: Lars Francke --- modules/ROOT/pages/release-notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 0e11c67e0..893423f2d 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -49,7 +49,7 @@ The following new product versions are now supported: * https://github.com/stackabletech/hbase-operator/pull/403[HBase: 2.4.17] * https://github.com/stackabletech/hdfs-operator/pull/409[HDFS: 3.2.4, 3.3.6] * https://github.com/stackabletech/kafka-operator/pull/627[Kafka: 2.8.2, 3.4.1, 3.5.1] -* https://github.com/stackabletech/nifi-operator/pull/513[Nifi: 1.23.2] +* https://github.com/stackabletech/nifi-operator/pull/513[NiFi: 1.23.2] * https://github.com/stackabletech/opa-operator/pull/482[Opa: 0.57] * https://github.com/stackabletech/spark-k8s-operator/pull/291[Spark: 3.4.1, 3.5.0] * https://github.com/stackabletech/superset-operator/pull/415[Superset: 2.1.1, 3.0.1] From 9648c49b200bcb76e44ef88a31d0e2e20021077e Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Wed, 15 Nov 2023 12:01:36 +0100 Subject: [PATCH 11/52] review feedback --- modules/ROOT/pages/release-notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 0e11c67e0..a5bd92a30 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -27,7 +27,7 @@ Airflow KubernetesExecutor:: Airflow clusters can now be configured to use Kubernetes executors, whereby pods are spun up for job tasks and terminated when complete, thus offering an alternative way to use resources without the need for job queuing. Overridable Java security settings:: -For JVM-based products (i.e. Druid, HBase, HDFS, Hive, Kafka, Nifi, Spark, Trino and Zookeeper) it is now possible to provide custom security settings that override the default values. This allows the user to control things such as DNS lookup caches. +For JVM-based products (i.e. Druid, HBase, HDFS, Hive, Kafka, NiFi, Spark, Trino and ZooKeeper) it is now possible to provide custom security settings that override the default values. This allows the user to control things such as DNS lookup caches. Stackable Cockpit:: This release includes a very early preview version of Stackable Cockpit, a browser-based management tool which interacts with the Stackable data platform to display e.g. deployed stacklets and their status. From e6fdf814d540474e1e75bfc792eecab41a218c6a Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Wed, 15 Nov 2023 12:03:53 +0100 Subject: [PATCH 12/52] Update modules/ROOT/pages/release-notes.adoc Co-authored-by: Lars Francke --- modules/ROOT/pages/release-notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 554c618c2..116316517 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -64,7 +64,7 @@ The following product versions are deprecated and will be removed in a later rel * HBase: 2.4.12 * HDFS: 3.2.4, 3.3.4 * Kafka: 2.8.2, 3.4.0 -* Opa: 0.51.0 +* OpenPolicyAgent: 0.51.0 * Spark: 3.4.0 * Superset: 2.1.0 * Zookeeper: 381 From 2851d21c8fdd518f112f675aacdfea7791d498e9 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Wed, 15 Nov 2023 12:04:00 +0100 Subject: [PATCH 13/52] Update modules/ROOT/pages/release-notes.adoc Co-authored-by: Lars Francke --- modules/ROOT/pages/release-notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 116316517..0821d353b 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -67,7 +67,7 @@ The following product versions are deprecated and will be removed in a later rel * OpenPolicyAgent: 0.51.0 * Spark: 3.4.0 * Superset: 2.1.0 -* Zookeeper: 381 +* ZooKeeper: 381 N.B. in some cases a newly supported version is also immediately marked as deprecated. This is done to allow an update path from the latest patch of a minor version (e.g. Kafka 2.8.2 --> 3.4.1). From d3837995555a795bd035f387efcc2fa40d670359 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Wed, 15 Nov 2023 12:04:07 +0100 Subject: [PATCH 14/52] Update modules/ROOT/pages/release-notes.adoc Co-authored-by: Lars Francke --- modules/ROOT/pages/release-notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 0821d353b..09d1ca443 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -50,7 +50,7 @@ The following new product versions are now supported: * https://github.com/stackabletech/hdfs-operator/pull/409[HDFS: 3.2.4, 3.3.6] * https://github.com/stackabletech/kafka-operator/pull/627[Kafka: 2.8.2, 3.4.1, 3.5.1] * https://github.com/stackabletech/nifi-operator/pull/513[NiFi: 1.23.2] -* https://github.com/stackabletech/opa-operator/pull/482[Opa: 0.57] +* https://github.com/stackabletech/opa-operator/pull/482[OpenPolicyAgent: 0.57] * https://github.com/stackabletech/spark-k8s-operator/pull/291[Spark: 3.4.1, 3.5.0] * https://github.com/stackabletech/superset-operator/pull/415[Superset: 2.1.1, 3.0.1] * https://github.com/stackabletech/trino-operator/pull/491[Trino: 428] From 5de6e858a44c36f5eefa4aef757f74dc645866b4 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Wed, 15 Nov 2023 12:04:15 +0100 Subject: [PATCH 15/52] Update modules/ROOT/pages/release-notes.adoc Co-authored-by: Lars Francke --- modules/ROOT/pages/release-notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 09d1ca443..13611560e 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -162,7 +162,7 @@ Additionally, the `AirflowDB` custom resource is not used anymore. Those CRs sho * https://github.com/stackabletech/hdfs-operator/pull/422[Removed field autoFormatFs] This field was not used. -==== Stackable Operator for Apache Nifi +==== Stackable Operator for Apache NiFi * https://github.com/stackabletech/nifi-operator/pull/498[AuthenticationClass references] Consolidated authentication config to a list of AuthenticationClasses From 92029ed1187320f1b0d85ec9c758a432493be8fd Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Wed, 15 Nov 2023 12:08:08 +0100 Subject: [PATCH 16/52] added TODO --- modules/ROOT/pages/release-notes.adoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 13611560e..1fcf1b7cc 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -133,6 +133,8 @@ This release supports the following OpenShift versions: === Breaking changes +_*TODO*_: add before-and-after yamls to make this transparent + You will need to adapt your existing CRDs due to the following breaking changes detailed below. ==== Stackable Operator for Apache Airflow From 03c7b52ecf53812081e5694b34e4ad3a1eaee088 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Wed, 15 Nov 2023 13:13:20 +0100 Subject: [PATCH 17/52] minor changes --- modules/ROOT/pages/release-notes.adoc | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 1fcf1b7cc..21f6a83d1 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -33,7 +33,7 @@ Stackable Cockpit:: This release includes a very early preview version of Stackable Cockpit, a browser-based management tool which interacts with the Stackable data platform to display e.g. deployed stacklets and their status. stackablectl:: -Our command line tool has been re-worked as it is now the backbone for Stackable Cockpit: you can find out about the recent enhancements by visiting the online https://docs.stackable.tech/management/stable/stackablectl/[documentation] (TODO check link). +Our command line tool has been re-worked as it is now the backbone for Stackable Cockpit: you can find out about the recent enhancements by visiting the online xref:management:stackablectl:index.adoc[documentation]. Listener operator?:: @@ -81,12 +81,12 @@ The following product versions are no longer supported (although images for rele * HDFS: 3.3.1, 3.3.3 * Hive: 2.3.9 * Kafka: 2.7.1, 3.1.0, 3.2.0, 3.3.1 -* Nifi: 1.15.x, 1.16.x, 1.18.0, 1.20.0 +* NiFi: 1.15.x, 1.16.x, 1.18.0, 1.20.0 * Opa: 0.27.1, 0.28.0, 0.37.2, 0.41.0, 0.45.0 * Spark: 3.2.1, 3.3.0 * Superset: 1.3.2, 1.4.1, 1.4.2, 1.5.1, 1.5.3, 2.0.1 * Trino: 377, 387, 395, 396, 403 -* Zookeeper: 3.5.8, 3.6.3, 3.7.0, 3.8.0 +* ZooKeeper: 3.5.8, 3.6.3, 3.7.0, 3.8.0 Product features:: @@ -149,6 +149,11 @@ Additionally, the `AirflowDB` custom resource is not used anymore. Those CRs sho * https://github.com/stackabletech/airflow-operator/pull/303[AuthenticationClass references] `spec.clusterConfig.authenticationConfig` has been consolidated to `spec.clusterConfig.authentication` which takes a vector of AuthenticationClass references +These consolidated changes would require a change from e.g. + +to: + + ==== Stackable Operator for Apache Superset * https://github.com/stackabletech/superset-operator/pull/396[Remove SupersetDB] `.spec.clusterConfig.loadExamplesOnInit` was removed from the CRD. Already loaded examples in Superset will not be removed by this change. Additionally, the `SupersetDB` custom resource is not used anymore. Those CRs should be deleted. Loading examples is still supported, the process is now described in the https://docs.stackable.tech/home/stable/superset/getting_started/first_steps[documentation]. From 28177bf0b766b8fade12cbc0a534bf2dc946a9b5 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Wed, 15 Nov 2023 14:54:12 +0100 Subject: [PATCH 18/52] breaking changes example: airflow --- modules/ROOT/pages/release-notes.adoc | 71 ++++++++++++++++++++++++++- 1 file changed, 70 insertions(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 21f6a83d1..db8c8b101 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -33,7 +33,7 @@ Stackable Cockpit:: This release includes a very early preview version of Stackable Cockpit, a browser-based management tool which interacts with the Stackable data platform to display e.g. deployed stacklets and their status. stackablectl:: -Our command line tool has been re-worked as it is now the backbone for Stackable Cockpit: you can find out about the recent enhancements by visiting the online xref:management:stackablectl:index.adoc[documentation]. +Our command line tool has been re-worked to use the same backbone as Stackable Cockpit: you can find out about the recent enhancements by visiting the online xref:management:stackablectl:index.adoc[documentation]. Listener operator?:: @@ -151,8 +151,77 @@ Additionally, the `AirflowDB` custom resource is not used anymore. Those CRs sho These consolidated changes would require a change from e.g. +``` +--- +apiVersion: airflow.stackable.tech/v1alpha1 +kind: AirflowCluster +metadata: + name: airflow +spec: + image: + productVersion: "2.6.1" + stackableVersion: "23.7" # <1> + clusterConfig: + executor: CeleryExecutor # <2> + ... + databaseInitialization: # <3> + logging: + enableVectorAgent: false + containers: + ... + authenticationConfig: # <4> + authenticationClass: server-tls + userRegistrationRole: Admin + webservers: + ... + workers: # <5> + config: + logging: + enableVectorAgent: False + roleGroups: + default: + replicas: 1 +... +``` + to: +``` +--- +apiVersion: airflow.stackable.tech/v1alpha1 +kind: AirflowCluster +metadata: + name: airflow +spec: + image: + productVersion: "2.7.2" + #stackableVersion: "0.0.0-dev" + clusterConfig: + authentication: # <4> + - authenticationClass: server-tls + userRegistrationRole: Admin + webservers: + config: + logging: + enableVectorAgent: False + roleGroups: + default: + replicas: 1 + celeryExecutors: # <5> + config: + logging: + enableVectorAgent: False + roleGroups: + default: + replicas: 1 +... +``` + +<1> this field is no longer needed +<2> field removed +<3> section removed +<4> `authenticationConfig` replaced with a list of authentication classes +<5> workers replaced with either `celeryExecutors` or `kubernetesExecutors` ==== Stackable Operator for Apache Superset From ca9c7a57541d732f936f39370883fac7f1a17420 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Wed, 15 Nov 2023 16:04:17 +0100 Subject: [PATCH 19/52] airflow BC notes --- modules/ROOT/pages/release-notes.adoc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index db8c8b101..f57586843 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -195,7 +195,6 @@ metadata: spec: image: productVersion: "2.7.2" - #stackableVersion: "0.0.0-dev" clusterConfig: authentication: # <4> - authenticationClass: server-tls @@ -223,6 +222,8 @@ spec: <4> `authenticationConfig` replaced with a list of authentication classes <5> workers replaced with either `celeryExecutors` or `kubernetesExecutors` +N.B. the StatefulSets belonging to the Airflow cluster will need to be deleted before patching the Airflow resource and any existing AirflowDB job should be deleted as well. This will allow a database update to be followed through where necessary. + ==== Stackable Operator for Apache Superset * https://github.com/stackabletech/superset-operator/pull/396[Remove SupersetDB] `.spec.clusterConfig.loadExamplesOnInit` was removed from the CRD. Already loaded examples in Superset will not be removed by this change. Additionally, the `SupersetDB` custom resource is not used anymore. Those CRs should be deleted. Loading examples is still supported, the process is now described in the https://docs.stackable.tech/home/stable/superset/getting_started/first_steps[documentation]. From a8990ec1b2edae41a5f936d5e6f20c18b67e2365 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Wed, 15 Nov 2023 17:29:24 +0100 Subject: [PATCH 20/52] BC example: hdfs --- modules/ROOT/pages/release-notes.adoc | 47 ++++++++++++++++++++++++--- 1 file changed, 42 insertions(+), 5 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index f57586843..11f9fc7c8 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -216,7 +216,7 @@ spec: ... ``` -<1> this field is no longer needed +<1> this field is no longer needed if the product and operator image versions match <2> field removed <3> section removed <4> `authenticationConfig` replaced with a list of authentication classes @@ -224,6 +224,47 @@ spec: N.B. the StatefulSets belonging to the Airflow cluster will need to be deleted before patching the Airflow resource and any existing AirflowDB job should be deleted as well. This will allow a database update to be followed through where necessary. +==== Stackable Operator for Apache HDFS + +* https://github.com/stackabletech/hdfs-operator/pull/422[Removed field autoFormatFs] This field was not used. + +This requires a change from e.g. + +``` +--- +apiVersion: hdfs.stackable.tech/v1alpha1 +kind: HdfsCluster +metadata: + name: hdfs +spec: + image: + productVersion: "3.3.4" + stackableVersion: "23.7" # <1> + clusterConfig: + zookeeperConfigMapName: hdfs-zk + autoFormatFs: False # <2> + ... +``` + +to: + +``` +--- +apiVersion: hdfs.stackable.tech/v1alpha1 +kind: HdfsCluster +metadata: + name: hdfs +spec: + image: + productVersion: "3.3.4" + clusterConfig: + zookeeperConfigMapName: hdfs-zk + ... +``` + +<1> this field is no longer needed if the product and operator image versions match +<2> field removed + ==== Stackable Operator for Apache Superset * https://github.com/stackabletech/superset-operator/pull/396[Remove SupersetDB] `.spec.clusterConfig.loadExamplesOnInit` was removed from the CRD. Already loaded examples in Superset will not be removed by this change. Additionally, the `SupersetDB` custom resource is not used anymore. Those CRs should be deleted. Loading examples is still supported, the process is now described in the https://docs.stackable.tech/home/stable/superset/getting_started/first_steps[documentation]. @@ -235,10 +276,6 @@ N.B. the StatefulSets belonging to the Airflow cluster will need to be deleted b * https://github.com/stackabletech/kafka-operator/pull/621[Certificate conversion] The secret-operator now handles certificate conversion. This allows for the removal of the prepare init container but means that you can't configure the log level for this container anymore. You will need to remove the field `spec.brokers.config.logging.container.prepare` in case it is specified. -==== Stackable Operator for Apache HDFS - -* https://github.com/stackabletech/hdfs-operator/pull/422[Removed field autoFormatFs] This field was not used. - ==== Stackable Operator for Apache NiFi * https://github.com/stackabletech/nifi-operator/pull/498[AuthenticationClass references] Consolidated authentication config to a list of AuthenticationClasses From cb0316f62c6a7dc1c9926c18cf4b65a025382fab Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Wed, 15 Nov 2023 18:13:35 +0100 Subject: [PATCH 21/52] BC example: kafka --- modules/ROOT/pages/release-notes.adoc | 60 ++++++++++++++++++++++++--- 1 file changed, 54 insertions(+), 6 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 11f9fc7c8..c5fdfc035 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -265,16 +265,57 @@ spec: <1> this field is no longer needed if the product and operator image versions match <2> field removed -==== Stackable Operator for Apache Superset +==== Stackable Operator for Apache Kafka -* https://github.com/stackabletech/superset-operator/pull/396[Remove SupersetDB] `.spec.clusterConfig.loadExamplesOnInit` was removed from the CRD. Already loaded examples in Superset will not be removed by this change. Additionally, the `SupersetDB` custom resource is not used anymore. Those CRs should be deleted. Loading examples is still supported, the process is now described in the https://docs.stackable.tech/home/stable/superset/getting_started/first_steps[documentation]. +* https://github.com/stackabletech/kafka-operator/pull/621[Certificate conversion] The secret-operator now handles certificate conversion. This allows for the removal of the prepare init container but means that you can't configure the log level for this container anymore. You will need to remove the field `spec.brokers.config.logging.container.prepare` in case it is specified. -* https://github.com/stackabletech/superset-operator/pull/394[Rename service port name] - The service port name has been renamed from superset to http for consistency reasons. This change should normally not be breaking, as we only change the name, not the port. However, there might be some e.g. Ingresses that rely on the port name and need to be updated. +This requires a change from e.g. -==== Stackable Operator for Apache Kafka +``` +--- +apiVersion: kafka.stackable.tech/v1alpha1 +kind: KafkaCluster +metadata: + name: kafka +spec: + image: + productVersion: "3.4.0" + stackableVersion: "23.7" # <1> + clusterConfig: + ... + brokers: + config: + logging: + containers: + prepare: # <2> + console: + level: INFO + ... +... +``` -* https://github.com/stackabletech/kafka-operator/pull/621[Certificate conversion] The secret-operator now handles certificate conversion. This allows for the removal of the prepare init container but means that you can't configure the log level for this container anymore. You will need to remove the field `spec.brokers.config.logging.container.prepare` in case it is specified. +to: + +``` +--- +apiVersion: kafka.stackable.tech/v1alpha1 +kind: KafkaCluster +metadata: + name: kafka +spec: + image: + productVersion: "3.4.0" + clusterConfig: + ... + brokers: + config: + logging: + ... +... +``` + +<1> this field is no longer needed if the product and operator image versions match +<2> section removed for `prepare` container ==== Stackable Operator for Apache NiFi @@ -291,6 +332,13 @@ spec: * https://github.com/stackabletech/spark-k8s-operator/pull/277[Configuration structure] Refactored application roles to use CommonConfiguration structures from the operator framework. +==== Stackable Operator for Apache Superset + +* https://github.com/stackabletech/superset-operator/pull/396[Remove SupersetDB] `.spec.clusterConfig.loadExamplesOnInit` was removed from the CRD. Already loaded examples in Superset will not be removed by this change. Additionally, the `SupersetDB` custom resource is not used anymore. Those CRs should be deleted. Loading examples is still supported, the process is now described in the https://docs.stackable.tech/home/stable/superset/getting_started/first_steps[documentation]. + +* https://github.com/stackabletech/superset-operator/pull/394[Rename service port name] + The service port name has been renamed from superset to http for consistency reasons. This change should normally not be breaking, as we only change the name, not the port. However, there might be some e.g. Ingresses that rely on the port name and need to be updated. + ==== Stackable Operator for Trino * https://github.com/stackabletech/trino-operator/pull/491[New OPA Authorizer] Version 428 uses the https://github.com/bloomberg/trino/tree/add-open-policy-agent[new OPA authorizer] which requires changes to existing rego rules. From 815de900bc9cc1849d63105cc1405ed6f33fc6f8 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 16 Nov 2023 10:43:12 +0100 Subject: [PATCH 22/52] added admonition and collapsable code sections --- modules/ROOT/pages/release-notes.adoc | 66 ++++++++++++++++----------- 1 file changed, 39 insertions(+), 27 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index c5fdfc035..52a91287e 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -133,10 +133,10 @@ This release supports the following OpenShift versions: === Breaking changes -_*TODO*_: add before-and-after yamls to make this transparent - You will need to adapt your existing CRDs due to the following breaking changes detailed below. +NOTE: For all operators: the field `spec.image.stackableVersion` is no longer needed when the operator and product stackable versions match e.g. both are `23.7`. However, if the operator has been upgraded from e.g. `23.7` to `23.11`, and if the product image should _also_ be upgraded to the same _stackable_ version, then this field must be specified (otherwise the patched resource will not prompt for this). + ==== Stackable Operator for Apache Airflow * https://github.com/stackabletech/airflow-operator/pull/322[Removed AirflowDB] `.spec.clusterConfig.databaseInitialization` was removed from the CRD. This allowed configuring logging for the database initialization job, which does not exist anymore. Instead, database initialization is done by the scheduler pod, which has its own logging configuration. This was necessary to remove the `AirflowDB` Custom Resource, which caused problems when upgrading or reinstalling Airflow clusters. @@ -151,8 +151,11 @@ Additionally, the `AirflowDB` custom resource is not used anymore. Those CRs sho These consolidated changes would require a change from e.g. -``` ---- +.Breaking changes details +[%collapsible] +==== +[source,yaml] +---- apiVersion: airflow.stackable.tech/v1alpha1 kind: AirflowCluster metadata: @@ -181,20 +184,20 @@ spec: roleGroups: default: replicas: 1 -... -``` + ... +---- to: -``` ---- +[source,yaml] +---- apiVersion: airflow.stackable.tech/v1alpha1 kind: AirflowCluster metadata: name: airflow spec: image: - productVersion: "2.7.2" + productVersion: "2.6.1" clusterConfig: authentication: # <4> - authenticationClass: server-tls @@ -213,8 +216,8 @@ spec: roleGroups: default: replicas: 1 -... -``` + ... +---- <1> this field is no longer needed if the product and operator image versions match <2> field removed @@ -223,15 +226,19 @@ spec: <5> workers replaced with either `celeryExecutors` or `kubernetesExecutors` N.B. the StatefulSets belonging to the Airflow cluster will need to be deleted before patching the Airflow resource and any existing AirflowDB job should be deleted as well. This will allow a database update to be followed through where necessary. +==== ==== Stackable Operator for Apache HDFS * https://github.com/stackabletech/hdfs-operator/pull/422[Removed field autoFormatFs] This field was not used. +.Breaking changes details +[%collapsible] +==== This requires a change from e.g. -``` ---- +[source,yaml] +---- apiVersion: hdfs.stackable.tech/v1alpha1 kind: HdfsCluster metadata: @@ -244,12 +251,12 @@ spec: zookeeperConfigMapName: hdfs-zk autoFormatFs: False # <2> ... -``` +---- to: -``` ---- +[source,yaml] +---- apiVersion: hdfs.stackable.tech/v1alpha1 kind: HdfsCluster metadata: @@ -260,19 +267,23 @@ spec: clusterConfig: zookeeperConfigMapName: hdfs-zk ... -``` +---- <1> this field is no longer needed if the product and operator image versions match <2> field removed +==== ==== Stackable Operator for Apache Kafka * https://github.com/stackabletech/kafka-operator/pull/621[Certificate conversion] The secret-operator now handles certificate conversion. This allows for the removal of the prepare init container but means that you can't configure the log level for this container anymore. You will need to remove the field `spec.brokers.config.logging.container.prepare` in case it is specified. +.Breaking changes details +[%collapsible] +==== This requires a change from e.g. -``` ---- +[source,yaml] +---- apiVersion: kafka.stackable.tech/v1alpha1 kind: KafkaCluster metadata: @@ -291,13 +302,12 @@ spec: console: level: INFO ... -... -``` +---- to: -``` ---- +[source,yaml] +---- apiVersion: kafka.stackable.tech/v1alpha1 kind: KafkaCluster metadata: @@ -305,18 +315,20 @@ metadata: spec: image: productVersion: "3.4.0" + stackableVersion: "23.11" # <1> clusterConfig: ... brokers: config: logging: ... -... -``` - -<1> this field is no longer needed if the product and operator image versions match +---- +<1> this field is no longer needed if the product and operator image versions match: it is included here to illustrate how to force an update of the product image to `stackable23.11`. <2> section removed for `prepare` container +NOTE: for details about how Kafka uses a PVC to persist a reference to its ZNode, and how this may be relevant to upgrade scenarios, please read the documentation xref:kafka:usage-guide:operations:znode-id.adoc[here]. +==== + ==== Stackable Operator for Apache NiFi * https://github.com/stackabletech/nifi-operator/pull/498[AuthenticationClass references] Consolidated authentication config to a list of AuthenticationClasses From 789f9af61a3b7f034137bb71ce8c25bc77af0c8a Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 16 Nov 2023 11:15:52 +0100 Subject: [PATCH 23/52] corrected BC notes --- modules/ROOT/pages/release-notes.adoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 52a91287e..bac7d689a 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -135,7 +135,7 @@ This release supports the following OpenShift versions: You will need to adapt your existing CRDs due to the following breaking changes detailed below. -NOTE: For all operators: the field `spec.image.stackableVersion` is no longer needed when the operator and product stackable versions match e.g. both are `23.7`. However, if the operator has been upgraded from e.g. `23.7` to `23.11`, and if the product image should _also_ be upgraded to the same _stackable_ version, then this field must be specified (otherwise the patched resource will not prompt for this). +NOTE: For all operators: the field `spec.image.stackableVersion` is no longer needed when the operator and product stackable versions match e.g. both are `23.7`. If the operator has been upgraded from e.g. `23.7` to `23.11`, and if the product image should _also_ be upgraded to the same _stackable_ version, then this field can be omitted. ==== Stackable Operator for Apache Airflow @@ -315,7 +315,6 @@ metadata: spec: image: productVersion: "3.4.0" - stackableVersion: "23.11" # <1> clusterConfig: ... brokers: @@ -323,10 +322,11 @@ spec: logging: ... ---- -<1> this field is no longer needed if the product and operator image versions match: it is included here to illustrate how to force an update of the product image to `stackable23.11`. +<1> this field is no longer needed if the product and operator image versions match. <2> section removed for `prepare` container -NOTE: for details about how Kafka uses a PVC to persist a reference to its ZNode, and how this may be relevant to upgrade scenarios, please read the documentation xref:kafka:usage-guide:operations:znode-id.adoc[here]. +NOTE: for details about how Kafka uses a PVC to persist a reference to its ZNode, and how this may be relevant to upgrade scenarios, please read the documentation xref:kafka:usage-guide/operations/znode-id.adoc[here]. +xref:operators:ka ==== ==== Stackable Operator for Apache NiFi From 9ffd7bf9e4ea25b3ca0d4aef67711af01d046f8a Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 16 Nov 2023 11:27:32 +0100 Subject: [PATCH 24/52] minor typo --- modules/ROOT/pages/release-notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index bac7d689a..9501c9b40 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -326,7 +326,6 @@ spec: <2> section removed for `prepare` container NOTE: for details about how Kafka uses a PVC to persist a reference to its ZNode, and how this may be relevant to upgrade scenarios, please read the documentation xref:kafka:usage-guide/operations/znode-id.adoc[here]. -xref:operators:ka ==== ==== Stackable Operator for Apache NiFi @@ -337,6 +336,7 @@ xref:operators:ka * https://github.com/stackabletech/nifi-operator/pull/498[Remove redundant authenticaiton field] Removed crd support for `nifi.security.allow.anonymous.authentication` that was never used. + ==== Stackable Operator for Apache Spark * https://github.com/stackabletech/spark-k8s-operator/pull/275[Image specification] Use product image selection instead of version. From a49dc2eeb8ef316e3ee8b9c7faccfb197d4f91f9 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 16 Nov 2023 14:22:11 +0100 Subject: [PATCH 25/52] BC example: nifi --- modules/ROOT/pages/release-notes.adoc | 78 ++++++++++++++++++++++++++- 1 file changed, 77 insertions(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 9501c9b40..acd38be59 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -334,8 +334,68 @@ NOTE: for details about how Kafka uses a PVC to persist a reference to its ZNode * https://github.com/stackabletech/nifi-operator/pull/498[Remove credential generation] Removed crd support for the auto generation of admin credentials -* https://github.com/stackabletech/nifi-operator/pull/498[Remove redundant authenticaiton field] Removed crd support for `nifi.security.allow.anonymous.authentication` that was never used. +* https://github.com/stackabletech/nifi-operator/pull/498[Remove redundant authentication field] Removed crd support for the `nifi.security.allow.anonymous.authentication` property that was never used. +.Breaking changes details +[%collapsible] +==== +This requires a change from e.g. + +[source,yaml] +---- +--- +apiVersion: nifi.stackable.tech/v1alpha1 +kind: NifiCluster +metadata: + name: test-nifi +spec: + image: + productVersion: "1.21.0" + stackableVersion: "23.7" # <1> + clusterConfig: + zookeeperConfigMapName: test-zk + authentication: # <2> + allowAnonymousAccess: False # <3> + method: + singleUser: + adminCredentialsSecret: nifi-admin-credentials-simple + autoGenerate: False # <4> + sensitiveProperties: + keySecret: nifi-sensitive-property-key + nodes: + roleGroups: + default: + replicas: 1 +---- + +to: + +[source,yaml] +---- +--- +apiVersion: nifi.stackable.tech/v1alpha1 +kind: NifiCluster +metadata: + name: test-nifi +spec: + image: + productVersion: "1.23.2" + clusterConfig: + zookeeperConfigMapName: test-zk + authentication: # <2> + - authenticationClass: simple-nifi-users + sensitiveProperties: + keySecret: nifi-sensitive-property-key + nodes: + roleGroups: + default: + replicas: 1 +---- +<1> this field is no longer needed if the product and operator image versions match. +<2> this section has been changed to take a list of authentication classes +<3> functionality has been removed +<4> functionality has been removed +==== ==== Stackable Operator for Apache Spark @@ -344,6 +404,22 @@ NOTE: for details about how Kafka uses a PVC to persist a reference to its ZNode * https://github.com/stackabletech/spark-k8s-operator/pull/277[Configuration structure] Refactored application roles to use CommonConfiguration structures from the operator framework. +.Breaking changes details +[%collapsible] +==== +This requires a change from e.g. + +[source,yaml] +---- +---- + +to: + +[source,yaml] +---- +---- +==== + ==== Stackable Operator for Apache Superset * https://github.com/stackabletech/superset-operator/pull/396[Remove SupersetDB] `.spec.clusterConfig.loadExamplesOnInit` was removed from the CRD. Already loaded examples in Superset will not be removed by this change. Additionally, the `SupersetDB` custom resource is not used anymore. Those CRs should be deleted. Loading examples is still supported, the process is now described in the https://docs.stackable.tech/home/stable/superset/getting_started/first_steps[documentation]. From a7d869d87f163831c384dae9cca787206a62e736 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 16 Nov 2023 14:40:29 +0100 Subject: [PATCH 26/52] BC example: spark --- modules/ROOT/pages/release-notes.adoc | 58 ++++++++++++++++++++++++--- 1 file changed, 52 insertions(+), 6 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index acd38be59..c3f2e3380 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -269,8 +269,8 @@ spec: ... ---- -<1> this field is no longer needed if the product and operator image versions match -<2> field removed +<1> this field is no longer needed if the product and operator image versions match. +<2> field removed. ==== ==== Stackable Operator for Apache Kafka @@ -323,7 +323,7 @@ spec: ... ---- <1> this field is no longer needed if the product and operator image versions match. -<2> section removed for `prepare` container +<2> section removed for `prepare` container. NOTE: for details about how Kafka uses a PVC to persist a reference to its ZNode, and how this may be relevant to upgrade scenarios, please read the documentation xref:kafka:usage-guide/operations/znode-id.adoc[here]. ==== @@ -392,9 +392,9 @@ spec: replicas: 1 ---- <1> this field is no longer needed if the product and operator image versions match. -<2> this section has been changed to take a list of authentication classes -<3> functionality has been removed -<4> functionality has been removed +<2> this section has been changed to take a list of authentication classes. +<3> functionality has been removed. +<4> functionality has been removed. ==== ==== Stackable Operator for Apache Spark @@ -411,13 +411,59 @@ This requires a change from e.g. [source,yaml] ---- +apiVersion: spark.stackable.tech/v1alpha1 +kind: SparkApplication +metadata: + name: spark-examples +spec: + version: "1.0" + sparkImage: "docker.stackable.tech/stackable/spark-k8s:3.4.0-stackable23.7" # <1> + mode: cluster + mainClass: org.apache.spark.examples.SparkALS + mainApplicationFile: "local:///stackable/spark/examples/jars/spark-examples.jar" + job: # <2> + logging: + enableVectorAgent: False + driver: # <2> + logging: + enableVectorAgent: False + executor: # <2> + instances: 1 + logging: + enableVectorAgent: False ---- to: [source,yaml] ---- +apiVersion: spark.stackable.tech/v1alpha1 +kind: SparkApplication +metadata: + name: spark-examples-2 +spec: + version: "1.0" + sparkImage: # <1> + productVersion: "3.5.0" + mode: cluster + mainClass: org.apache.spark.examples.SparkALS + mainApplicationFile: "local:///stackable/spark/examples/jars/spark-examples.jar" + job: + config: # <2> + logging: + enableVectorAgent: False + driver: + config: # <2> + logging: + enableVectorAgent: False + executor: + replicas: 1 + config: # <2> + logging: + enableVectorAgent: False ---- +<1> this field has been changed to be consistent with product image selection, documented xref:concepts:product_image_selection.adoc[here]. +<2> this section has been changed to be consistent with common configuration definitions used for other operators. ==== ==== Stackable Operator for Apache Superset From c436dd71117d1fc83948725513fd36c13960c488 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 16 Nov 2023 15:40:14 +0100 Subject: [PATCH 27/52] BC example: superset --- modules/ROOT/pages/release-notes.adoc | 54 +++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index c3f2e3380..6d0a02861 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -473,6 +473,60 @@ spec: * https://github.com/stackabletech/superset-operator/pull/394[Rename service port name] The service port name has been renamed from superset to http for consistency reasons. This change should normally not be breaking, as we only change the name, not the port. However, there might be some e.g. Ingresses that rely on the port name and need to be updated. +.Breaking changes details +[%collapsible] +==== +This requires a change from e.g. + +[source,yaml] +---- +apiVersion: superset.stackable.tech/v1alpha1 +kind: SupersetCluster +metadata: + name: superset +spec: + image: + productVersion: "2.1.0" + stackableVersion: "23.7" # <1> + clusterConfig: + credentialsSecret: superset-credentials + loadExamplesOnInit: false # <2> + nodes: + config: + logging: + enableVectorAgent: False + roleGroups: + default: + replicas: 1 +---- + +to: + +[source,yaml] +---- +apiVersion: superset.stackable.tech/v1alpha1 +kind: SupersetCluster +metadata: + name: superset +spec: + image: + productVersion: "2.1.0" + clusterConfig: + credentialsSecret: superset-credentials + nodes: + config: + logging: + enableVectorAgent: False + roleGroups: + default: + replicas: 1 +---- +<1> this field is no longer needed if the product and operator image versions match. +<2> this field has been removed. + +NOTE: it will be necessary to remove the stateful sets before updating the Superset resource. +==== + ==== Stackable Operator for Trino * https://github.com/stackabletech/trino-operator/pull/491[New OPA Authorizer] Version 428 uses the https://github.com/bloomberg/trino/tree/add-open-policy-agent[new OPA authorizer] which requires changes to existing rego rules. From 4e9a038da4d483920004360f20c0ef91f79306e7 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 16 Nov 2023 20:31:21 +0100 Subject: [PATCH 28/52] BC example: trino --- modules/ROOT/pages/release-notes.adoc | 97 +++++++++++++++++++++++++++ 1 file changed, 97 insertions(+) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 6d0a02861..8b6e135df 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -531,6 +531,103 @@ NOTE: it will be necessary to remove the stateful sets before updating the Super * https://github.com/stackabletech/trino-operator/pull/491[New OPA Authorizer] Version 428 uses the https://github.com/bloomberg/trino/tree/add-open-policy-agent[new OPA authorizer] which requires changes to existing rego rules. +.Breaking changes details +[%collapsible] +==== +This requires a change from e.g. + +[source,yaml] +---- +apiVersion: trino.stackable.tech/v1alpha1 +kind: TrinoCluster +metadata: + name: trino +spec: + image: + productVersion: "414" + stackableVersion: "23.7" + clusterConfig: + ... +---- + +to: + +[source,yaml] +---- +--- +apiVersion: trino.stackable.tech/v1alpha1 +kind: TrinoCluster +metadata: + name: trino +spec: + image: + productVersion: "428" + stackableVersion: 23.11 + clusterConfig: + ... +--- +---- + +To adapt the rego rules to work with the new authorizer visit the documentation xref:trino:usage-guide/security.adoc[here], detailed under the section on "Authorization". For example, you can upgrade to the last version of Open Policy Agent and enter the relevant syntax in a ConfigMap like this: + +[source,yaml] +---- +--- +apiVersion: opa.stackable.tech/v1alpha1 +kind: OpaCluster +metadata: + name: opa +spec: + image: + productVersion: "0.57.0" + stackableVersion: 23.11 + servers: + roleGroups: + default: {} +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: simple-trino-opa-bundle + labels: + opa.stackable.tech/bundle: "trino" +data: + trino.rego: | + package trino + + import future.keywords.in + + default allow = false + + allow { + is_admin + } + extended[i] { + some i + input.action.filterResources[i] + is_admin + } + + allow { + input.action.operation in ["ExecuteQuery", "AccessCatalog"] + is_bob + } + extended[i] { + input.action.operation in ["FilterCatalogs"] + some i + input.action.filterResources[i] + is_bob + } + + is_admin() { + input.context.identity.user == "admin" + } + + is_bob() { + input.context.identity.user == "bob" + } +---- +==== === Upgrade from 23.7 From 23c4b6cc3edf1700678ef2398f6be4d7804371b8 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 16 Nov 2023 20:47:06 +0100 Subject: [PATCH 29/52] clarification note re. port names and stateful sets --- modules/ROOT/pages/release-notes.adoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 8b6e135df..6a21d1773 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -144,7 +144,7 @@ Additionally, the `AirflowDB` custom resource is not used anymore. Those CRs sho * https://github.com/stackabletech/airflow-operator/pull/311[Implement KubernetesExecutor] As an alternative to the CeleryExecutor we now also support the KubernetesExecutor, whereby pods are spun up for job tasks and are terminated afterwards. This removes the need a Redis job queue and offers an alternative approach to resource management. -* https://github.com/stackabletech/airflow-operator/pull/316[Rename service port name] The service port name has been renamed from airflow to http for consistency reasons. This change should normally not be breaking as only name has been changed, not the port. However, there might be some e.g. Ingresses that rely on the port name and need to be updated +* https://github.com/stackabletech/airflow-operator/pull/316[Rename service port name] The service port name has been renamed from airflow to http for consistency reasons. This change requires that the statefulset be removed before upgrading. There might also be some e.g. Ingresses that rely on the port name that will need to be updated. * https://github.com/stackabletech/airflow-operator/pull/303[AuthenticationClass references] `spec.clusterConfig.authenticationConfig` has been consolidated to `spec.clusterConfig.authentication` which takes a vector of AuthenticationClass references @@ -225,7 +225,7 @@ spec: <4> `authenticationConfig` replaced with a list of authentication classes <5> workers replaced with either `celeryExecutors` or `kubernetesExecutors` -N.B. the StatefulSets belonging to the Airflow cluster will need to be deleted before patching the Airflow resource and any existing AirflowDB job should be deleted as well. This will allow a database update to be followed through where necessary. +NOTE: it will be necessary to remove the stateful sets before updating the Airflow resource due to the change to the name of container port. Any existing AirflowDB jobs should be deleted as well. This will allow a database update to be followed through where necessary. ==== ==== Stackable Operator for Apache HDFS @@ -471,7 +471,7 @@ spec: * https://github.com/stackabletech/superset-operator/pull/396[Remove SupersetDB] `.spec.clusterConfig.loadExamplesOnInit` was removed from the CRD. Already loaded examples in Superset will not be removed by this change. Additionally, the `SupersetDB` custom resource is not used anymore. Those CRs should be deleted. Loading examples is still supported, the process is now described in the https://docs.stackable.tech/home/stable/superset/getting_started/first_steps[documentation]. * https://github.com/stackabletech/superset-operator/pull/394[Rename service port name] - The service port name has been renamed from superset to http for consistency reasons. This change should normally not be breaking, as we only change the name, not the port. However, there might be some e.g. Ingresses that rely on the port name and need to be updated. + The service port name has been renamed from superset to http for consistency reasons. This change requires that the statefulset be removed before upgrading. There might also be some e.g. Ingresses that rely on the port name that will need to be updated. .Breaking changes details [%collapsible] @@ -524,7 +524,7 @@ spec: <1> this field is no longer needed if the product and operator image versions match. <2> this field has been removed. -NOTE: it will be necessary to remove the stateful sets before updating the Superset resource. +NOTE: it will be necessary to remove the stateful sets before updating the Superset resource due to the change to the name of container port. ==== ==== Stackable Operator for Trino From 8910aecf9056008b852b4edaca4ae069be32df98 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Fri, 17 Nov 2023 11:06:08 +0100 Subject: [PATCH 30/52] listener-operator note --- modules/ROOT/pages/release-notes.adoc | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 6a21d1773..e80e04eac 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -35,7 +35,9 @@ This release includes a very early preview version of Stackable Cockpit, a brows stackablectl:: Our command line tool has been re-worked to use the same backbone as Stackable Cockpit: you can find out about the recent enhancements by visiting the online xref:management:stackablectl:index.adoc[documentation]. -Listener operator?:: +Listener operator:: +The listener-operator was introduced in release 23.1 and the associated ServiceType field in 23.4. In this release we introduce configurable ListenerClass _presets_ that map to the service types appropriate for different environments. This is discussed in more detail in the xref:listener:concepts/listenerclass.adoc[documentation]. + SBOMs?:: ... From 1bfe8e21be5f510b0285d58b18ce0cbdf9d9435d Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Fri, 17 Nov 2023 11:09:56 +0100 Subject: [PATCH 31/52] corrected xref --- modules/ROOT/pages/release-notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index e80e04eac..62192bcb9 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -36,7 +36,7 @@ stackablectl:: Our command line tool has been re-worked to use the same backbone as Stackable Cockpit: you can find out about the recent enhancements by visiting the online xref:management:stackablectl:index.adoc[documentation]. Listener operator:: -The listener-operator was introduced in release 23.1 and the associated ServiceType field in 23.4. In this release we introduce configurable ListenerClass _presets_ that map to the service types appropriate for different environments. This is discussed in more detail in the xref:listener:concepts/listenerclass.adoc[documentation]. +The listener-operator was introduced in release 23.1 and the associated ServiceType field in 23.4. In this release we introduce configurable ListenerClass _presets_ that map to the service types appropriate for different environments. This is discussed in more detail in the xref:listener:listenerclass.adoc[documentation]. SBOMs?:: From e54e801f43fe657f5bede126b31ce8d2e2f89bc3 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Fri, 17 Nov 2023 11:15:49 +0100 Subject: [PATCH 32/52] corrected xref II --- modules/ROOT/pages/release-notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 62192bcb9..8093d4243 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -36,7 +36,7 @@ stackablectl:: Our command line tool has been re-worked to use the same backbone as Stackable Cockpit: you can find out about the recent enhancements by visiting the online xref:management:stackablectl:index.adoc[documentation]. Listener operator:: -The listener-operator was introduced in release 23.1 and the associated ServiceType field in 23.4. In this release we introduce configurable ListenerClass _presets_ that map to the service types appropriate for different environments. This is discussed in more detail in the xref:listener:listenerclass.adoc[documentation]. +The listener-operator was introduced in release 23.1 and the associated ServiceType field in 23.4. In this release we introduce configurable ListenerClass _presets_ that map to the service types appropriate for different environments. This is discussed in more detail in the xref:listener-operator:listenerclass.adoc[documentation]. SBOMs?:: From d8cfb000d1fd0e3fa8b61a2fbcc3e8d9a0bf663f Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Fri, 17 Nov 2023 15:17:31 +0100 Subject: [PATCH 33/52] Update modules/ROOT/pages/release-notes.adoc Co-authored-by: Malte Sander --- modules/ROOT/pages/release-notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 8093d4243..abd8fd703 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -52,7 +52,7 @@ The following new product versions are now supported: * https://github.com/stackabletech/hdfs-operator/pull/409[HDFS: 3.2.4, 3.3.6] * https://github.com/stackabletech/kafka-operator/pull/627[Kafka: 2.8.2, 3.4.1, 3.5.1] * https://github.com/stackabletech/nifi-operator/pull/513[NiFi: 1.23.2] -* https://github.com/stackabletech/opa-operator/pull/482[OpenPolicyAgent: 0.57] +* https://github.com/stackabletech/opa-operator/pull/482[OpenPolicyAgent: 0.57.0] * https://github.com/stackabletech/spark-k8s-operator/pull/291[Spark: 3.4.1, 3.5.0] * https://github.com/stackabletech/superset-operator/pull/415[Superset: 2.1.1, 3.0.1] * https://github.com/stackabletech/trino-operator/pull/491[Trino: 428] From 46d4fa2af4f0b509552c2191ef68ac25786f7e7e Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Fri, 17 Nov 2023 15:17:47 +0100 Subject: [PATCH 34/52] Update modules/ROOT/pages/release-notes.adoc Co-authored-by: Malte Sander --- modules/ROOT/pages/release-notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index abd8fd703..88c7baf4d 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -69,7 +69,7 @@ The following product versions are deprecated and will be removed in a later rel * OpenPolicyAgent: 0.51.0 * Spark: 3.4.0 * Superset: 2.1.0 -* ZooKeeper: 381 +* ZooKeeper: 3.8.1 N.B. in some cases a newly supported version is also immediately marked as deprecated. This is done to allow an update path from the latest patch of a minor version (e.g. Kafka 2.8.2 --> 3.4.1). From cda4f1b3cbbd2123dfb1ddfc9cc95bd289a685ff Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Fri, 17 Nov 2023 15:17:56 +0100 Subject: [PATCH 35/52] Update modules/ROOT/pages/release-notes.adoc Co-authored-by: Malte Sander --- modules/ROOT/pages/release-notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 88c7baf4d..52d57fdec 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -100,7 +100,7 @@ Additionally, there are some other individual product features that are notewort * https://github.com/stackabletech/docker-images/pull/505[Hive: update postgresql driver to support SCRAM authentication] * https://github.com/stackabletech/docker-images/pull/436[Spark: all product images contain pyspark] * https://github.com/stackabletech/trino-operator/issues/491[Trino: support for the new OPA authorizer] -* https://github.com/stackabletech/issues/issues/444[Vector: upgrade to 0.33] +* https://github.com/stackabletech/issues/issues/444[Vector: upgrade to 0.33.0] === stackablectl From f02a333c842418fd328702e2c336e7544c3df9b6 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Fri, 17 Nov 2023 16:47:57 +0100 Subject: [PATCH 36/52] trimmed down examples --- modules/ROOT/pages/release-notes.adoc | 17 ++--------------- 1 file changed, 2 insertions(+), 15 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 52d57fdec..d7f6074dd 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -180,9 +180,6 @@ spec: webservers: ... workers: # <5> - config: - logging: - enableVectorAgent: False roleGroups: default: replicas: 1 @@ -205,16 +202,10 @@ spec: - authenticationClass: server-tls userRegistrationRole: Admin webservers: - config: - logging: - enableVectorAgent: False roleGroups: default: replicas: 1 celeryExecutors: # <5> - config: - logging: - enableVectorAgent: False roleGroups: default: replicas: 1 @@ -494,12 +485,10 @@ spec: credentialsSecret: superset-credentials loadExamplesOnInit: false # <2> nodes: - config: - logging: - enableVectorAgent: False roleGroups: default: replicas: 1 + ... ---- to: @@ -516,12 +505,10 @@ spec: clusterConfig: credentialsSecret: superset-credentials nodes: - config: - logging: - enableVectorAgent: False roleGroups: default: replicas: 1 + ... ---- <1> this field is no longer needed if the product and operator image versions match. <2> this field has been removed. From 83f80ffe0f3e1dc85eeb9bb7debdba1fe8de9263 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Fri, 17 Nov 2023 16:53:55 +0100 Subject: [PATCH 37/52] nifi BC: added authentication class example --- modules/ROOT/pages/release-notes.adoc | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index d7f6074dd..d26c2df06 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -376,18 +376,36 @@ spec: clusterConfig: zookeeperConfigMapName: test-zk authentication: # <2> - - authenticationClass: simple-nifi-users + - authenticationClass: nifi-users # <5> sensitiveProperties: keySecret: nifi-sensitive-property-key nodes: roleGroups: default: replicas: 1 +--- +apiVersion: authentication.stackable.tech/v1alpha1 +kind: AuthenticationClass +metadata: + name: nifi-users # <5> +spec: + provider: + static: + userCredentialsSecret: + name: nifi-admin-credentials +--- +apiVersion: v1 +kind: Secret +metadata: + name: nifi-admin-credentials +stringData: + admin: supersecretpassword ---- <1> this field is no longer needed if the product and operator image versions match. <2> this section has been changed to take a list of authentication classes. <3> functionality has been removed. <4> functionality has been removed. +<5> the authentication class referenced by the NiFi cluster. ==== ==== Stackable Operator for Apache Spark From 0c1016839aa2aa1b2569451831d4aeb9e479db53 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Fri, 17 Nov 2023 16:55:47 +0100 Subject: [PATCH 38/52] product version headers --- modules/ROOT/pages/release-notes.adoc | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index d26c2df06..b9fe54f74 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -42,7 +42,9 @@ The listener-operator was introduced in release 23.1 and the associated ServiceT SBOMs?:: ... -New Versions:: +=== Product Versions + +==== New Versions The following new product versions are now supported: @@ -58,7 +60,7 @@ The following new product versions are now supported: * https://github.com/stackabletech/trino-operator/pull/491[Trino: 428] * https://github.com/stackabletech/zookeeper-operator/pull/732[ZooKeeper: 3.8.3] -Deprecated Versions:: +==== Deprecated Versions The following product versions are deprecated and will be removed in a later release: @@ -73,7 +75,7 @@ The following product versions are deprecated and will be removed in a later rel N.B. in some cases a newly supported version is also immediately marked as deprecated. This is done to allow an update path from the latest patch of a minor version (e.g. Kafka 2.8.2 --> 3.4.1). -Removed Versions:: +==== Removed Versions The following product versions are no longer supported (although images for released product versions remain available https://repo.stackable.tech/#browse/browse:docker:v2%2Fstackable[here]): From b6c1cbba80ef2ec7507915beca8b6322ba5c2c86 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Fri, 17 Nov 2023 17:00:41 +0100 Subject: [PATCH 39/52] moved product feature section --- modules/ROOT/pages/release-notes.adoc | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index b9fe54f74..5e20d62d2 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -42,6 +42,18 @@ The listener-operator was introduced in release 23.1 and the associated ServiceT SBOMs?:: ... +Product features:: + +Additionally, there are some other individual product features that are noteworthy: + +* https://github.com/stackabletech/docker-images/pull/494[HBase: support for Hadoop native compression] +* https://github.com/stackabletech/docker-images/pull/497[HBase: add operator tools to product image] +* https://github.com/stackabletech/docker-images/pull/400[HDFS: support for FUSE] +* https://github.com/stackabletech/docker-images/pull/505[Hive: update postgresql driver to support SCRAM authentication] +* https://github.com/stackabletech/docker-images/pull/436[Spark: all product images contain pyspark] +* https://github.com/stackabletech/trino-operator/issues/491[Trino: support for the new OPA authorizer] +* https://github.com/stackabletech/issues/issues/444[Vector: upgrade to 0.33.0] + === Product Versions ==== New Versions @@ -92,18 +104,6 @@ The following product versions are no longer supported (although images for rele * Trino: 377, 387, 395, 396, 403 * ZooKeeper: 3.5.8, 3.6.3, 3.7.0, 3.8.0 -Product features:: - -Additionally, there are some other individual product features that are noteworthy: - -* https://github.com/stackabletech/docker-images/pull/494[HBase: support for Hadoop native compression] -* https://github.com/stackabletech/docker-images/pull/497[HBase: add operator tools to product image] -* https://github.com/stackabletech/docker-images/pull/400[HDFS: support for FUSE] -* https://github.com/stackabletech/docker-images/pull/505[Hive: update postgresql driver to support SCRAM authentication] -* https://github.com/stackabletech/docker-images/pull/436[Spark: all product images contain pyspark] -* https://github.com/stackabletech/trino-operator/issues/491[Trino: support for the new OPA authorizer] -* https://github.com/stackabletech/issues/issues/444[Vector: upgrade to 0.33.0] - === stackablectl The following demo has been added to `stackablectl`: From 6a6218c9ca22c430fb5ef8709484a6699b006721 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Fri, 17 Nov 2023 17:25:00 +0100 Subject: [PATCH 40/52] provisional OS version support --- modules/ROOT/pages/release-notes.adoc | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 5e20d62d2..473575863 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -131,9 +131,8 @@ This Kubernetes version is no longer supported: This release supports the following OpenShift versions: -* `4.xx` _*TODO*_ - -... +* `4.13` _*CHECK*_ +* `4.12` === Breaking changes From 7e23940a8e57bc27a2455bb9076bf61afe3eaa6d Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Mon, 20 Nov 2023 14:41:00 +0100 Subject: [PATCH 41/52] remove SBOM from higlights --- modules/ROOT/pages/release-notes.adoc | 4 ---- 1 file changed, 4 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 473575863..3b177797f 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -38,10 +38,6 @@ Our command line tool has been re-worked to use the same backbone as Stackable C Listener operator:: The listener-operator was introduced in release 23.1 and the associated ServiceType field in 23.4. In this release we introduce configurable ListenerClass _presets_ that map to the service types appropriate for different environments. This is discussed in more detail in the xref:listener-operator:listenerclass.adoc[documentation]. - -SBOMs?:: -... - Product features:: Additionally, there are some other individual product features that are noteworthy: From 5ce1336b3d7487439ca7956421911b76c7eec45f Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Mon, 27 Nov 2023 10:56:03 +0100 Subject: [PATCH 42/52] removed unnecessary stackableVersion fields --- modules/ROOT/pages/release-notes.adoc | 2 -- 1 file changed, 2 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 3b177797f..5a08bc89a 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -549,7 +549,6 @@ metadata: spec: image: productVersion: "414" - stackableVersion: "23.7" clusterConfig: ... ---- @@ -566,7 +565,6 @@ metadata: spec: image: productVersion: "428" - stackableVersion: 23.11 clusterConfig: ... --- From c75c92db13b057461b13b620aa78d5010c42ac0c Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 30 Nov 2023 13:14:13 +0100 Subject: [PATCH 43/52] added link to signing tutorial --- modules/ROOT/pages/release-notes.adoc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 5a08bc89a..716cf053f 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -21,7 +21,8 @@ Graceful shutdown refers to the managed, controlled shutdown of service instance documentation] for more details. Signed SDP product images:: -As of this release all Stackable product images are signed (the signing of operator images was delivered in SDP 23.7). +As of this release all Stackable product images are signed (the signing of operator images was delivered in SDP 23.7). Please see this +xref:tutorials:enabling_verification_of_image_signatures.adoc[tutorial] for more information. Airflow KubernetesExecutor:: Airflow clusters can now be configured to use Kubernetes executors, whereby pods are spun up for job tasks and terminated when complete, thus offering an alternative way to use resources without the need for job queuing. From a5cb95b4d36d23d79be5a262da30ee79873761f9 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 30 Nov 2023 13:44:50 +0100 Subject: [PATCH 44/52] OS/k8s versions --- modules/ROOT/pages/release-notes.adoc | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 716cf053f..9cb9d9e05 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -115,7 +115,7 @@ This link lists the xref:demos:index.adoc[available demos]. This release supports the following Kubernetes versions: -* `1.28` _*CHECK*_ +* `1.28` * `1.27` * `1.26` @@ -128,8 +128,9 @@ This Kubernetes version is no longer supported: This release supports the following OpenShift versions: -* `4.13` _*CHECK*_ +* `4.13` * `4.12` +* `4.11` === Breaking changes From f0d911ebe471575e19ea2ca9e861610fe3f74eca Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 30 Nov 2023 13:47:52 +0100 Subject: [PATCH 45/52] Update modules/ROOT/pages/release-notes.adoc Co-authored-by: Malte Sander --- modules/ROOT/pages/release-notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 9cb9d9e05..5248f5668 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -119,7 +119,7 @@ This release supports the following Kubernetes versions: * `1.27` * `1.26` -This Kubernetes version is no longer supported: +These Kubernetes versions are no longer supported: * `1.25` * `1.24` From 184e6b70be6d5b3c8ef181ec79219f2eb06f94c3 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 30 Nov 2023 13:48:01 +0100 Subject: [PATCH 46/52] Update modules/ROOT/pages/release-notes.adoc Co-authored-by: Malte Sander --- modules/ROOT/pages/release-notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 5248f5668..2976258a2 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -143,7 +143,7 @@ NOTE: For all operators: the field `spec.image.stackableVersion` is no longer ne * https://github.com/stackabletech/airflow-operator/pull/322[Removed AirflowDB] `.spec.clusterConfig.databaseInitialization` was removed from the CRD. This allowed configuring logging for the database initialization job, which does not exist anymore. Instead, database initialization is done by the scheduler pod, which has its own logging configuration. This was necessary to remove the `AirflowDB` Custom Resource, which caused problems when upgrading or reinstalling Airflow clusters. Additionally, the `AirflowDB` custom resource is not used anymore. Those CRs should be deleted. -* https://github.com/stackabletech/airflow-operator/pull/311[Implement KubernetesExecutor] As an alternative to the CeleryExecutor we now also support the KubernetesExecutor, whereby pods are spun up for job tasks and are terminated afterwards. This removes the need a Redis job queue and offers an alternative approach to resource management. +* https://github.com/stackabletech/airflow-operator/pull/311[Implement KubernetesExecutor] As an alternative to the CeleryExecutor we now also support the KubernetesExecutor, whereby pods are spun up for job tasks and are terminated afterwards. This removes the need for a Redis job queue and offers an alternative approach to resource management. * https://github.com/stackabletech/airflow-operator/pull/316[Rename service port name] The service port name has been renamed from airflow to http for consistency reasons. This change requires that the statefulset be removed before upgrading. There might also be some e.g. Ingresses that rely on the port name that will need to be updated. From 28d900163ed6b394cee676986c2c95eb885a50da Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 30 Nov 2023 13:48:15 +0100 Subject: [PATCH 47/52] Update modules/ROOT/pages/release-notes.adoc Co-authored-by: Malte Sander --- modules/ROOT/pages/release-notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 2976258a2..de67f21ef 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -148,7 +148,7 @@ Additionally, the `AirflowDB` custom resource is not used anymore. Those CRs sho * https://github.com/stackabletech/airflow-operator/pull/316[Rename service port name] The service port name has been renamed from airflow to http for consistency reasons. This change requires that the statefulset be removed before upgrading. There might also be some e.g. Ingresses that rely on the port name that will need to be updated. * https://github.com/stackabletech/airflow-operator/pull/303[AuthenticationClass references] -`spec.clusterConfig.authenticationConfig` has been consolidated to `spec.clusterConfig.authentication` which takes a vector of AuthenticationClass references +`spec.clusterConfig.authenticationConfig` has been consolidated to `spec.clusterConfig.authentication` which takes a vector of AuthenticationClass references. These consolidated changes would require a change from e.g. From c6e5974d7aa8759eeb248f5660f1131ad65343fb Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 30 Nov 2023 13:48:24 +0100 Subject: [PATCH 48/52] Update modules/ROOT/pages/release-notes.adoc Co-authored-by: Malte Sander --- modules/ROOT/pages/release-notes.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index de67f21ef..e433dce56 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -322,9 +322,9 @@ NOTE: for details about how Kafka uses a PVC to persist a reference to its ZNode ==== Stackable Operator for Apache NiFi -* https://github.com/stackabletech/nifi-operator/pull/498[AuthenticationClass references] Consolidated authentication config to a list of AuthenticationClasses +* https://github.com/stackabletech/nifi-operator/pull/498[AuthenticationClass references] Consolidated authentication config to a list of AuthenticationClasses. -* https://github.com/stackabletech/nifi-operator/pull/498[Remove credential generation] Removed crd support for the auto generation of admin credentials +* https://github.com/stackabletech/nifi-operator/pull/498[Remove credential generation] Removed crd support for the auto generation of admin credentials. * https://github.com/stackabletech/nifi-operator/pull/498[Remove redundant authentication field] Removed crd support for the `nifi.security.allow.anonymous.authentication` property that was never used. From 8490d00bb0ba67243b09a93885a5b89fa91eb4de Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 30 Nov 2023 13:52:02 +0100 Subject: [PATCH 49/52] review comments --- modules/ROOT/pages/release-notes.adoc | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index e433dce56..07f0e33c4 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -572,7 +572,7 @@ spec: --- ---- -To adapt the rego rules to work with the new authorizer visit the documentation xref:trino:usage-guide/security.adoc[here], detailed under the section on "Authorization". For example, you can upgrade to the last version of Open Policy Agent and enter the relevant syntax in a ConfigMap like this: +To adapt the rego rules to work with the new authorizer visit the documentation xref:trino:usage-guide/security.adoc#_authorization [here]. For example, you can upgrade to the last version of Open Policy Agent and enter the relevant syntax in a ConfigMap like this: [source,yaml] ---- @@ -599,8 +599,6 @@ data: trino.rego: | package trino - import future.keywords.in - default allow = false allow { From 1f7e0d39ffb82ee0ca70fac02ac73339af317c3f Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 30 Nov 2023 13:55:23 +0100 Subject: [PATCH 50/52] review comment --- modules/ROOT/pages/release-notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 07f0e33c4..965f400e3 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -136,7 +136,7 @@ This release supports the following OpenShift versions: You will need to adapt your existing CRDs due to the following breaking changes detailed below. -NOTE: For all operators: the field `spec.image.stackableVersion` is no longer needed when the operator and product stackable versions match e.g. both are `23.7`. If the operator has been upgraded from e.g. `23.7` to `23.11`, and if the product image should _also_ be upgraded to the same _stackable_ version, then this field can be omitted. +NOTE: For all operators: the field `spec.image.stackableVersion` is no longer needed when the operator and product stackable versions match e.g. both are `23.11`. If this field has been _previously_ set this will *not* happen: in such cases `spec.image.stackableVersion` should be set explicitly, either to a release version or to `null`. ==== Stackable Operator for Apache Airflow From 23824297fa268af0b4040a7fbd2c66215b13e122 Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 30 Nov 2023 14:08:11 +0100 Subject: [PATCH 51/52] reverted import change --- modules/ROOT/pages/release-notes.adoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/modules/ROOT/pages/release-notes.adoc b/modules/ROOT/pages/release-notes.adoc index 965f400e3..2b4e2773c 100644 --- a/modules/ROOT/pages/release-notes.adoc +++ b/modules/ROOT/pages/release-notes.adoc @@ -599,6 +599,8 @@ data: trino.rego: | package trino + import future.keywords.in + default allow = false allow { From 266012c91de2485e5d3b007fb0e2ec43684a093e Mon Sep 17 00:00:00 2001 From: Andrew Kenworthy Date: Thu, 30 Nov 2023 14:27:00 +0100 Subject: [PATCH 52/52] updated k8s versions --- modules/ROOT/partials/supported-kubernetes-versions.adoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/modules/ROOT/partials/supported-kubernetes-versions.adoc b/modules/ROOT/partials/supported-kubernetes-versions.adoc index 93d339bc4..ab4a70570 100644 --- a/modules/ROOT/partials/supported-kubernetes-versions.adoc +++ b/modules/ROOT/partials/supported-kubernetes-versions.adoc @@ -2,6 +2,6 @@ // This is a separate file to refer to the version at multiple places // in the documentation -- 1.26 -- 1.25 -- 1.24 \ No newline at end of file +- 1.28 +- 1.27 +- 1.26 \ No newline at end of file