Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Kafka template should support configuration of replication factor params #64

Closed
jpechane opened this issue Oct 20, 2017 · 3 comments
Closed

Kafka template should support configuration of replication factor params #64

jpechane opened this issue Oct 20, 2017 · 3 comments
Labels
enhancement kafka broker kafka connect
Milestone
0.1.0

Comments

@jpechane
Copy link
Contributor

As it is possible to configure number of replicas for Kafka it would be also useful to configure default replication factors for

offsets.topic.replication.factor
transaction.state.log.replication.factor
config.storage.replication.factor
offset.storage.replication.factor
status.storage.replication.factor

default.replication.factor

The default value for the first 5 is 3, for the last one is 1. So I propose to provide two parameters - one for the system topics and one for the default. The reason is that if you dpeloy cluster with less than 3 brokers (typically one for dev environment) it is by definition non-functional as the settings require at leas 3 replicas.
@jpechane jpechane mentioned this issue Oct 20, 2017
Closed
@scholzj
Copy link
Member

scholzj commented Oct 20, 2017

@jpechane I'm a bit confused since my understanding is that some of these options are in Kafka:

  • offsets.topic.replication.factor
  • transaction.state.log.replication.factor
  • default.replication.factor

And some are in Kafka Connect:

  • config.storage.replication.factor
  • offset.storage.replication.factor
  • status.storage.replication.factor

Otherwise the requirement is valid. I had run into this kind of problem my self. So what different configuration parameters would you need? What about this?

  • 1 parameter for Kafka Connect to configure all three factors
  • 1 parameter for Kafka to set the default.replication.factor
  • 1 parameter for Kafka to set offsets.topic.replication.factor and transaction.state.log.replication.factor

That should probably cover it for you, right?

@jpechane
Copy link
Contributor Author

I am sorry for the confusion. Yes, you are right I've mixed both Kafka and Connect together. I agree with your propal as it makes sens to group the options as described.

@scholzj scholzj mentioned this issue Oct 23, 2017
Merged
@ppatierno ppatierno changed the title Kafka template should support confiugration of replication factor params Kafka template should support configuration of replication factor params Oct 23, 2017
scholzj added a commit to scholzj/strimzi-kafka-operator that referenced this issue Oct 23, 2017
@scholzj
Add parameters to configure the replication factors - Fixes strimzi#64
966d692 
Make a separate parameter for each configuration field. strimzi#64
@scholzj
Copy link
Member

scholzj commented Oct 23, 2017

After some further discussion in the PR, we decided to use separate parameter for each Kafka / Kafka Connect option.

@scholzj scholzj added this to the 0.1.0 milestone Nov 8, 2017
tomncooper pushed a commit to tomncooper/strimzi-kafka-operator that referenced this issue May 11, 2020
@ppatierno
Fixed the testNewRebalance test (strimzi#64)
20e8900 
* Fixed the testNewRebalance test

Signed-off-by: Paolo Patierno <ppatierno@live.com>

* Fixed comment

Signed-off-by: Paolo Patierno <ppatierno@live.com>
samuel-hawker pushed a commit that referenced this issue Jul 11, 2022
@dalelane @jhughes24816
parent 5eb4fb3
9671c6f 
author Dale Lane <Dale.Lane@uk.ibm.com> 1574169185 +0000
committer Samuel Hawker <samuel.hawker@ibm.com> 1651571459 +0100

chore: squash commits for ES releases up to 10.4.0
  * Contributes to mhub/qp-planning#7089

  * chore: squash commits for 10.0.0 ES release
    * Contributes to: mhub/qp-planning#6997

  * feat: Support for UBI-minimal as a base layer for Strimzi images (#4)
    * This adds UBI support as an optional base-layer when building the
    * Strimzi Kafka and cluster-operator Docker images.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * chore: update stunnel version (#6)
    * The previous stunnel version (5.55) has been updated and removed from
    * Stunnel's archive, so we need to update it to 5.56 in order to properly
    * build the Strimzi images.
    * Contributes to: mhub/qp-planning#4417
    * Signed-off-by: Stelios Gkiokas <Stylianos.Gkiokas@ibm.com>

  * feat: Build custom kafka binary image (#7) (#8)
    * Contributes to: mhub/qp-planning#4366
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * chore: add script to update fork (#10)
    * This script will be called by Jenkins in order to update the forked
    * repo.
    * Contributes to: mhub/qp-planning#4366
    * Signed-off-by: Stelios Gkiokas <Stylianos.Gkiokas@ibm.com>

  * feat: build custom Kafka (#12)
    * Contributes to: mhub/qp-planning#4366
    * Signed-off-by: Stelios Gkiokas <Stylianos.Gkiokas@ibm.com>

  * chore: pull stunnel from Artifactory (#13)
    * Contributes to: mhub/qp-planning#4416
    * Signed-off-by: Stelios Gkiokas <Stylianos.Gkiokas@ibm.com>

  * feat: move Strimzi artifact to Artifactory (#14)
    * Contributes to: mhub/qp-planning#4366
    * Signed-off-by: Stelios Gkiokas <Stylianos.Gkiokas@ibm.com>

  * fix: Add tini to ES UBI images (#15)
    * Add backup file and cleanup function to prevent
    * eventstreams-kafka-versions.yaml from being deleted
    * Contributes to: mhub/qp-planning#4447
    * Signed-off-by: Tom Jefferson <thomas.jefferson1@ibm.com>

  * feat: Fix stunnel copy (#16)
    * Contributes to: mhub/qp-planning#4511
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: stunnel fix (#18)
    * Contributes to: mhub/qp-planning#4511
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Add stunnel to bin (#19)
    * Contributes to: mhub/qp-planning#4511
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Change mode (#20)
    * Contributes to: mhub/qp-planning#0
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: eventstreams strimzi deploy (#21)
    * Contributes to: mhub/qp-planning#5411
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * chore: Update kafka-vnext image tag (#22)
    * Update strimzi-kafka-operator local build script to pull in the tarball
    * containing the latest build of the interceptor framework and producer
    * interceptor.
    * Contributes to: mhub/qp-planning#4475
    * Signed-off-by: Tom Aley <thomas.aley@ibm.com>

  * chore: add imagePullSecret to ES Operator (#23)
    * Installing the Strimzi Operator is currently failing since there is no
    * imagePullSecret set to pull the images for the ES Operator.
    * Contributes to: mhub/qp-planning#4540
    * Contributes to: mhub/qp-planning#4469
    * Signed-off-by: Stelios Gkiokas <Stylianos.Gkiokas@ibm.com>

  * test: check checksum (#24)
    * Test new kafka image
    * Signed-off-by: Julian Goh <julian.goh@uk.ibm.com>

  * fix: position ips in correct place (#25)
    * Contributes to: mhub/qp-planning#4540
    * Signed-off-by: Stelios Gkiokas <Stylianos.Gkiokas@ibm.com>

  * chore: upgrade operator kafka image to 2.4.0 with interceptor (#28)
    * Contributes to: mhub/qp-planning#4639
    * Signed-off-by: Steve Dare <steve.dare@ibm.com>

  * chore: Update to Kafka 2.4.0 (part 2) (#29)
    * Finishes the work for updating to the new Kafka image with
    * the interceptors support.
    * Contributes to: mhub/qp-planning#4636
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: build ubi images on openjdk (#27)
    * make is already installed as a dependency of openssl so shouldn't be
    * re-installed and then deleted as a build dep for stunnel
    * Contributes to: mhub/qp-planning#4550
    * Signed-off-by: Dave Lane <davilane@uk.ibm.com>

  *  fix: local build, new image tag env var
    * ensure docker build args have default values
    * use correct image tag
    * Contributes to: mhub/qp-planning#4550
    * Signed-off-by: Dave Lane <davilane@uk.ibm.com>

  * chore: Update Kafka/ZooKeeper image tags
    * I've also removed the STRIMZI_DEFAULT_ZOOKEEPER_IMAGE environment
    * variable as it hasn't been used in Strimzi for a while now and
    * has no impact.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * chore: rename image pull secret (#35)
    * regcred to ibm-entitlement-key
    * Contributes to: mhub/qp-planning#4640
    * Signed-off-by: Steve Dare <steve.dare@ibm.com>

  * chore: Update image tags
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: add UBI version of JMX Dockerfile (#37)
    * We need a UBI version of the Dockerfile user for the JMX trans.
    * Contributes to: mhub/qp-planning#4739
    * Signed-off-by: Stelios Gkiokas <Stylianos.Gkiokas@ibm.com>

  * chore: Update image tags
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * chore: Update version of ZooKeeper to match upstream (#38)
    * This updates us to ZooKeeper 3.5.6 which brings us in-line with the
    * upstream Strimzi project.
    * Closes: mhub/qp-planning#4743
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * Eventstreams domain delta and resource group (#39)
    * Labels and annotations
    * Contributes to: mhub/qp-planning#4618
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * Combined eventstreams operator (#41)
    * This commit ports the code from mhub/qp-eventstreams-operator
    * into its own submodule in the strimzi project fork
    * Contributes to: mhub/qp-planning#4618
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Fix tests (#42)
    * Contributes to: mhub/qp-planning#4618
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Move install folder
    * Contributes to: mhub/qp-planning#4618
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Try this (#47)
    * Contributes to: mhub/qp-planning#0
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Remove deprecated future() calls (#44)
    * Contributes to: mhub/qp-planning#4554
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>
    * Signed-off-by: Katherine Stanley <katheris@uk.ibm.com>

  * feat: Fixes to install and examples (#45)
    * Contributes to: mhub/qp-planning#4554
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Uniquely name kube resources (#48)
    * Contributes to: mhub/qp-planning#0
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: es makefile for rapid development
    * this commit adds and es makefile to allow for rapid
    * development.
    * once changes have been made to the eventstreams cluster
    * operator, running `make eventstreams_operator_build` will
    * rapidly build the jar and docker image for the operator
    * Contributes to: mhub/qp-planning#4616
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * feat: Build init container (#46)
    * Contributes to: mhub/qp-planning#0
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * fix: fix dep analyze and re-enable on build (#53)
    * * fix: fix dep analyze and re-enable on build
    * * fix: remove unnecessary dependencies
    * Contributes to: mhub/qp-planning#4616
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * feat: Add eventstreams checkstyle files (#54)
    * Contributes to mhub/qp-planning#4616
    * Signed-off-by: Katherine Stanley <katheris@uk.ibm.com>

  * fix: update snapshot to latest version (#55)
    * Contributes to: mhub/qp-planning#4616
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * feat: Update integration test install location (#56)
    * Contributes to mhub/qp-planning#4616
    * Signed-off-by: Katherine Stanley <katheris@uk.ibm.com>

  * fix: fix dependencies in pom (#57)
    * also delete unwanted file
    * Contributes to: mhub/qp-planning#4616
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * fix: correct role name (#59)
    * Contributes to: mhub/qp-planning#0
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Add failure handling to chain promise (#60)
    * Add failure handling to chain promise and
    * correctly fail async unit tests.
    * Contributes to mhub/qp-planning#4795
    * Signed-off-by: Katherine Stanley <katheris@uk.ibm.com>

  * feat: Update Documentation (#51)
    * Contributes to: mhub/qp-planning#0
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * chore: Fix bad merge
    * Import statement was lost when resolving merge conflicts.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: Run eventstreams_java_build in travis
    * Run eventstreams_java_build in travis and cleanup docker meta.
    * Contributes to mhub/qp-planning#4795
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>
    * Signed-off-by: Katherine Stanley <katheris@uk.ibm.com>

  * feat: add run as listener to broker config (#50)
    * This commit adds the RunAs listener to the Strimzi Broker
    * configuration. This is not a configurable listener and will
    * always be configured in the brokers.
    * * feat: Add new kafka vnext image tag to operator
    * This commit updates the kafka image in the strimzi fork.
    * This issue also contains the RunAs kafka implementations.
    * * feat: Remove old version file from commit
    * This commit removes the eventstreams-kafka-version.yaml
    * as this existed when the PR first was submitted, but
    * isn't required anymore.
    * Contributes to: mhub/qp-planning#4687
    * Signed-off-by: Tim Mitchell <tim_mitchell@uk.ibm.com>

  * fix: README and revert install changes on build (#63)
    * Contributes to: mhub/qp-planning#0
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Fix api project tests (#62)
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Fix builds (#65)
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Add AdminAPI security config (#61)
    * This commit updates the AdminAPI pod
    * configuration with the new RunAs properties.
    * This commit fixes a couple of issues and also
    * adds the new kafka and operator image tags to the
    * deployment yaml.
    * It also introduces a secured eventstreams yaml that
    * people can use to run a secured eventstreams.
    * Contributes to: mhub/qp-planning#4684
    * Signed-off-by: Tim Mitchell <tim_mitchell@uk.ibm.com>

  * feat: Remove namespace scoping from webhooks (#66)
    * The motivation for this is to remove the need to add labels to
    * namespaces, as we're not removing these when an operator is
    * uninstalled.
    * This will mean that every webhook will invoke every operator, but
    * I think this is relatively low-cost, so it's an acceptable impact.
    * Closes: mhub/qp-planning#4829
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: Updating Redis version to 5.0.7 (#67)
    * The objective of this change is to update the redis version to 5.0.7
    * so that we receive all the latest fixes, as we are currently using redis 4.0.10.
    * Contributes to: mhub/qp-planning#4784
    * Signed-off-by: Damian Harateh <damian.harateh@ibm.com>

  * feat: enable schema registry to read in accessMode (#71)
    * Those changes enable the setting of the accessMode
    * through the CR, and the defaults behave the same
    * as current eventstreams.
    * Contributes to: mhub/qp-planning#4587
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>
    * Signed-off-by: Damian Harateh <damian.harateh@ibm.com>

  * feat: Adding support for autoUpgradeVersions to our CR status (#70)
    * To improve the experience of our users, we are introducing
    * loose versioning for non-exact version strings.
    * Contributes to: mhub/qp-planning#4760
    * Signed-off-by: Damian Harateh <damian.harateh@ibm.com>

  * feat: Port OLM to Strimzi combined operator (#64)
    * fix: Update list of resources in our CSV
    * The resources section of the CSV has been updated with the list of actual resources.
    * Since the empty name fields were not documenting anything, they have been removed.
    * Furthermore, new resources have been added to accurately reflect the complete list of actual resources that make up an instance of ES.
    * Update init container to run as nonroot
    * It was running as root which is not allowed on
    * openshift-operators namespace
    * Commit changed crd
    * Contributes to: mhub/qp-planning#4756
    * Contributes to: mhub/qp-planning#4777
    * Signed-off-by: Damian Harateh damian.harateh@ibm.com
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Eventstreams cluster admin role (#73)
    * Contributes to: mhub/qp-planning#4744
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: remove ibm-es (#76)
    * Make the name of the Kafka and the EventStreams instance
    * the same, to improve user experience.
    * Contributes to: mhub/qp-planning#4844
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * fix: Use correct clusterrole for es nodeports (#74)
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * fix: Adding necessary compands to the script to fix failing Jenkins (#79)
    * Closes: mhub/qp-planning#4853
    * Signed-off-by: Damian Harateh <damian.harateh@ibm.com>

  * fix: Update image tag for admin proxy (#78)
    * Contributes to: mhub/qp-planning#4856
    * Signed-off-by: Tom Jefferson <thomas.jefferson1@ibm.com>

  * feat: helm charts and install folder generation (#80)
    * this commit creates a set of helm charts and a makefile which
    * when run merges the strimzi charts and the eventstreams charts
    * to produce a set of helm charts that can be used to install
    * eventstreams. these charts are then used to generate the install folder.
    * Contributes to: mhub/qp-planning#4616
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * refactor: remove openssl - now included in qp-base (#83)
    * Contributes to: mhub/qp-planning#4858
    * Signed-off-by: Dave Lane <davilane@uk.ibm.com>

  * feat: Fix Kafka secrets naming (#81)
    * Fix Kafka secrets to always reference correct instance name
    * Contributes to: mhub/qp-planning#4844
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Add Security to KafkaConnect to broker (#69)
    * Three kafka users made, 1 for source side, 2 for destination
    * side.  These are only created if
    * a)No oauth client auth set (which will error)
    * b)If client auth set (which will not error, but instead run
    * unsecured)
    * The source side user creation is dependent on the security level
    * at the external listener level, whereas the dest side
    * is dependent on the internal listener security.
    * If tls is enabled (client auth or just server side cert
    * presented) then connect connects on the 9093 internal
    * service, otherwise it connects on 9092.
    * Contributes to: mhub/qp-planning#4432
    * Signed-off-by: Emma Humber <emma.humber@uk.ibm.com>

  * fix: Update service account to allow init container to run (#85)
    * The init container needs permission to manage validatingwebhookconfigurations
    * so this updates the service accounts to make this available.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Switch to using latest tag
    * Contributes to mhub/qp-planning#4870
    * Signed-off-by: Katherine Stanley <katheris@uk.ibm.com>

  * fix: Update bootstrap env vars for rest (#89)
    * Update Kafka bootstrap env vars to match
    * the new name now the ibm-es has been
    * dropped.
    * Contributes to mhub/qp-planning#4872
    * Signed-off-by: Katherine Stanley <katheris@uk.ibm.com>

  * feat: Validating webhook for KafkaUser cluster labels (#86)
    * This commit adds a webhook for KafkaUser entities to reject any
    * that are missing the label that identifies the Kafka cluster they
    * are for.
    * Although we're not encouraging the use of the topic operator, I
    * also added the same webhook for KafkaTopic entities as it wasn't
    * much extra code.
    * The two webhooks are using different URLs as it was the simplest
    * way to know which class to use to deserialize the request payload
    * without having to peek at the body. Functionally the two hook URLs
    * are otherwise identical.
    * I also left in a logger for uncaught exceptions that I found
    * helpful while debugging my tests, as they're currently swallowed.
    * Closes: mhub/qp-planning#4843
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: add checks in travis for generated files (#87)
    * * feat: add checks in travis for generated files
    * * fix: changes for yq version 3.1.1
    * Contributes to: mhub/qp-planning#4867
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * fix: don't hard-code java image tag in init (#93)
    * Contributes to: mhub/qp-planning#4858
    * Signed-off-by: Dave Lane <davilane@uk.ibm.com>

  * fix: Fix NPE when only external security set (#94)
    * Contributes to: mhub/qp-planning#4876
    * Signed-off-by: Emma Humber <emma.humber@uk.ibm.com>

  * feat: Start using Strimzi Name Label (#84)
    * Contributes to: mhub/qp-planning#4758
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * fix: Cleaning up OLM bundle info (#97)
    * * Removed a placeholder
    * * Added IBM Event Streams official email
    * Since these are public/external values that are shown to customers in the OpenShift web UI, it needed cleaning up.
    * Contributes to: mhub/qp-planning#4845
    * Signed-off-by: Damian Harateh damian.harateh@ibm.com

  * feat: Fix Runas Listener network policy definition (#99)
    * This commit fixes the runas network policy definition
    * which was failing because we called it after the
    * network policy had been defined.
    * Contributes to: mhub/qp-planning#4879
    * Signed-off-by: Tim Mitchell <tim_mitchell@uk.ibm.com>

  * fix: Remove anti-pattern of metadata (#102)
    * Contributes to: mhub/qp-planning#0
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Update docs for OLM (#104)
    * Contributes to: mhub/qp-planning#0
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Address review comments for NPE fix
    * Address review comments for NPE fix
    * Ensure the kafka user auth is set properly
    * Contributes to: mhub/qp-planning#4876
    * Signed-off-by: Emma Humber <emma.humber@uk.ibm.com>

  * feat: Move linter from eventstreams-operator
    * Contributes to: mhub/qp-planning#4796
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * fix: Allow topic operator to be disabled (#91)
    * * fix: Allow topic operator to be disabled
    * This commit updates our Kafka model so that it checks for the
    * presence of a topic operator object in the requested spec
    * before adding a topic operator container to the entity
    * operator pod.
    * I've updated the test that checked for resource limits, and
    * manually verified both with and without topic operators to
    * make sure it behaves as expected.
    * I haven't done this for the user operator as we require this
    * for the security between Event Streams components. This means
    * that the user operator container is always deployed,
    * regardless of what is requested.
    * As we're not advocating the use of the topic operator, I've
    * also removed it from our example yaml files, however this
    * won't stop a user from adding it to their strimziOverrides
    * if they want it.
    * Closes: mhub/qp-planning#4664
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: Move KafkaUser creation, use correct labels + tests (#103)
    * fix: labels for KafkaUsers
    * Add context to why one would call the
    * getComponentLabelsWithoutResourceGroup
    * method due to Strimzi requiring them to not be present
    * Move createKafkaUser to the AbstractModel to standardize the
    * calls
    * Rework replicator user model to simplify based on the above
    * move.
    * Sort out lables
    * Add in a tests for labels and also to check that all kafkas and kafkausers
    * are created in ESOperatorTest
    * Contributes to: mhub/qp-planning#4758
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Move to using MM2 instead of connect
    * Set group.id to be (relatively) unique, set acl to access this
    * group.  Also set the connect cluster name to be unique.
    * config needs to be set at top level not at cluster spec level
    * (includes updates to unit tests)
    * Change user names to match what onCLoud use
    * Make a generic method for querying the replicator name
    * Contributes to: mhub/qp-planning#4873
    * Signed-off-by: Emma Humber <emma.humber@uk.ibm.com>

  * feat: review the CR spec
    * this commit adds in the validation of required fields in the spec
    * It also reverts the yq to a version that doesnt break the build pipeline
    * Contributes to: mhub/qp-planning#4629
    * Contributes to: mhub/qp-planning#4907
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * fix: Allow to run with default restricted scc (#96)
    * Remove hard-code user Id to allow to run with any
    * default user.
    * Update the frontend-rest image tag to allow to run with
    * any default user.
    * Use the image build from a new branch of the
    * expose-tls-and-nontls-ports branch with the fix.
    * Contributes to: mhub/qp-planning#4753
    * Signed-off-by: Erik Hu <erik.yu.shing.hu@ibm.com>

  * feat: Update status spec to match CP4I conventions (#109)
    * This commit updates the shape of our status object to match the
    * latest requirements from CP4I.
    * There are two main changes:
    * 1) The shape of the versions info has changed, and the names
    * have changed from available/autoupgrade to strict/loose.
    * 2) A new object "endpoints" is added, which allows the platform
    * navigator to discover API and UI links to Event Streams in a
    * standard way.
    * Closes: mhub/qp-planning#4916
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: Configure the RunAs port in the Adminapi
    * This commit configures the kafka-bootstrap-servers
    * env var to be the kafka runas port.
    * This commit updates the image tag of the rest admin
    * pod to pick up the runas changes.
    * Contributes to: mhub/qp-planning#4684
    * Signed-off-by: Tim Mitchell <tim_mitchell@uk.ibm.com>

  * feat: set kafka bootstrap in admin api
    * This change sets the internal plain, internal tls and external kafka
    * bootstrap information in the admin api container. To do this, it
    * passes the internal plain, internal tls and external kafka bootstrap
    * urls to admin api as environment variables, and volume mounts
    * the kafka config map in the admin api pod
    * Contributes to: mhub/qp-planning#4875
    * Signed-off-by: Steve Dare <steve.dare@ibm.com>

  * fix: Update replicator model and spec for updated MM2 operator
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * docs: Add descriptions to CRD
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * fix: Correct README install command (#111)
    * Contributes to mhub/qp-planning#0
    * Signed-off-by: Katherine Stanley <katheris@uk.ibm.com>

  * chore: Plugging in admin-api component (#112)
    * - pointing UI to admin-api component
    *  - add missing env vars for admin-api and UI
    *  - update network policy for UI
    *  - add tests
    * Contributes to: mhub/qp-planning#4821
    * Signed-off-by: Gavin Royce Quadros <gavin.r.quadros@ibm.com>

  * chore: Update generated files
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: Initial implementation of mounting certificate + env var (#116)
    * Created a new volume and new volume mount for the client ca
    * certificate as this will need to be used to enable mutual auth
    * on admin-rest.
    * Contributes to: mhub/qp-planning#4960
    * Signed-off-by: Julian Goh <julian.goh@ibm.com>

  * feat switch to new rest producer (#117)
    * Change rest producer model to deploy the new
    * rest producer
    * Contributes to: mhub/qp-eventstreams-operator#4763
    * Signed-off-by: Chris Patmore <christopher.patmore@ibm.com>

  * feat: Add unit tests for mm2 security (#114)
    * Also reworks the replicator spec slightly so we are no
    * longer doubling up on any properties (eg the bootstrap
    * and connect name).  Extending the MM2Spec didn't work
    * as the fluent builders objected to there being a
    * MM2 overrides object in the class, and simply extending
    * the class didn't make any of the parent spec object's
    * fields available.
    * Now the code matches how we do it for the KafkaSpec.
    * Also needed to do some rearranging to fit in with the
    * merge to the latest strimzi
    * Contributes to: mhub/qp-planning#4432
    * Signed-off-by: Emma Humber <emma.humber@uk.ibm.com>

  * feat: routes refactor (#115)
    * We need to be able to delete routes if they're not set. The current
    * system by which routes are generated means their names are
    * unpredictable
    * By pre-seeding the map with the expected route keys and null values
    * the deletion logic can determine which routes are not present and
    * should be deleted. This comes with the caveat that custom listeners
    * must be manually deleted by a user or deleted by owner reference on
    * cluster deletion.
    * Contributes to: mhub/qp-planning#4629
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * chore: Removing frontend-rest (#119)
    * Removing frontend-rest container from admin-api pod.
    * admin-api pod now has only the new admin-api (eventstreams-admin)
    * Contributes to: mhub/qp-planning#4834
    * Signed-off-by: Gavin Royce Quadros <gavin.r.quadros@ibm.com>

  * feat: Structural schema (#120)
    * One of the requirements of structural schemas is that all objects
    * have a type. This commit makes our CRDs compliant with this by
    * adding a type that was missing at the top level.
    * This is something that the upstream Strimzi project are explicitly
    * and intentionally omitting because they need to support Kubernetes
    * 1.11 (OpenShift 3.11) which does not support this.
    * We'll keep this change in our fork only until they drop support
    * for OpenShift 3.11, at which point the change can be made in both
    * places. This is something that is already in plan for Strimzi.
    * Contributes to: mhub/qp-planning#4805
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * refactor: remove redundant route from admin api (#121)
    * now that the old rest container has been removed
    * the route and service that were pointing to it
    * can be removed. Additionally the port numbers
    * for the external-tls and external-plain listeners
    * can be changed to the proper values
    * Contributes to: mhub/qp-eventstreams-operator#4763
    * Signed-off-by: Chris Patmore <christopher.patmore@ibm.com>

  * feat: Add volume mounts for replicator plus tidy up (#118)
    * Add volume mounts for 3 replicator kafka user secrets
    * Add unit tests for these and existing volume mounts
    * Add envs so that replicator Admin api code knows what
    * level of kafka security it is dealing with
    * Rename Destination to Target in replicator code to
    * match what onlcoud use for their terminology (also
    * matche mm2 terminology)
    * Contributes to: mhub/qp-planning#4432
    * Signed-off-by: Emma Humber <emma.humber@uk.ibm.com>

  * feat: Enable georeplication in admin-api and admin-ui deployments (#123)
    * - This commit enables geo-replication via the GEOREPLICATION_ENABLED
    * env vars on admin-api and admin-ui.
    * - It updates the admin-api image to avoid the KafkaAdminClient
    * OOM errors seen in earlier versions.
    * - It also adds a missing serviceSelector label on the replicator
    * deployment which is checked by admin-api to ensure that the cluster
    * that is being connected to is able to do geo-replication.
    * Contributes to: mhub/qp-planning#4486
    * Signed-off-by: Andrew Borley <borley@uk.ibm.com>

  * feat: Reconcile components for deletion (#124)
    * Contributes to: mhub/qp-planning#4629
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Removing admin proxy (#125)
    * Removing admin proxy component from models, tests,
    * spec and relevant crd's.
    * Contributes to: mhub/qp-planning#4834
    * Signed-off-by: Gavin Royce Quadros <gavin.r.quadros@ibm.com>

  * fix: Webhook failed due to missing secret volume (#128)
    * Webhook validation failed due to missing secret files
    * on the HTTPServer during startup
    * Contributes to: mhub/qp-planning#4944
    * Signed-off-by: Erik Hu <erik.yu.shing.hu@ibm.com>

  * chore: Updates for upstream Strimzi
    * - Update version of yq to match the version used in upstream
    * - Fix checkstyle defect introduced in ExamplesTest during rebase
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * chore: Update generated files
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: OSDK Scorecard (#131)
    * * feat: OSDK Scorecard
    * This commit adds the ability to run the OSDK Scorecard locally in order
    * to get insights regarding the mandatory tests we are passing or failing.
    * Also, updates Travis to use yq v3.2.1.
    * The eventstreams_build stage has also been run in order to pick up the
    * changes introduced by the Strimzi yq upgrade.
    * Contributes to: mhub/qp-planning#4773
    * Signed-off-by: Stelios Gkiokas <Stylianos.Gkiokas@ibm.com>

  * fix: admin api logging level (#134)
    * Sets the TRACE_SPEC env var to the first value in the admin api
    * logging section instead of a comma separated list
    * Fixed unit test accordingly
    * Contributes to: mhub/qp-planning#4987
    * Signed-off-by: Steve Dare <steve.dare@ibm.com>

  * chore: remove CRDs fodler from recipe (#136)
    * We need to run an extra step to remove the CRDs folder while running the
    * scorecard otherwise the step is failing.
    * Contributes to: mhub/qp-planning#4773
    * Signed-off-by: Stelios Gkiokas <Stylianos.Gkiokas@ibm.com>

  * feat: Add component create/delete tests for resources (#135)
    * Contributes to: mhub/qp-planning#4692
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * chore: removing build_csv recipe from scorecard (#138)
    * We are having problems to run the operator_courier in a Docker container
    * due to some dependencies of Python3 that make it difficult to convert
    * ASCII to UTF8 (the relevant issue has been raised against their repo).
    * Since running the build_csv recipe before build_scorecard has been added
    * only for sanity purposes in order to always have a fresh OLM bundle, it
    * can be removed for now until the issue is answer from the maintainers.
    * Contributes to: mhub/qp-planning#4773
    * Signed-off-by: Stelios Gkiokas <Stylianos.Gkiokas@ibm.com>

  * chore: recreate crds folder for Scorecard (#139)
    * There was a bug in the previous logic since the recipe couldn't be
    * executed since the crds file could not be created.
    * Contributes to: mhub/qp-planning#4773
    * Signed-off-by: Stelios Gkiokas <Stylianos.Gkiokas@ibm.com>

  * * fix: Refactor kafkausers to use reconcile (#137)
    * Replicator kafkausers should be created under
    * the correct conditions also
    * Contributes to: mhub/qp-planning#4692
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * chore: Update generated files (#140)
    * Updates to the Java model classes pulled in from upstream Strimzi
    * need reflecting in the CRD files that we generate.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * chore: Edit CR Tiles (#141)
    * Remove CR tiles from upstream Strimzi we don't want to display in
    * Event Streams.
    * Improve descriptions of remaining CR tiles.
    * Contributes to: mhub/qp-planning#4997
    * Signed-off-by: Tom Jefferson <thomas.jefferson1@ibm.com>

  * chore: retrieve scorecard output on Jenkins (#144)
    * It seems that some of the instructions from the operator-sdk repo can be
    * ingored in order to properly run the scorecard. This commit removes the
    * volume and volumeMount update of the ES deployment since it "confuses"
    * the scorecard for some reason.
    * Contributes to: mhub/qp-planning#4773
    * Signed-off-by: Stelios Gkiokas <Stylianos.Gkiokas@ibm.com>

  * chore: remove Architecture fromt the spec (#146)
    * remove Architecture from the eventstreams spec and all references to
    * it in the code
    * Contributes to: mhub/qp-planning#5026
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * fix: yq version check (#148)
    * Contributes to: mhub/qp-planning#5029
    * Signed-off-by: Tom Jefferson <thomas.jefferson1@ibm.com>

  * chore: code cleanup and documenting (#143)
    * This commit does some code cleanup. it puts all the default resource requirements in a central file it removes the unnecessary egress rules from the network policy and it adds a javadoc comment to most methods for an improvement in readability.
    * Contributes to: mhub/qp-planning#5024
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * feat: New example Event Streams CR templates (#142)
    * This commit replaces the previous examples with a new core set.
    * One is a simple minimal CR called "quickstart", which is the name
    * that the rest of CP4I is using for their demo/PoC CR templates.
    * It is a single Kafka broker, single ZK node cluster, with no
    * auth enabled and no geo-replicator. The Kafka cluster config is
    * modified to reduce the footprint requirements.
    * The remaining examples are production cluster specifications, for
    * a three, five and nine broker cluster. The configuration is based
    * on the setup described in the performance report for the previous
    * release of Event Streams.
    * I've tried to reduce the amount of detail needed in the CR, such
    * as removing some of the attributes we've had in examples that
    * already matched the default values.
    * I've modified the spec to remove the requirement to specify the
    * number of replicas for geo-rep pods, to make the minimum CR yaml
    * smaller.
    * I've also fixed the order of attributes in the Event Streams spec
    * which had gotten out of date with the actual attributes.
    * I've left "architecture" at the top as it'll be the next thing
    * that we remove.
    * Contributes to: mhub/qp-planning#4622
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * fix: Add missing replicator deployment label (#149)
    * - This commit re-adds a serviceSelector label for the replicator
    * deployment which was accidentally removed in a refactoring PR.
    * Contributes to: mhub/qp-planning#4486
    * Signed-off-by: Andrew Borley <borley@uk.ibm.com>

  * feat: CP4I Header as a Service (#147)
    * Create a Cp4iServicesBinding and wait for it as part of our
    * reconcile loop to check for an instance to retrieve the header
    * URL to be added to our UI.
    * Added additional mocking to fix tests and a non-static wrapper
    * for a static method to allow mocking to be done on it
    * Create Cluster Role for Cp4iServicesBinding
    * Add method to check presence of a Crd.
    * Add checks to prevent failure if Cp4iServicesBinding Crd
    * is not present
    * Create ClusterRole for crds with list permission
    * Added tests to check the behaviour of the Cp4iServicesBinding
    * adding the Header URL to the Admin UI envars
    * Contributes to: mhub/qp-planning#4751
    * Signed-off-by: Tom Jefferson <thomas.jefferson1@ibm.com>

  * feat: Add app.kubernetes.io/part-of label to ES components (#151)
    * This commit adds app.kubernetes.io/part-of to the standard labels that
    * Strimzi operators add. It's set to match what we're currently using
    * for app.kubernetes.io/instance and is set in the same way.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: Introduce Endpoints into CRD (#145)
    * Create new endpoints spec in the CRD. This is an initial commit
    * as part of developing the security model for defining listeners,
    * but there aren't any parts of the CR using this new model yet.
    * Contributes to: mhub/qp-planning#4990
    * Signed-off-by: Julian Goh <julian.goh@ibm.com>

  * fix: generated files mistake (#152)
    * also delete a file accidentally checked in
    * Contributes to: mhub/qp-planning#5024
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * fix: remove missed arch reference from csv (#154)
    * remove missed arch reference from csv
    * also add in pull policy of always to make sure we quickly catch
    * and breaks.
    * Contributes to: mhub/qp-planning#5024
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * fix: add ES_CACERT env var to admin api (#159)
    * * fix: add ES_CACERT env var to admin api
    * This env var was removed when rest was removed, however it is
    * needed by admin-api to retrieve the PEM certificate so has been
    * re-added
    * Contributes to: mhub/qp-planning#5037
    * Signed-off-by: Steve Dare <steve.dare@ibm.com>

  * fix: Use correct encryption value (#161)
    * Use encryption value from the CR for admin-api and schema registry
    * urls since the UI encryption value is set to TLS.
    * Contributes to: mhub/qp-planning#4821
    * Signed-off-by: Gavin Royce Quadros <gavin.r.quadros@ibm.com>

  * feat: tag and push init image (#132)
    * Contributes to: mhub/qp-planning#5009
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * License accept webhook (#162)
    * this commit adds a top level `licenseAccept` field into the spec.
    * It adds a webhook to do validation that the field has been set to true
    * Contributes to: mhub/qp-planning#4621
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * fix: Modifying the tests to correctly handle failed assertions (#164)
    * Previously, many of the unit tests in the ES operator
    * did not handle the failed assertions correctly.
    * This has been addressed by correctly catching the assertion errors
    * and failing the context where necessary
    * to prevent the TimeOut Exception Error, which would occur when an assertion would fail
    * but because the context was not failing, the test would continue
    * and then throw a timeout error.
    * Contributes to: mhub/qp-planning#4795
    * Signed-off-by: Damian Harateh <damian.harateh@ibm.com>

  * fix: Adding an example of improved way of structuring tests (#165)
    * In order to ensure that future tests handle assertions correctly,
    * I added an old and new (recommended) way of structuring the tests.
    * Contributes to: mhub/qp-planning#4795
    * Signed-off-by: Damian Harateh <damian.harateh@ibm.com>

  * feat: Create Endpoints Model (#153)
    * Created the abstract class which will be used to create the
    * services, volumes, and volume mounts needed to create
    * the appropriate services that are needed. Will always
    * create the P2P service regardless of whatever
    * is needed.
    * Contributes to: mhub/qp-planning#4990
    * Signed-off-by: Julian Goh <julian.goh@ibm.com>

  * feat: Creating HMAC secret (#155)
    * Create secret which has a key that is used by Rest Producer
    * and Schema Registry.
    * Contributes to: mhub/qp-planning#4999
    * Signed-off-by: Gavin Royce Quadros <gavin.r.quadros@ibm.com>

  * docs: Clarify that custom SCCs are no longer needed (#166)
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: Correct resources list displayed in OpenShift console (#167)
    * For custom resources, the name is required and must be fully-qualified.
    * I've also removed the replica set from the UI listing, as I don't
    * think it is helpful and just adds noise.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * chore: Rename status variables to match latest CP4I agreement (#169)
    * spec.appVersion is now spec.version
    * This change was motivated to become consistent with OpenShift UI.
    * status.versions.reconciledVersion is now status.versions.installed
    * status.versions.availableVersions.strictVersions is now
    *  status.versions.available.versions
    * status.versions.availableVersions.looseVersions is now
    *  status.versions.available.channels
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * fix: add javadoc @returns (#174)
    * Contributes to: mhub/qp-planning#5048
    * Signed-off-by: Kit Davies <kit.davies@uk.ibm.com>

  * docs: Top-level summary of resource requirements for examples (#173)
    * This is my estimate of the resource requirements for the examples as we
    * currently have them, but it is best treated as a placeholder as the
    * resource requirements for almost all our components will be changing
    * once the performance measurements work has been completed.
    * Contributes to: mhub/qp-planning#4622
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: Use STRIMZI_IMAGE_PULL_SECRETS as a default pull secret (#170)
    * The cluster operator deployment environment variable
    * STRIMZI_IMAGE_PULL_SECRETS is already used as a default pull secret
    * for the images for Kafka, ZooKeeper, TLS sidecar, JMX trans, topic
    * operator, entity operator, etc.
    * This commit extends the use of this environment variable to the
    * Event Streams containers adminApi, adminUi, Collector, REST Producer,
    * and Schema Registry.
    * It doesn't introduce an Event Streams-specific environment variable
    * (e.g. a "EVENTSTREAMS_IMAGE_PULL_SECRETS" option) as I intend to
    * address the difference between STRIMZI... and EVENTSTREAMS... env
    * vars in a later commit.
    * Contributes to: mhub/qp-planning#5034
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * fix: Fix example names in scorecard config
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * fix: Fix error message if license is not accepted
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * fix: Remove duplicate item in CSV resources list (#175)
    * I'm not sure how this got into the list, but the presence of a duplicate
    * breaks the OLM UI.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * fix: Check if Kafka has a Ready condition (#178)
    * The previous implementation had a hard-coded assumption that the
    * ready condition would always be the first condition in the list.
    * This isn't a safe assumption, as other warnings and conditions
    * can be included in the list.
    * This commit makes our check a bit more flexible, so that we can
    * cope with no list of conditions, an empty list, a list with other
    * warnings in, etc.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * refactor: Improve checking Kafka status with new Kafka operator (#179)
    * Contributes to: mhub/qp-planning#5024
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: move function from script to makefile (#182)
    * this commit moves the functionality of the eventstreams_local_build.sh
    * to the eventstreams_makefile and fixes the build issue for quickly
    * building the eventstreams_operator
    * Contributes to: mhub/qp-planning#5051
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * revert: fix: yq version check (#148) (#183)
    * This reverts commit 124253250a9a7ae06b79c28094857a5f9b67fa35.
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * fix: Revert "Improve checking Kafka status with new Kafka operator" (#184)
    * This reverts commit fcb34e497eb83ff5b1ba682b00b74dba295949f6.
    * Contributes to: mhub/qp-planning#5089
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * fix: configuration for admin auth (#185)
    * - fix location for client cert
    * - add in ssl enabled / disabled
    * - separate run as from internal
    * - add envar for runas
    * - admin api now chooses to use run as
    * If admin has done authentication, i.e. if authentication has been
    * enabled it will address runas.
    * If ssl is disabled with no auth then it will hit port 9092.
    * If ssl is enabled, no auth will go to runas with anonymous user.
    * Contributes to: mhub/qp-planning#5068
    * Signed-off-by: Chris Patmore <christopher.patmore@ibm.com>

  * fix: Only query replicator dest side users if replicator enabled (#176)
    * The setClientAuthForReplicator queries the destination side
    * kafka users if client auth is enabled.  However it does this regardless
    * of whether replication is enabled.  And the users are only made if
    * replication is enabled.
    * So, the case where client auth is on, but replication is not enabled
    * it needs to not query the destination side kafka users.
    * Contributes to: mhub/qp-planning#5063
    * Signed-off-by: Emma Humber <emma.humber@uk.ibm.com>
    * Signed-off-by: Gavin Royce Quadros <gavin.r.quadros@ibm.com>

  * feat: Provide user feedback in status conditions (#187)
    * This commit makes a number of changes to our CR status object, with the
    * overall aim of improving usability through user feedback:
    * - Adds status.phase which will be displayed at various points in the
    *  OpenShift UI
    * - Adds warnings with potential problems to status.conditions
    * - Adds a temporary "Creating" condition to status.conditions during
    *  the first run through the reconcile loop
    * - Adds a "Ready" condition to status conditions after the first run
    *  through the reconcile loop
    * Contributes to: mhub/qp-planning#4810
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * docs: Added CRD documentation (#189)
    * Updates to resolve errors reported by the operator scorecard.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * chore: Rename status fields (#190)
    * Updates based on the latest spec agreed with CP4I.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: watch multiple namespaces via olm (#186)
    * this commit also cleans up the deployment by removing an
    * unnecessary EVENTSTREAMS_NAMESPACE env var
    * Contributes to: mhub/qp-planning#5008
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * feat: Convert Admin REST To Endpoints (#177)
    * Convert Admin REST to use new Endpoints Model. This will allow users
    * to configure the endpoints that they want to be created with a
    * specific configuration. Users can specify what type of service they
    * want to create, the authentication mechanisms, if the endpoint is TLS,
    * and what port to access. It will also reconcile the certificates for
    * each endpoint.
    * Note that NodePort Certificate Generation is currently not
    * implemented. Only the route name is put into the SANS if using an
    * external route otherwise it will put in the service.
    * Contributes to: mhub/qp-planning#4990
    * Signed-off-by: Julian Goh <julian.goh@ibm.com>

  * feat: Sample YAML for use in OpenShift YAML editor (#191)
    * Closes: mhub/qp-planning#5071
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * fix: Enable the rest producer to work with security
    * Update the image and envars / mounts etc. for the REST producer
    * to work properly.
    * Add kafka extension network policy. Need to extend the Kafka network
    * policy to allow certain traffic to the runas port.
    * This was in Strimzi code. Have moved into Event Streams code and
    * added a rule for the REST producer.
    * Set the owner reference.
    * Test the new netpol is created.
    * Contributes to: mhub/qp-planning#4763
    * Signed-off-by: Chris Patmore <christopher.patmore@ibm.com>

  * feat: case bundle initial commit (#181)
    * This commit initialises the case bundle directory structure and
    * contains a first pass at populating the files
    * Contributes to: mhub/qp-planning#4365
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * feat: Remove Listeners and Convert Everything to Endpoints  (#192)
    * Remove Listener and Abstract Secure Endpoints from Schema Registry and
    * REST Producer. Listener and Abstract Secure Endpoints have now been
    * deleted from the code. Both components now use Endpoints model to create
    * configurable routes and services. This also introduces the ability for
    * endpoints to delete routes that have been renamed as it now checks the
    * status field for existing routes. Also introduces the ability to pass in
    * a null spec
    * Closes: mhub/qp-planning#4990
    * Signed-off-by: Julian Goh <julian.goh@ibm.com>

  * chore: Image tag updates (#198)
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: Block unsupported authorizers (#194)
    * EventStreams does not support the 'simple' or 'keycloak' authorizers.
    * Block these by preventing them from being parsed, allowing only the
    * 'runas' type to pass. Users can also omit this section altogether.
    * runas is the only supported authorization type if security is enabled.
    * Update the description text to reflect that.
    * Contributes to: mhub/qp-planning#5067
    * Signed-off-by: John Beaven <beavenj@uk.ibm.com>

  * feat: Rename TLS to Internal_TLS (#200)
    * Renamed TLS to INTERNAL_TLS because it was confusing. Also
    * added a description
    * Contributes to: mhub/qp-planning#5107
    * Signed-off-by: Julian Goh <julian.goh@ibm.com>

  * fix: Update Labels method calls to match upstream changes (#201)
    * The Labels class in upstream Strimzi has been refactored, so this
    * commit updates our calls to match the new method names.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * fix: update generated files (#202)
    * Contributes to: mhub/qp-planning#5024
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * feat: webhook to validate endpoints (#203)
    * this commit adds a webhook to validate the list on endpoints provided
    * in the CR has a valid configuration
    * Contributes to: mhub/qp-planning#5031
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * fix: Truncate Default Resource Name (#197)
    * Changes to truncate various parts of default resource names when
    * necessary.
    * With this commit, the app name will still normally be ibm-es as before,
    * but we will switch to a shorter app name of es when we are short of
    * space
    * Added behaviour for excessively long suffixes being truncated.
    * Contributes to: mhub/qp-planning#4943
    * Signed-off-by: Tom Jefferson <thomas.jefferson1@ibm.com>

  * fix: Fix issue with mounting wrong cert in Admin API (#205)
    * The SSL_TRUSTSTORE_LOCATION env var needs to be changed from
    * /certs/cluster/podtls.p12 to /certs/cluster/ca.p12 or else
    * Admin API can't create the Kafka Admin Client.
    * Contributes to: mhub/qp-planning#5114
    * Signed-off-by: Julian Goh <julian.goh@ibm.com>

  * feat: Add ability to set TLS version & rename to internalTls (#207)
    * Add the ability for the operator to configure Admin REST, REST
    * Producer, Schema Registry, Collector and UI with a TLS Version
    * variable. This will be used to drive config to configure the
    * specified TLS Version for an endpoint or per component.
    * Also fixed an issue where the operator tried to create the same
    * service multiple times. Also renamed encryption to internalTls
    * Contributes to: mhub/qp-planning#5115
    * Signed-off-by: Julian Goh <julian.goh@ibm.com>

  * feat: Specify status as a sub-resource in the main CRD (#208)
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: Consolidate envars into EVENTSTREAMS (#206)
    * Trim envars down to reduce duplication
    * Make all envars EVENTSTREAMS specific
    * write logic to clone eventstreams envars as strimzi envars
    * Use WATCHED_NAMESPACE envar as per cloud pak instructions
    * Contributes to: mhub/qp-planning#5076
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * fix: Change the yaml files in operator (#211)
    * Due to the renaming of the variable of TLS to InternalTLS we need
    * to change the yaml to reflect this. This should fix the E2Es
    * produce tls test.
    * Contributes to: mhub/qp-planning#5126
    * Signed-off-by: Julian Goh <julian.goh@ibm.com>

  * feat: Implement validation of plain listeners (#199)
    * Check the configuration of EventStreams security and if it is NONE
    * then make sure that the Kafka Plain Listener is configured WITHOUT
    * security. This will prevent the situation where REST can't talk to
    * Kafka due to a lack of a Plain Listener/it can't authenticate with
    * Kafka.
    * Contributes to: mhub/qp-planning#4899
    * Signed-off-by: Julian Goh <julian.goh@ibm.com>

  * feat: Status modifications as a sub-resource (#215)
    * Closes: mhub/qp-planning#5095
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: Fix spotbugs errors (#210)
    * Updated spotbugs version in our pom to match upstream Strimzi.
    * Fixed some errors reported by Strimzi in our code.
    * Added spotbugs check to Travis for running in future PRs.
    * Spotbugs on the entire repo is overkill. Scanning the cluster
    * operator and Event Streams operator is enough.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * chore: Update version to 2020.2.1 (#209)
    * When we started development, we envisaged a Q1 release, but now that
    * we're targeting June we need to update the version number to match.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: Add rest module to schema pod (#212)
    * Add rest module to schema pod to enable
    * auth calls to be made against schema registry
    * Fix some naming conventions and tidy some of the code
    * and tests
    * Contributes to: mhub/qp-planning#4889
    * Signed-off-by: A. Garrard <GARRARD@uk.ibm.com>

  * chore: Add auth & protocol labels to routes (#204)
    * Adding authentication and protocol labels to routes so that the CLI
    * can discover correct endpoint.
    * Contributes to: mhub/qp-planning#5035
    * Signed-off-by: Gavin Royce Quadros <gavin.r.quadros@ibm.com>

  * feat: Add kafka image to fix ACL issue (#219)
    * This commit updates the kafka image tag which
    * fixes an ACL problem in the RunAs Authorizer.
    * Closes: mhub/qp-planning#5139
    * Signed-off-by: Tim Mitchell <tim_mitchell@uk.ibm.com>

  * feat: Differentiate P2P auth by component name (#218)
    * Will now create different authentication mechanisms on the
    * P2P ports based on what component is being created. This
    * will allow us secure everything.
    * Fixed an issue where routes weren't being created because
    * the labels didn't follow a specific regex.
    * Closes: mhub/qp-planning#5064
    * Signed-off-by: Julian Goh <julian.goh@ibm.com>

  * fix: image tag update (#220)
    * update the RP image tag
    * Contributes to: mhub/qp-planning#4763
    * Signed-off-by: Chris Patmore <christopher.patmore@ibm.com>

  * chore: Image tag updates (#221)
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * fix: Now uses different port for P2P (#224)
    * Previously UI would always use the TLS port to talk to
    * admin rest and schema registry because UI was always
    * TlsV1.2. Now will create p2p port based on the overall
    * CRD security of the CRD
    * Contributes to: mhub/qp-planning#5147
    * Signed-off-by: Julian Goh <julian.goh@ibm.com>

  * feat: Seed help file with error info (#223)
    * Contributes to: mhub/qp-planning#5024
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * fix: Use correct Envar for Log level (#225)
    * Contributes to: mhub/qp-planning#4988
    * Signed-off-by: Tom Jefferson <thomas.jefferson1@ibm.com>

  * feat: Add the fixed public endpoints image tag (#227)
    * This commit adds the image tag containing the
    * fix for the AdminApi Public endpoints into the
    * operator.
    * Contributes to: mhub/qp-planning#5119
    * Signed-off-by: Tim Mitchell <tim_mitchell@uk.ibm.com>

  * feat: Report reconcile failures in CR status (#229)
    * The cluster-operator verticle writes status conditions to explain
    * reconcile failures. As we don't expect users to look at the Kafka
    * CR, these are effectively hidden from users, while the Event Streams
    * instance stays in a Pending state.
    * This commit looks for these failure states, and in the event of
    * a failure in the cluster-operator, the error message is copied
    * into our status conditions list, and the status phase is set to
    * Failed.
    * The onFailure implementation for this will also catch any other
    * general failures and similarly write an explanation into our
    * status and set the instance into a Failed state.
    * Contributes to: mhub/qp-planning#5141
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * chore: Sed command to quote envars (#217)
    * for space-delimited lists, export fails
    * re-quote strings to avoid these errors
    * Contributes to: mhub/qp-planning#5076
    * Signed-off-by: Samuel Hawker <samuel.hawker@ibm.com>

  * feat: Pass default cipher suite to collector (#222)
    * Add the default suites to the collector model
    * and pass these via a new CIPHER_SUITE environment
    * variable to the collector. This can be overriden
    * in the yaml (see issue for details)
    * Contributes to: mhub/qp-planning#5122
    * Signed-off-by: John Beaven <beavenj@uk.ibm.com>

  * feat: Add env vars and mount for geo-replication client auth handling (#230)
    * - Adds a client auth env var to the admin-api pod to enable
    * geo-replication client auth handling.
    *  - Mounts the replicator secret in the replicator pod so that client
    * auth certs & keys can be used in the MM2 connectors.
    *  - Fixes the issue where an empty replicator stanza was not bringing up
    * any replicator pods.
    *  - Ensures the admin-api GEOREPLICATION_ENABLED env var correctly
    * reflects whether georeplication is enabled in the instance.
    * Contributes to: mhub/qp-planning#5081
    * Signed-off-by: Andrew Borley <borley@uk.ibm.com>

  * chore: Update security labels (#226)
    * Updating security labels in routes and rename mutual tls to tls
    * in tests.
    * Contributes to: mhub/qp-planning#5064
    * Contributes to: mhub/qp-planning#5113
    * Signed-off-by: Gavin Royce Quadros <gavin.r.quadros@ibm.com>

  * chore: Remove AdminApiSpec (#233)
    * AdminApiSpec is no longer providing any unique values, so ahead of some
    * changes/refactoring planned for this week, I'm removing it to reduce the
    * number of places that will need changes.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: New webhook for validating characters in metadata.name (#232)
    * Instance names are used in services created by both ES and Strimzi,
    * so this means that we have to enforce the character limitations for
    * Services in Event Streams names.
    * This commit introduces a new webhook to do that, with an error
    * message consistent with the format of errors messages in Kubernetes
    * client exceptions.
    * I've also renamed some of the existing webhooks to make them more
    * consistent.
    * Closes: mhub/qp-planning#5141
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * chore: Update endpoint type capitalisation (#231)
    * Kubernetes convention is for acronyms to be in upper-case.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: Enforce required dependencies between ES components (#234)
    * The UI and geo-replicator both depend upon the adminApi component
    * being enabled. This commit updates the operator to check for this
    * to prevent it continuing and encountering downstream errors.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: New env var to inform admin components if schemareg enabled (#235)
    * I've copied the pattern used in ReplicatorModel.isReplicatorEnabled
    * These new environment variables aren't used yet, but the intention
    * is that they will be used to enable/disable calls to the Schema
    * Registry to avoid errors when the schema registry is excluded from
    * the ES CR.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: Fix CA Cert in Truststore EnvVar (#239)
    * This commit fixes the CA cert that is configured in
    * the Truststore EnvVar of the Rest Producer. It is
    * referring to a non-existant file, podtls.p12 and should
    * be the ca.p12.
    * Also I have removed the CLIENT_P12_PASSWORD as this is
    * no longer used.
    * Contributes to: mhub/qp-planning#5157
    * Signed-off-by: Tim Mitchell <tim_mitchell@uk.ibm.com>

  * fix: image tag update (#244)
    * update the RP image tag
    * Contributes to: mhub/qp-planning#5123
    * Signed-off-by: Chris Patmore <christopher.patmore@ibm.com>

  * chore: remove case from repo (#241)
    * this commit removes the case package from the repository as it is being
    * moved to it's own repository where it wwill be generated from a
    * submodule of this repo
    * Contributes to: mhub/qp-planning#5146
    * Signed-off-by: Harvey Elsom <harvey.elsom@uk.ibm.com>

  * chore: pass k8sapi version to admin api (#245)
    * Added env var to pass the instance api version
    * to admin api
    * Contributes to: mhub/qp-planning#4938
    * Signed-off-by: Steve Dare <steve.dare@ibm.com>

  * feat: Update tag for the collector (#242)
    * Required the base name and tag to be updated to
    * reflect whats currently generated.
    * Contributes to: mhub/qp-planning#5122
    * Signed-off-by: John Beaven <beavenj@uk.ibm.com>

  * chore: Cleanup environment variables on ES containers (#240)
    * This commit removes some unused env vars, and sets the value
    * for some env vars that were still left as unset placeholders.
    * Images needed rebuilding to use the new base layer that removes
    * the need for environment variables that were helping to label
    * nodes in the Helm release.
    * Contributes to: mhub/qp-planning#4667
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * fix: Handle disabled ES components (#246)
    * Admin UI was inadvertently made into a required component, because
    * the OIDC registration was dependent on the UI route, and the OIDC
    * client secret is used in admin-api.
    * This commit fixes some mistakes like that, so components can be
    * removed from a CR without errors.
    * The exception is admin-api which will require more substantial
    * refactoring to allow it to be removed, so this commit explicitly
    * makes it a required component, but otherwise leaves that as-is.
    * Allowing admin-api to be excluded can be left to a follow-on PR.
    * Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>

  * feat: add a script to read kafka version from yaml (#236)
    * This will be used by jenkins to read the version of kafka used in
    * the latest image tag, avoiding the need to hard-code it somewhere
    * in qp-jenkins-jobs
    * Contributes to: mhub/qp-planning#2844
    * Signed-off-by: Dave Lane <davilane@uk.ibm.com>

  * feat: New operator for geo-replicator clusters (#238)
    * When a new georeplicator is added, if admin-api updates the MM2 CR to
    * add in the new config then the ES operator will splat over the changes
    * with what was set originally in the Operator config and all the new
    * config is lost.
    * Strimzi mm2 operator attempts to honour the changes made by REST and
    * recreates the MM2 pod with the new settings, before recreating it
    * again without them after the ES operator changes them.
    * Really, REST needs to update the ES CR, not the mm2 cr as that is
    * where we define the config, but this is pretty risking as patching our
    * main, top level CR each time a mm2 is made isn't wise (what if we
    * patch it badly....)
    * So, as discussed with Sam (and a tiny bit with Dale), separating out
    * the Replicator part of the ES operator into its own resource that we
    * can then patch from admin-api.
    * REST can then make changes to the georeplicator cr, the operator then
    * updates the MM2 cr, and strimzi deals with the update and we should,
    * in theory, be all good.
    * A few items to help with the review
    * In abstract model I needed to change the following so that the
    * setAbstractMethod works for both ES and ESGeorep instances
    * protected void setOwnerReference(EventStreams instance) {
    * protected void setOwnerReference(CustomResource instance) {
    * I've made a separate ReplicatorSecretModel, as that's the …
fvaleri added a commit to fvaleri/strimzi-kafka-operator that referenced this issue Jan 30, 2024
@fvaleri
Fix CC not restarted when API secret changes
a299de8 
The user may need to create new CC API credentials because they may have been compromised.
This can be done by simply deleting the secret containing CC API credentials, which is then recreated by the CO.
The problem is that CC is not restarted, which leads to the following Rebalance error:

```sh
2024-01-29 17:44:08 ERROR KafkaRebalanceAssemblyOperator:483 - Reconciliation strimzi#64(kafkarebalance-watch) KafkaRebalance(test/my-rebalance): Status updated to [NotReady] due to error: Unexpected status code 401 for request to my-cluster-cruise-control.test.svc:9090/kafkacruisecontrol/rebalance?json=true&dryrun=true&verbose=true&skip_hard_goal_check=false&rebalance_disk=false
```

To fix this issue, I'm adding the API secret hash as CC annotation, so that any change will trigger a CC pod restart.

Signed-off-by: Federico Valeri <fedevaleri@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement kafka broker kafka connect
Projects
None yet
Milestone
0.1.0
Development

No branches or pull requests
2 participants
@jpechane @scholzj