Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add support for druid 0.22.1 and zookeeper 3.7.0 #267

Merged
merged 2 commits into from Jan 11, 2022
Merged

add support for druid 0.22.1 and zookeeper 3.7.0 #267

merged 2 commits into from Jan 11, 2022

Conversation

RoelofKuijpers
Copy link
Contributor

Fixes #256.

Description

  • Upgrade Apache Druid in example specs for deploying a tiny druid cluster to version 0.22.1, to solve the critical vulnerability CVE-2021-44228 in Apache Log4j.
  • Upgrade ZooKeeper to 3.7.0, since starting with Apache Druid 0.22.0, support for ZooKeeper 3.4.x has been removed.
  • Fix breaking changes in ZooKeeper configuration parameters: clientPort and clientPortAddress. Starting with 3.5.0, this information is part of the server keyword specification.

This PR has:

  • been tested on a real K8S cluster to ensure creation of a brand new Druid cluster works.
  • been tested for backward compatibility on a real K*S cluster by applying the changes introduced here on an existing Druid cluster. If there are any backward incompatible changes then they have been noted in the PR description.
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added documentation for new or modified features or behaviors.
Key changed/added files in this PR
  • examples/tiny-cluster-hpa.yaml
  • examples/tiny-cluster-zk.yaml
  • examples/tiny-cluster.yaml

@AdheipSingh
Copy link
Contributor

Thanks @RoelofKuijpers
LGTM

@nishantmonu51 nishantmonu51 merged commit a30f19e into druid-io:master Jan 11, 2022
@CodingParsley CodingParsley mentioned this pull request Dec 19, 2022
Merged
4 tasks
CodingParsley added a commit to confluentinc/druid-operator that referenced this pull request Jan 5, 2023
@CodingParsley @piotrkmita
@AdheipSingh @achetronic @shrutimantri @harinirajendran @RoelofKuijpers @youngwookim @vladislavPV @zhangluva @cintoSunny
OBSDATA-492: Pull upstream changes to confluentinc master (#18)
72998b1 
* Restrict K8s role definitions to actually needed roles (druid-io#226)

* PK | druid-io#255 | Restrict K8s role definitions to actually needed roles

* PK | druid-io#255 | Restrict K8s role definitions to actually needed roles - templates/

* PK | druid-io#255 | Add patch to event and newline at the end of the file

* PK | druid-io#255 | Add patch to event to templates

Co-authored-by: Piotr Kmita <piotr.kmita@zalando.de>

* Introduce State Driven Status, use record pkg for event and clean logging (druid-io#223)

* status v1, records

* use druid constructor in main.go

* fix smoke tests

* fix comments

* update delete event, status

* push emitter changes

Co-authored-by: AdheipSingh <adheip.singh@rilldata.com>

* update to go.mod to k8s 1.22 (druid-io#237)

* update to k8s 1.22

* update go.mod

* use 1.19.2 for suit_test

* revert yaml change

* update identation/spaces in tiny

* update identation/spaces in tiny

Co-authored-by: AdheipSingh <adheip.singh@rilldata.com>

* Create watched_namespace.yaml (druid-io#248)

* Create watched_namespace.yaml

Create namespace that is being watched by Druid when declared and not being the default

* Update watched_namespace.yaml

Add some clarifications on comments

* Update watched_namespace.yaml

Fix to create all required namespaces by WATCH_NAMESPACE env var (comma separated list. This is intended to handle this future feature) only when they are not created yet

* Fix Helm Chart type error  (druid-io#253)

* run helm lint github actions

* add helm lint, template in CI

* Fix76: Scale STS / PVC vertically to support size resizing of druid nodes (druid-io#97)

* add storage class expansion check, generation check, refactor

* add flag for scalePvcSts

* add component label, update crd

* fix unit tests

* identation in testdata

* Fixing the image tag to be pulled for druid-operator (druid-io#259)

* the image tag fixed to druidio/druid-operator:latest

* Update Druid Operator Documentation (druid-io#255)

* update docs

* update typos

* update docs

* udpate docs

* Adding support for custom metricDimensions.json for statsd metrics emitter (druid-io#250)

* Adding support for custom metricDimensions.json.

* Addressing Adheip's comments.

* Fixing test failure.

* add patch to pvc rbac (druid-io#261)

* add support for druid 0.22.1 and zookeeper 3.7.0 (druid-io#267)

* add support for druid 0.22.1 and zookeeper 3.7.0

* Update tiny-cluster-zk.yaml

change ZooKeeper version from 3.7 to 3.7.0

* update appVersion to 0.0.8-release (druid-io#238)

Co-authored-by: AdheipSingh <adheip.singh@rilldata.com>

* Fix-214-Merge-Labels (druid-io#273)

* add health checks to operator manage (druid-io#278)

* safe guard pod pending deletion of pvc (druid-io#289)

* make nodeselector consistent (druid-io#284)

* make nodeselector consitent

* update handler

* fix revert, scale down pods replicas to zero (druid-io#283)

* fix typo for startupProbe (druid-io#282)

* Bump up chart version to 0.1.2 (druid-io#271) (druid-io#272)

* Watch Multiple Namespaces (druid-io#240)

* WIP: on druid k8s extension, do, and watch ns

* support for multiple namespaces to watch

* support for multiple namespaces to watch

* fix typo

* fix identation

* Release 0.0.9 - Go Mod and Dockerfile Update (druid-io#291)

* go modules update

* add icon

* update dockerfile

* fix tests

* update helm version

* max reconciles workerqueue (druid-io#299)

* emit detect underlying object type on CRUD (druid-io#300)

* make crd optional (druid-io#307)

* multiple watched namespaces (druid-io#309)

* Sidecar container spec for druid (druid-io#296)

* Sidecar container spec for druid

* Fix PR comments on additionalcontainer

* Add envFrom and add comments

* Add example spec to example.md

* Add multi container deployment to features.md

Co-authored-by: cinto <cinto@apple.com>

* update makefile, tests, controller-gen (druid-io#315)

* update make

* update makefile

* Fix-312-Run Test on CI, Controller Gen Version Update (druid-io#319)

* update fixes

* add more tests

* fix test kubebuilder

* seperate build

* revert build

Co-authored-by: AdheipSingh <adheips1222@gmail.com>

Co-authored-by: piotrkmita <43473995+piotrkmita@users.noreply.github.com>
Co-authored-by: Piotr Kmita <piotr.kmita@zalando.de>
Co-authored-by: AdheipSingh <34169002+AdheipSingh@users.noreply.github.com>
Co-authored-by: AdheipSingh <adheip.singh@rilldata.com>
Co-authored-by: Alby Hernández <61636487+achetronic@users.noreply.github.com>
Co-authored-by: shrutimantri <shruti1810@gmail.com>
Co-authored-by: Harini Rajendran <harini.rajendran@yahoo.com>
Co-authored-by: RoelofKuijpers <roelof@datanetic.com>
Co-authored-by: Youngwoo Kim <ywkim@apache.org>
Co-authored-by: Vladislav <vladislavPV@users.noreply.github.com>
Co-authored-by: Zhang Lu <91473238+zhangluva@users.noreply.github.com>
Co-authored-by: cintoSunny <67714887+cintoSunny@users.noreply.github.com>
Co-authored-by: cinto <cinto@apple.com>
Co-authored-by: AdheipSingh <adheips1222@gmail.com>
CodingParsley added a commit to confluentinc/druid-operator that referenced this pull request Jan 6, 2023
@CodingParsley @piotrkmita
@AdheipSingh @achetronic @shrutimantri @harinirajendran @RoelofKuijpers @youngwookim @vladislavPV @zhangluva @cintoSunny @lyoung-confluent @freyayunfu @nlou9 @cchristous @ConfluentJenkins
OBSDATA-492: Sync with upstream druid-operator (#16)
4595a63 
* Restrict K8s role definitions to actually needed roles (druid-io#226)

* PK | druid-io#255 | Restrict K8s role definitions to actually needed roles

* PK | druid-io#255 | Restrict K8s role definitions to actually needed roles - templates/

* PK | druid-io#255 | Add patch to event and newline at the end of the file

* PK | druid-io#255 | Add patch to event to templates

Co-authored-by: Piotr Kmita <piotr.kmita@zalando.de>

* Introduce State Driven Status, use record pkg for event and clean logging (druid-io#223)

* status v1, records

* use druid constructor in main.go

* fix smoke tests

* fix comments

* update delete event, status

* push emitter changes

Co-authored-by: AdheipSingh <adheip.singh@rilldata.com>

* update to go.mod to k8s 1.22 (druid-io#237)

* update to k8s 1.22

* update go.mod

* use 1.19.2 for suit_test

* revert yaml change

* update identation/spaces in tiny

* update identation/spaces in tiny

Co-authored-by: AdheipSingh <adheip.singh@rilldata.com>

* Create watched_namespace.yaml (druid-io#248)

* Create watched_namespace.yaml

Create namespace that is being watched by Druid when declared and not being the default

* Update watched_namespace.yaml

Add some clarifications on comments

* Update watched_namespace.yaml

Fix to create all required namespaces by WATCH_NAMESPACE env var (comma separated list. This is intended to handle this future feature) only when they are not created yet

* Fix Helm Chart type error  (druid-io#253)

* run helm lint github actions

* add helm lint, template in CI

* Fix76: Scale STS / PVC vertically to support size resizing of druid nodes (druid-io#97)

* add storage class expansion check, generation check, refactor

* add flag for scalePvcSts

* add component label, update crd

* fix unit tests

* identation in testdata

* Fixing the image tag to be pulled for druid-operator (druid-io#259)

* the image tag fixed to druidio/druid-operator:latest

* Update Druid Operator Documentation (druid-io#255)

* update docs

* update typos

* update docs

* udpate docs

* Adding support for custom metricDimensions.json for statsd metrics emitter (druid-io#250)

* Adding support for custom metricDimensions.json.

* Addressing Adheip's comments.

* Fixing test failure.

* add patch to pvc rbac (druid-io#261)

* add support for druid 0.22.1 and zookeeper 3.7.0 (druid-io#267)

* add support for druid 0.22.1 and zookeeper 3.7.0

* Update tiny-cluster-zk.yaml

change ZooKeeper version from 3.7 to 3.7.0

* update appVersion to 0.0.8-release (druid-io#238)

Co-authored-by: AdheipSingh <adheip.singh@rilldata.com>

* Fix-214-Merge-Labels (druid-io#273)

* add health checks to operator manage (druid-io#278)

* safe guard pod pending deletion of pvc (druid-io#289)

* make nodeselector consistent (druid-io#284)

* make nodeselector consitent

* update handler

* fix revert, scale down pods replicas to zero (druid-io#283)

* fix typo for startupProbe (druid-io#282)

* Bump up chart version to 0.1.2 (druid-io#271) (druid-io#272)

* Watch Multiple Namespaces (druid-io#240)

* WIP: on druid k8s extension, do, and watch ns

* support for multiple namespaces to watch

* support for multiple namespaces to watch

* fix typo

* fix identation

* Release 0.0.9 - Go Mod and Dockerfile Update (druid-io#291)

* go modules update

* add icon

* update dockerfile

* fix tests

* update helm version

* max reconciles workerqueue (druid-io#299)

* emit detect underlying object type on CRUD (druid-io#300)

* make crd optional (druid-io#307)

* multiple watched namespaces (druid-io#309)

* Sidecar container spec for druid (druid-io#296)

* Sidecar container spec for druid

* Fix PR comments on additionalcontainer

* Add envFrom and add comments

* Add example spec to example.md

* Add multi container deployment to features.md

Co-authored-by: cinto <cinto@apple.com>

* update makefile, tests, controller-gen (druid-io#315)

* update make

* update makefile

* Fix-312-Run Test on CI, Controller Gen Version Update (druid-io#319)

* update fixes

* add more tests

* fix test kubebuilder

* seperate build

* revert build

Co-authored-by: AdheipSingh <adheips1222@gmail.com>

* Adding minikube-setup instructions.

Update tiny-cluster.yaml to make it work in minikube.

Fixing operator startup bug.

Miscellaneous fixes to make local minikube setup work.

Fixing MM readiness probe and steps for minikube ingress-dns issue for local minikube setup.

* semaphore

path

generate

fix format

init-ci

add ci bin

path

kubebuilder move

dummy change

* Add command args to container creation (#6)

* go/codeowners: Generate CODEOWNERS [ci skip] (#7)

* [METRICS-4348] update obs-data team as codeownderswq (#8)

* [METRICS-4487] add obs-oncall as codeowners (#10)

* DP-8085 - Migrate to Sempahore self-hosted agent (#9)

* DP-9370 - Migrate to Semaphore self-hosted agent (#15)

* chore: update repo semaphore project

* Update service.yml (#17)

Update semaphore job commands and go version

update build triggers

update semaphore whitelist

Update project.yml

* update CI 

* nodespec should take precedence (#13)

Co-authored-by: piotrkmita <43473995+piotrkmita@users.noreply.github.com>
Co-authored-by: Piotr Kmita <piotr.kmita@zalando.de>
Co-authored-by: AdheipSingh <34169002+AdheipSingh@users.noreply.github.com>
Co-authored-by: AdheipSingh <adheip.singh@rilldata.com>
Co-authored-by: Alby Hernández <61636487+achetronic@users.noreply.github.com>
Co-authored-by: shrutimantri <shruti1810@gmail.com>
Co-authored-by: Harini Rajendran <harini.rajendran@yahoo.com>
Co-authored-by: RoelofKuijpers <roelof@datanetic.com>
Co-authored-by: Youngwoo Kim <ywkim@apache.org>
Co-authored-by: Vladislav <vladislavPV@users.noreply.github.com>
Co-authored-by: Zhang Lu <91473238+zhangluva@users.noreply.github.com>
Co-authored-by: cintoSunny <67714887+cintoSunny@users.noreply.github.com>
Co-authored-by: cinto <cinto@apple.com>
Co-authored-by: AdheipSingh <adheips1222@gmail.com>
Co-authored-by: Harini Rajendran <hrajendran@confluent.io>
Co-authored-by: Luke Young <91491244+lyoung-confluent@users.noreply.github.com>
Co-authored-by: Yun Fu <fuyun12345@gmail.com>
Co-authored-by: nlou9 <39046184+nlou9@users.noreply.github.com>
Co-authored-by: Corey Christous <cchristous@gmail.com>
Co-authored-by: Confluent Jenkins Bot <jenkins@confluent.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nishantmonu51 nishantmonu51 nishantmonu51 approved these changes

@AdheipSingh AdheipSingh AdheipSingh approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Update zookeeper to work for druid 0.22.0
3 participants
@RoelofKuijpers @AdheipSingh @nishantmonu51