Skip to content

Latest commit

 

History

History
204 lines (145 loc) · 8.02 KB

CHANGELOG.md

File metadata and controls

204 lines (145 loc) · 8.02 KB
Raw

Changelog

All notable changes to this project will be documented in this file.

[Unreleased]

Added

  • Default stackableVersion to operator version. It is recommended to remove spec.image.stackableVersion from your custom resources (#267, #268).
  • Configuration overrides for the JVM security properties, such as DNS caching (#272).

Changed

  • vector 0.26.0 -> 0.31.0 (#269).
  • operator-rs 0.44.0 -> 0.48.0 (#267, #275).
  • Removed usages of SPARK_DAEMON_JAVA_OPTS since it's not a reliable way to pass extra JVM options (#272).
  • [BREAKING] use product image selection instead of version (#275).
  • BREAKING refactored application roles to use CommonConfiguration structures from the operator framework (#277).

[23.7.0] - 2023-07-14

Added

  • Generate OLM bundle for Release 23.4.0 (#238).
  • Add support for Spark 3.4.0 (#243).
  • Add support for using custom certificates when accessing S3 with TLS (#247).
  • Use bitnami charts for testing S3 access with TLS (#247).
  • Set explicit resources on all containers (#249).
  • Support pod overrides (#256).

Changed

  • operator-rs 0.38.0 -> 0.44.0 (#235, #259).
  • Use 0.0.0-dev product images for testing (#236).
  • Use testing-tools 0.2.0 (#236).
  • Run as root group (#241).
  • Added kuttl test suites (#252).

Fixed

  • Fix quoting issues when spark config values contain spaces (#243).
  • Increase the size limit of log volumes (#259).
  • Typo in executor cpu limit property (#263).

[23.4.0] - 2023-04-17

Added

  • Deploy default and support custom affinities (#217)
  • Log aggregation added (#226).

Changed

  • [BREAKING] Support specifying Service type for HistoryServer. This enables us to later switch non-breaking to using ListenerClasses for the exposure of Services. This change is breaking, because - for security reasons - we default to the cluster-internal ListenerClass. If you need your cluster to be accessible from outside of Kubernetes you need to set clusterConfig.listenerClass to external-unstable or external-stable (#228).
  • [BREAKING]: Dropped support for old spec.{driver,executor}.nodeSelector field. Use spec.{driver,executor}.affinity.nodeSelector instead (#217)
  • Revert openshift settings (#207)
  • BUGFIX: assign service account to history pods (#207)
  • Merging and validation of the configuration refactored (#223)
  • operator-rs 0.36.0 → 0.38.0 (#223)

[23.1.0] - 2023-01-23

Added

  • Create and manage history servers (#187)

Changed

  • Updated stackable image versions (#176)
  • operator-rs 0.22.0 → 0.27.1 (#178)
  • operator-rs 0.27.1 -> 0.30.2 (#187)
  • Don't run init container as root and avoid chmod and chowning (#183)
  • [BREAKING] Implement fix for S3 reference inconsistency as described in the issue #162 (#187)

[0.6.0] - 2022-11-07

Changed

  • Bumped image to 3.3.0-stackable0.2.0 in tests and docs (#145)
  • BREAKING: use resource limit struct instead of passing spark configuration arguments (#147)
  • Fixed resources test (#151)
  • Fixed inconsistencies with resources usage (#166)

[0.5.0] - 2022-09-06

Added

  • Add Getting Started documentation (#114).

Fixed

  • Add missing role to read S3Connection and S3Bucket objects (#112).
  • Update annotation due to update to rust version (#114).
  • Update RBAC properties for OpenShift compatibility (#126).

[0.4.0] - 2022-08-03

Changed

  • Include chart name when installing with a custom release name (#97)
  • Pinned MinIO version for tests (#100)
  • operator-rs 0.21.0 → 0.22.0 (#102).
  • Added owner-reference to pod templates (#104)
  • Added kuttl test for the case when pyspark jobs are provisioned using the image property of the SparkApplication definition (#107)

[0.3.0] - 2022-06-30

Added

Changed

  • BREAKING: Use current S3 connection/bucket structs (#86)
  • Add node selector to top-level job and specify node selection in PVC-relevant tests (#90)
  • Update kuttl tests to use Spark 3.3.0 (#91)
  • Bugfix for duplicate volume mounts in PySpark jobs (#92)

[0.2.0] - 2022-06-21

Added

  • Added new fields to govern image pull policy (#75)
  • New nodeSelector fields for both the driver and the executors (#76)
  • Mirror driver pod status to the corresponding spark application (#77)

Changed

  • Updated examples (#71)

[0.1.0] - 2022-05-05

Added

  • Initial commit
  • ServiceAccount, ClusterRole and RoleBinding for Spark driver (#39)
  • S3 credentials can be provided via a Secret (#42)
  • Job information can be passed via a configuration map (#50)
  • Update S3 bucket specification to be conform with the corresponding ADR (#55)