Spark On K8S 0.7.2

What's Changed

• fix(airflow): fix spark_history_ui_link by removing -driver suffix by @hussein-awala in #68

Full Changelog: 0.7.1...0.7.2

Spark On K8S 0.7.1

What's Changed

• fix(airflow): configure provider metadata to make the extra-link discoverable by @hussein-awala in #60
• security: bump Jinja2 to avoid CVE-2024-34064 by @hussein-awala in #61

Full Changelog: 0.7.0...0.7.1

Spark On K8S 0.7.0

What's Changed

• dev: Create the doc with mkdocs and deploy to gh page by @hussein-awala in #52
• feat(airflow): Implement an operator link for Spark app and history UIs by @hussein-awala in #49
• test(airflow): add tests for SparkOnK8SOperatorLink by @hussein-awala in #54
• doc: improve examples doc by @hussein-awala in #56
• security: Stop returning the exception details to the API users by @hussein-awala in #57
• chore(helm): Improve the chart doc and migrate to SemVer versioning by @hussein-awala in #58
• docs(helm): add helm chart doc to the gh page by @hussein-awala in #59

Full Changelog: 0.6.0...0.7.0

Spark On K8S 0.6.0

What's Changed

• feat(airflow): add app_id_suffix argument to Airflow operator by @hussein-awala in #41
• dev: add code of conduct to the project by @hussein-awala in #43
• dev: add a security policy to the project by @hussein-awala in #44
• dev: add a contributing doc to the project by @hussein-awala in #45
• dev: setup sponsorship by adding funding.yml file by @hussein-awala in #46
• feat(airflow): try adopt running job in case of unexpected failures by @hussein-awala in #42

Full Changelog: 0.5.2...0.6.0

Spark On K8S 0.5.2

What's Changed

• fix(airflow): wait for termination status when there is no more log by @hussein-awala in #40

Full Changelog: 0.5.1...0.5.2

Spark On K8S 0.5.1

What's Changed

• fix(webserver): fix the websocket URL by @hussein-awala in #36
• security: bump aiohttp to avoid CVE-2024-27306 by @hussein-awala in #37
• security: bump apache-airflow to avoid CVE-2024-31869 by @hussein-awala in #38
• security: bump gunicorn to avoid CVE-2024-1135 by @hussein-awala in #39

Full Changelog: 0.5.0...0.5.1

Spark On K8S 0.5.0

What's Changed

• feat(docker): add a dockerfile for spark-history server by @hussein-awala in #28
• feat(webserver): integrate spark history server by @hussein-awala in #29
• feat(helm): add support for spark history deployment in the helm chart by @hussein-awala in #30
• security: bump Apache Airflow version to avoid CVE-2024-29735 by @hussein-awala in #31
• chore(helm): add SPARK_ON_K8S_IN_CLUSTER by default to the webserver pod by @hussein-awala in #32
• ci(helm): publish the chart to a github page by @hussein-awala in #33
• doc: add instructions to install helm chart from new repo by @hussein-awala in #34
• security: update poetry constraints by @hussein-awala in #35

Full Changelog: 0.4.0...0.5.0

Spark On K8S 0.4.0

What's Changed

This release introduces new features for the API and webserver, including the ability to kill and delete Spark jobs and read driver pod logs directly from the webserver, and some improvements for the helm chart access service account permissions.

• feat(webserver): implement a UI to stream driver pod logs by @hussein-awala in #14
• chore: separate html templates and css styles by @hussein-awala in #15
• chore: separate sync and async app manager in two modules by @hussein-awala in #17
• feat: api routes to kill and delete spark app by @hussein-awala in #16
• fix: use websocket client in async kill_app to run pod exec by @hussein-awala in #18
• fix: remove duplicated app from app endpoints paths by @hussein-awala in #19
• feat: add kill and delete features to the webserver by @hussein-awala in #20
• fix: close the ws client properly by @hussein-awala in #21
• security: bump airflow version to 2.8.3 to avoid CVE-2024-28746 by @hussein-awala in #23
• feat(helm): support specific namespaces and swtiching between read and edit permissions by @hussein-awala in #24
• fix(webserver): fix the static folder path by @hussein-awala in #25
• fix(api): add missing websockets packages to api extra by @hussein-awala in #26
• chore(webserver): add a page to redirect to when there is a K8S error by @hussein-awala in #27

Full Changelog: 0.3.0...0.4.0

Spark On K8S 0.3.0

What's Changed

• feat: support driver and executor node selectors by @hussein-awala in #10
• feat: support driver and executor labels and annotations by @hussein-awala in #11
• feat: support driver pod tolerations by @hussein-awala in #12
• feat: support executor pod template path by @hussein-awala in #13

Full Changelog: 0.2.3...0.3.0

Spark On K8S 0.2.3

What's Changed

• fix(cli): skip adding api commands when api extra is not installed by @hussein-awala in #8
• fix(security): update flask-appbuilder to fix CVE-2024-25128 by @hussein-awala in #9

Full Changelog: 0.2.2...0.2.3