Releases: exasol/script-languages-release
7.1.0: Template flavor with Python 3.10 support
Summary
This release added a new template flavor with Python 3.10 and updated packages.
Package Version Comparison between Release 7.0.0 and 7.1.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.17.0 of the container tool.
Bug Fixes
n/a
Features / Enhancements
- #861: Added flavor template-Exasol-all-python-3.10 with Python 3.10 support
Documentation
n/a
Refactoring
n/a
Security
- #861: Updated Packages and ignore CVE-2023-6176, because it only effects the Kernel
Attention: The CVE's below can't be fixed by the container itself, rather need to be addressed by the host system executing them.
(Updated script-languages ref)
- Ignored CVE-2023-6176 because they are kernel issues and not container issues
7.0.0: Flavor were Upgraded to at least Ubuntu 20.04
Summary
This release upgrades all flavors at least to Ubuntu 20.04 or newer.
- The standard-EXASOL-7.1.0 flavor was upgraded to Python 3.8, R 4.3 and Ubuntu 20.04
- The r-4-minimal flavor was upgraded to R 4.3 and Ubuntu 22.04
- The flavors python-3.7-minimal-EXASOL-6.2.0 and flavor standard-EXASOL-7.0.0 were removed, because EOL
Furthermore, many bugs and security issues were fixed and packages updated.
Package Version Comparison between Release 6.1.0 and 7.0.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.17.0 of the container tool.
Bug Fixes
- #836: ModuleNotFoundError: No module named 'numpy.testing.nosetester'
- #850: Fixed package management tests by updating Ubuntu 20.04
- #851: Fixed AvailableRPackages tests fail for standard flavor 8.0
Features / Enhancements
- #857: Updated R to 4.3 and updated R packages in standard 7.1 flavor
Documentation
n/a
Refactoring
- #376: Removed unused and outdated cpp and python clients (update script-languages ref)
- #843: Switch udf-script-signature-generator to Python 3 (#351)
Security
- #805: Upgraded r-4-minimal flavor to R 4.3 and Ubuntu 22.04
- #806: Removed flavor python-3.7-minimal-EXASOL-6.2.0
- #807: Removed flavor standard-EXASOL-7.0.0
- #827: Updated packages and ignore kernel CVEs
- #831: Updated conda flavors to cryptography 41.0.3 and openssl 3.1.2
- #839: Updated dependencies and ignore kernel CVEs
- #847: Updated packages and ignore Kernel CVEs
- #808: Updated standard-EXASOL-7.1.0 flavor to Ubuntu 20.04
- #824: Updated scipy for standard 7.* flavor to fix CVE-2023-29824
- #854: Updated pyarrow to fix CVE-2023-47248
Attention: The CVE's bellow can't be fixed by the container itself, rather need to be addressed by the host system executing them.
(Updated script-languages ref)
- Ignored CVE-2023-4622, CVE-2023-4623, CVE-2023-4921, CVE-2023-4244, because they are kernel issues and not container issues
- Ignored CVE-2023-20569, because it is an issue of the host system (AMD CPUs)
6.1.0: Pandas 2 support
Summary
This releases adds support for Pandas 2 pyarrow dtype columns for emitting dataframes from Python UDFs. Furthermore, it fixes a silent data corruption when emitting dateframes with float16 dtype columns from Python UDFs.
Note: We could not update scipy in the standard 7.* flavors such that these still contain CVE-2023-29824, because the fixed scipy version is not available for Python 3.7 which is still used in these flavors. We are going to replace Python 3.7 with Python 3.8 in the next release and than also update scipy. See also following GitHub issue.
Note: We also needed to remove numba-scipy from the standard 8.0.0 flavor, because it blocked the security update for scipy.
Package Version Comparison between Release 6.0.0 and 6.1.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.17.0 of the container tool.
Bug Fixes
- #792: Fixed Github workflow publish-test-container by updating script-languages-container-tool to 0.17.0 and script-languages-container-ci to 1.1.0
- #796: Fixed silent data corruption when emitting dataframes with float16 dtype columns from Python UDFs
- #819: Fixed CI, adapted project to Exasol 8 and updated apt packages
Features / Enhancements
- #793: Added support for Pandas 2 pyarrow dtype columns for emitting data from Python UDFs
Documentation
n/a
Refactoring
n/a
Security
- #785: Updated Ubuntu packages
- #789: Updated Ubuntu packages
- #777: Updated redis-py to fix CVE-2023-28859
- #800: Updated Ubuntu packages
- #803: Updated Ubuntu packages
- #815: Updated Ubuntu packages and poetry.lock
- #822: Updated scipy to 1.10.1 in standard 8.0.0 flavor to fix CVE-2023-29824. For this we had to remove scipy-numba. Updated apt packages.
6.0.0: Python 2 and Python 3.6 Removal
Summary
In this release, we removed Python 3.6 and Python 2 from the flavors and replaced it with Python 3.7, because the former ones are long time end of life and don't receive security fixes anymore.
Note: We had to remove samba support from the standard flavor 7.*, because it depended on Python 2. If you need samba support, please use the standard flavor 8.0.
Furthermore, this release fixed a bug with R SCALAR RETURNS UDFs where either a empty run function or returning null from the run function lead to a hanging UDF, which will fail after a while with a out of memory error.
Package Version Comparison between Release 5.0.0 and 6.0.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.15.0 of the container tool.
Bug Fixes
- #703: Fixed package diff workflow
- #714: Pinned Python3 cryptography package on legacy standard containers
- #754: Fixed R SCALAR RETURNS UDFs with empty run function hangs
Features / Enhancements
n/a
Documentation
n/a
Refactoring
- #697: Disabled version check GH action for master branch
- #702: Updated toolchain dependencies
- #706: Updated script-languages-container-ci-setup dependency
- #774: Updated CI to the version released on PyPi
- #768: Removed or replaced Python 2 tests
Security
- #710: Updated curl
- #713: Updated Python3.7
- #720: Updated ubuntu dependencies
- #723: Updated curl
- #725: Updated pip protobuf package on standard flavors
- #727: Ignored CVE-2022-23960
- #732: Updated openjdk for standard flavors
- #734: Ignored CVE-2022-43945 and update Ubuntu packages
- #737: Updated ubuntu packages
- #740: Updated Ubuntu packages
- #745: Updated Ubuntu python packages
- #748: Updated Ubuntu packages and removed Python 3.6
- #752: Updated ubuntu packages
- #756: Updated Ubuntu packages
- #759: Updated Ubuntu packages
- #762: Updated Conda/Ubuntu/Pypi packages
- #764: Removed Python 2 from the flavors
- #767: Updated libprotobuf-dev
- #775: Updated dependencies
5.0.0: Conda-Container and UDF client refactoring
Summary
This release adds two new python-flavors, which use the conda-package manager: A template flavor for Python3.8 with conda (template-Exasol-all-python-3.8-conda) and a template flavor for Python3.8 with conda and cuda (template-Exasol-all-python-3.8-cuda-conda), which replaces the old cuda Python3.6 container.
Also, the C++ part of the UDF client was refactored. Besides, there are several minor refactorings and features and many security updates for Ubuntu and Python packages.
Package Version Comparison between Release 4.3.0 and 5.0.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.14.0 of the container tool.
Bug Fixes
- #686: Fixed bazel crashes on conda flavor
Features / Enhancements
- #666: Make package diffs more robust for CI
- #590: Added flavor template-Exasol-all-python-3.8-conda
- #673: Added dev-tools-script to test container
Documentation
n/a
Refactoring
- #588: Removed virtualschema-jdbc-adapter.jar
- #329: Splitted exaudflib cc into several files
- #646: Updated ci dependencies and removed GCloud CI
- #650: Changed nightly cron and avoid running nightly builds during the weekend
- #648: Moved swig-2.0.4.tar.gz to AWS
- #572: Refactored package import tests
- #677: Improved poetry project structure
- #679: Added a duplicated pyproject.toml
- #451: Removed legacy install scripts
- #503: Added flavor template-Exasol-all-python-3.8-cuda-conda
Security
4.3.0: AWS CI Build and removal of standard-6.2.0 container
Summary
This release now uses AWS for CI and release builds (instead of GClouds). The old standard-6.2.0 container has been removed.
It also contains minor changes regarding the documentation of the packages in the container. The deployment of test container on Dockerhub has been fixed.
The source code of the exaudfclient has been now removed from the script-languages-container. Some integration tests for JVM owned native libraries have been added.
Besides, it contains several security related updates and the documentation has been improved.
Package Version Comparison between Release 4.2.0 and 4.3.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.14.0 of the container tool.
Bug Fixes
- #599: Fixed duplicated packages
- #573: Pushed test container to Dockerhub
- #616: Performed login to dockerhub for specific tests
- #631: Updated script-languages-container-ci-setup and script-languages-container-ci
Features / Enhancements
- #433: Removed the source of the udflclient from the standard containers
- #147: Added tests for JVM owned native libs
- #628: Added release GH actions
Documentation
- #78: Added Docker socket section to prerequisites in usage documentation
- #107: Referenced EXA_toolbox.upload_github_release_file_to_bucketfs in documentation
- #22: Added documentation about testing
- #161: Created FAQ and added solution for /tmp directory
- #181: Added faq about how to start exaslct from within a docker container
- #595: Fixed content of contributing.md
- #632: Prepared release 4.3.0
- #638: Adapted changelog to release-droid format
Refactoring
Security
- #582: Updated libssl
- #596: Updated packages
- #605: Updated packages
- #607: Ignored CVE-2022-1116 as it affects the Linux kernel only
- #613: Ignored CVE-2022-29581 as it affects the Linux kernel only
- #621: Ignored CVE-2022-21499 as it affects the Linux kernel only
- #620: Fixed trivy false positive during secret scan
- #623: Removed CVE-2021-43816 from ignore list
- #636: Updated Ubuntu packages
4.2.0: Vagrant setup and bugfix for Pandas Dataframe
This release fixes a bug in the Python UDF client which occurred when emitting Pandas data frames.
Also, it adds initial support for setting up a virtual machine via Vagrant, which can be used to build this project.
Besides, this releases contains several security fixes.
Package Version Comparison between Release 4.1.0 and 4.2.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.9.0 of the container tool.
Bug Fixes
- #561: Fixed emit of Pandas Dataframe for np.float32 dtype
Features / Enhancements
- #542: Add vagrant setup for libvirt
Documentation
- #578: Prepared release 4.2.0
Refactoring
- n/a
Security
- #541: Update packages python-dev, openjdk, paramiko and ignore CVE-2022-0492
- #553: Ignored CVE-2022-23648
- #558: Update git package
- #562: Ignored CVE-2022-27191
- #567: Updated Python3.7 packages
- #570: Updated Ubuntu packages
- #575: Updated NVidia CUDA Public Key and Updated Curl
4.1.0: Removed datascience flavors and security updates
This release removed two datascience flavors (namely python-3.6-data-science-EXASOL-6.2.0 and r-3.5-data-science-EXASOL-6.2.0).
Also, it contains many security updates and some cleanup.
Package Version Comparison between Release 4.0.0 and 4.1.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.9.0 of the container tool.
Bug Fixes
- #488: Ignore CVE-2021-43816
Features / Enhancements
- #150: Remove usage of pip-legacy-dependency-resolver
Documentation
- #533: Prepare release 4.1.0
Refactoring
Security
- #493: Ignore CVE-2022-0492
- #505: Update libsasl2-dev
- #510: update locales apt package
- #513: Update locales apt package in 20.04 flavors
- #511: Updated generate_package_diffs_for_flavors to Python3.8
- #517: Update openjdk-11-jdk-headless, libssl-dev and openssl
- #520: Ignored Kernel CVE's
- #525: Update package libxml2-dev on standard-8.0 flavor
- #528: Update libssl-dev
- #537: Updated Python3.7
4.0.0: Pandas upgrade to 1.1.5
This release contains a breaking change in the standard containers, because due to a security fix Pandas Python3 package was updated to version 1.1.5 (from 0.22.0). Please note that this version might break existing UDF's using Pandas (Read Pandas Release notes for more information).
Also, this release contains minor bug fixes and other security fixes.
Package Version Comparison between Release 3.2.0 and 4.0.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.9.0 of the container tool.
Bug Fixes
- #472: Fix github upload script
- #474: Fix Python3.6 installation script
- #480: Fix R security scan script
- #484: Update exaslct to 0.9.0
Features / Enhancements
n/a
Documentation
n/a
Refactoring
- #482: Increase version to 4.0
Security
3.2.0: New standard container for Exasol 8.0 and Java UDF conditional compilation
This release adds the new standard container for Exasol 8.0 with Ubuntu 20.04, Python 3.8, Java 11 and R 4.1. Furthermore, we added conditional compilation to Java UDF, which only invokes the Java compiler if the script contains Java code. If it only contains %jar and %scriptClass options it doesn't get used which saves considerable amount of time during startup of the UDFs. Finally, we also improved the startup time of Python3 UDFs.
Package Version Comparison between Release 3.1.0 and 3.2.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.8.0 of the container tool.
Bug Fixes
- #461: Fix Security scan for r-3.5-data-science flavor
- #118: rredis package got discontinued and the new flavor standard-8.0.0-Exasol replaces with redux
- #466: Remove unconditional pandas import in Python3 UDFs to improve startup time
Features / Enhancements
- #440: Updated exaslct to 0.8.0
- #441: updated to exasol python test framework 0.3.0
- #446: Add shellcheck GH actions
- #454: Java UDF conditional compilation
- #382: Add new standard flavor for Exasol 8.0
- #333: Use the new pip dependency resolver in the new flavor standard-8.0.0-Exasol
- #428: Add a security-scan script to the latest standard container
Documentation
n/a
Refactoring
n/a
Security
- #434: Update Python3.6 and Python3.8
- #437: Update openjdk-11-jdk-headless
- #447: Added tensorflow to ignore list on Python3.6-data-science flavor
- #453: Ignore CVE-2021-33430 on Python3.6 flavors
- #457: Use source package of numpy for all flavors. Undo of #453 and fix for CVE-2021-33430 (for major flavors).