Skip to content

v2.2.1

Choose a tag to compare

@jmcorreia jmcorreia released this 29 Oct 10:58
· 2 commits to master since this release

Maintenance release. It has no new engine features and no changes to the ACON API or the runtime behaviour of readers, writers or transformers. It updates dependencies for security fixes, replaces the build and dependency tooling, and rebuilds the CI/dev Docker image.

Security

  • Updated dependencies to resolve known vulnerabilities:
    • boto3 1.40.23 → 1.43.108
    • urllib3 → 2.8.0
    • certifi pinned to 2026.10.19 (transitive dependency of requests)
    • pyspark[connect] 4.1.0 → 4.1.4 and delta-spark 4.3.1 → 4.4.0 (os extra)
    • msal, pyjwt, multidict, msgraph-sdk and requests bumped in the azure and sharepoint extras

Developer tooling and dependency management:

  • Dependencies are now declared directly in pyproject.toml. They are locked with uv.lock.
  • Removed the cicd/requirements*.txt and cicd/requirements*.lock files and the build-lock-files flow. New Makefile targets: build-lock-file, upgrade-lock-file.
  • Replaced isort, black and mypy with ruff (lint and format) and ty (type checking). flake8 is kept for complexity checks.
  • Small code clean-ups in the engine from the new linting rules: the Great Expectations custom expectations, the email notifier and its exceptions, the notification templates and the file storage utils. No intended behaviour change.
  • Updated Makefile, Jenkinsfile, Jenkinsfile_deploy, CONTRIBUTING*.md, README.md and AGENTS.md for the new workflow.

Docker / CI image (LAKEHOUSE-2453, #315, #322)

  • Base image moved from python:3.12-slim-bullseye to python:3.12-slim-trixie (Debian 13).
  • Switched to a multi-stage build, which gives a smaller final image. Python dependencies are installed with uv into /opt/venv.
  • Spark in the image moved from 4.1.3 to 4.1.4 (the tarball is now removed after extraction).
  • The image now handles the case where the group ID already exists.

Documentation and examples

  • Added the table tennis example (bronze, silver and gold notebooks plus table DDLs), ported from the open-source repo, and references to it in README.md and AGENTS.md. Huge Thank You to @gadsc and @https://github.com/TemRevil for the contribution.