Release v2.1.0 · adidas/lakehouse-engine

🚀 Main Changes:

Added Support to Run Test with Spark Connect mode -> super important to guarantee better coverage;
Upgraded pyspark version from 4.0.0 to 4.0.2 and installed connect plugin: pyspark[connect]==4.0.2
Upgrade delta-spark version from 4.0.0 to 4.0.1

Fixed Serverless Streaming issues inside forEachBatch while using the DataFrame Writer
Fixed Serverless Issue dealing with Schema Evolution. There is now a merge_schema parameter in the OutputSpec Merge Options -> https://adidas.github.io/lakehouse-engine-docs/reference/packages/core/definitions.html?h=merge_schema#packages.core.definitions.MergeOptions
Fixed Internal to External Repo Sync to consider file deletions & include release workflow

Full Changelog: v2.0.0...v2.1.0