Skip to content

v1.27.0

Choose a tag to compare

View all tags
@jmcorreia jmcorreia released this 08 Sep 14:09
· 4 commits to master since this release
v1.27.0
1f2027d

🚀 Main Changes

  • Spark Upgrade from 3.5.0 to 3.5.5
  • Delta Spark Upgrade from 3.2.0 to 3.3.1
  • Python Upgrade from 3.11 to 3.12
  • Other Dependencies (including transitive dependencies) upgrade due to the aforementioned upgrades

🔧Breaking Changes:

  • Prisma Processed Keys table schema was changed to force writing in STRING, stabilising schema possible conflicts
  • Added logic to Cast to String all columns coming from the PRISMA DQ execution, except for the ones defined in DQ_COLUMNS_TO_KEEP_TYPES here. This change is also important to eliminate possible schema merging conflicts while using PRISMA DQ making the feature more stable to adopt.
  • To solve both breaking changes you might need to evolve your schema (or start from scratch). To evolve your schema, you can simply backup your current data and re-write it after casting.

📦 Upgrade now to benefit from improved reliability and smoother operations.
Full Changelog: v1.26.1...v1.27.0
If you have any questions or need support, feel free to reach out to the team.

Assets 2
Loading