Dataflow Templates 2024-04-23-00_RC00

Note: This release is being rolled out. It may not be in your region yet.

Improvements

[YamlTemplate] speed improvements via distroless Xlang image
[JDBC] discover schema from information system tables

Bug Fixes

[Spanner] preserve null and empty arrays correctly

Contributors

Akshara Uke
Cloud Teleport
Damon
Damon Douglas
Danny McCormick
Deep1998
Jack McCluskey
Yi Hu
aksharauke

Dataflow Templates 2024-04-16-00_RC01

Release Week of 2024-04-16

Note: This release is being rolled out. It may not be in your region yet.

Improvements

• Beam version updated to Beam 2.55.1
• Support FLOAT32 type in v1 templates of Spanner
• YAML template performance improvements

Contributors

Aravind Pedapudi
Damon
Damon Douglas
Danny McCormick
Deep1998
Dippatel98
Dmitrii Burlutckii
Jeffrey Kinard
Manit Gupta
Vardhan Thigle
Yi Hu
djagaluru
fozzie15
sharantej
vitaly.terentyev

Dataflow Templates 2024-04-02-00_RC00

Release week of 2024-04-01

Dataflow Templates 2024-03-27-00_RC00

Release Week of 2024-03-27

Note: This release is being rolled out. It may not be in your region yet.

New Templates

N/A

Improvements

• [Python templates] Python UDF Support
• support vector_length column annotation in DataFlow Import/Export
• [Spanner-changestreams-to-sharded-file-sink] load test fixes
• [Flex templates] Extend Python UDF support to GCS flex templates and clean up code reuse
• Rollback YamlTemplate Beam version to 2.54
• [DataGenerator templates] Return a bit more accurate count for DataGenerator Template
• Apache BEAM SDK updates to handle vector length column annotation

Bug Fixes

• [googlecloud-to-neo4j] Escape fields in generated sql
• Fix a bug in Stream_GCS_Text_to_BigQuery classic template
• [Datastream-to-Spanner] Fix conversion to Long
• [Python UDFs] Change logic to not fail if no UDF is provided.

Contributors

@aksharauke
@ambujojha
@Deep1998
@Dippatel98
@fozzie15
@manitgupta
@Polber

Dataflow Templates 2024-03-22-02_RC00

Release week of 2024-03-18

Dataflow Templates 2024-03-12-00_RC00

Release week of 2024-03-11

Dataflow Templates 2024-03-06-00_RC00

Release Week of 2024-03-06

Note: This release is being rolled out. It may not be in your region yet.

New Templates

• SourceDB to Spanner

Improvements

• Upgraded to use Beam 2.54.0.
• Terraform modules specific to each template
• Added At-least Once mode support to Streaming Templates

Bug Fixes

N/A

Contributors

@Abacn
@aksharauke
@bharadwaj-aditya
@damondouglas
@damccorm
@Deep1998
@Dippatel98
@Polber
@shunping

Dataflow Templates 2024-02-20-00_RC00

Release Week of 2024-02-20

Note: This release is being rolled out. It may not be in your region yet.

Contributors

• darshan-sj

Dataflow Templates 2024-02-14-00_RC00

Release week of 2024-02-12

Dataflow Templates 2024-02-13-01_RC00

Release week of 2024-02-12