Bump apache-beam from 2.68.0 to 2.69.0

[dependabot]

Bumps apache-beam from 2.68.0 to 2.69.0.

Release notes

Sourced from apache-beam's releases.

Beam 2.69.0 release

We are happy to present the new 2.69.0 release of Beam. This release includes both improvements and new functionality. See the download page for this release.

For more information on changes in 2.69.0, check out the detailed release notes.

Highlights

• (Python) Add YAML Editor and Visualization Panel (#35772).
• (Java) Java 25 Support (#35772).

I/Os

• Upgraded Iceberg dependency to 1.10.0 (#36123).

New Features / Improvements

• Enhance JAXBCoder with XMLInputFactory support (Java) (#36446).
• Python examples added for CloudSQL enrichment handler on Beam website (Python) (#35473).
• Support for batch mode execution in WriteToPubSub transform added (Python) (#35990).
• Added official support for Python 3.13 (#34869).
• Added an optional output_schema verification to all YAML transforms (#35952).
• Support for encryption when using GroupByKey added, along with --gbek pipeline option to automatically replace all GroupByKey transforms (Java/Python) (#36214).

Breaking Changes

• (Python) dill is no longer a required, default dependency for Apache Beam (#21298).
  • This change only affects pipelines that explicitly use the pickle_library=dill pipeline option.
  • While dill==0.3.1.1 is still pre-installed on the official Beam SDK base images, it is no longer a direct dependency of the apache-beam Python package. This means it can be overridden by other dependencies in your environment.
  • If your pipeline uses pickle_library=dill, you must manually ensure dill==0.3.1.1 is installed in both your submission and runtime environments.
    • Submission environment: Install the dill extra in your local environment pip install apache-beam[gcpdill].
    • Runtime (worker) environment: Your action depends on how you manage your worker's environment.
      • If using default containers or custom containers with the official Beam base image e.g. FROM apache/beam_python3.10_sdk:2.69.0
        • Add dill==0.3.1.1 to your worker's requirements file (e.g., requirements.txt)
        • Pass this file to your pipeline using the --requirements_file requirements.txt pipeline option (For more details see managing Dataflow dependencies).
      • If custom containers with a non-Beam base image e.g. FROM python:3.9-slim
        • Install apache-beam with the dill extra in your docker file e.g. RUN pip install --no-cache-dir apache-beam[gcp,dill]
  • If there is a dill version mismatch between submission and runtime environments you might encounter unpickling errors like Can't get attribute '_create_code' on <module 'dill._dill' from....
  • If dill is not installed in the runtime environment you will see the error ImportError: Pipeline option pickle_library=dill is set, but dill is not installed...
  • Report any issues you encounter when using pickle_library=dill to the GitHub issue (#21298)
• (Python) Added a pickle_library=dill_unsafe pipeline option. This allows overriding dill==0.3.1.1 using dill as the pickle_library. Use with extreme caution. Other versions of dill has not been tested with Apache Beam (#21298).
• (Python) The deterministic fallback coder for complex types like NamedTuple, Enum, and dataclasses now normalizes filepaths for better determinism guarantees. This affects streaming pipelines updating from 2.68 to 2.69 that utilize this fallback coder. If your pipeline is affected, you may see a warning like: "Using fallback deterministic coder for type X...". To update safely sepcify the pipeline option --update_compatibility_version=2.68.0 (#36345).
• (Python) Fixed transform naming conflict when executing DataTransform on a dictionary of PColls (#30445). This may break update compatibility if you don't provide a --transform_name_mapping.
• Removed deprecated Hadoop versions (2.10.2 and 3.2.4) that are no longer supported for Iceberg from IcebergIO (#36282).
• (Go) Coder construction on SDK side is more faithful to the specs from runners without stripping length-prefix. This may break streaming pipeline update as the underlying coder could be changed (#36387).
• Minimum Go version for Beam Go updated to 1.25.2 (#36461).

... (truncated)

Changelog

Sourced from apache-beam's changelog.

[2.69.0] - Unreleased

Highlights

• New highly anticipated feature X added to Python SDK (#X).
• New highly anticipated feature Y added to Java SDK (#Y).
• (Python) Add YAML Editor and Visualization Panel (#35772).
• (Java) Java 25 Support (#35772).

I/Os

• Support for X source added (Java/Python) (#X).
• Upgraded Iceberg dependency to 1.10.0 (#36123).

New Features / Improvements

• Enhance JAXBCoder with XMLInputFactory support (Java) (#36446).
• Python examples added for CloudSQL enrichment handler on Beam website (Python) (#35473).
• Support for batch mode execution in WriteToPubSub transform added (Python) (#35990).
• Added official support for Python 3.13 (#34869).
• Added an optional output_schema verification to all YAML transforms (#35952).
• Support for encryption when using GroupByKey added, along with --gbek pipeline option to automatically replace all GroupByKey transforms (Java/Python) (#36214).
• In Python SDK, the --element_processing_timeout_minutes option will also interrupt the SDK process if slowness happens during DoFn initialization, for example in DoFn.setup() (#36518).

Breaking Changes

• (Python) dill is no longer a required, default dependency for Apache Beam (#21298).
  • This change only affects pipelines that explicitly use the pickle_library=dill pipeline option.
  • While dill==0.3.1.1 is still pre-installed on the official Beam SDK base images, it is no longer a direct dependency of the apache-beam Python package. This means it can be overridden by other dependencies in your environment.
  • If your pipeline uses pickle_library=dill, you must manually ensure dill==0.3.1.1 is installed in both your submission and runtime environments.
    • Submission environment: Install the dill extra in your local environment pip install apache-beam[gcpdill].
    • Runtime (worker) environment: Your action depends on how you manage your worker's environment.
      • If using default containers or custom containers with the official Beam base image e.g. FROM apache/beam_python3.10_sdk:2.69.0
        • Add dill==0.3.1.1 to your worker's requirements file (e.g., requirements.txt)
        • Pass this file to your pipeline using the --requirements_file requirements.txt pipeline option (For more details see managing Dataflow dependencies).
      • If custom containers with a non-Beam base image e.g. FROM python:3.9-slim
        • Install apache-beam with the dill extra in your docker file e.g. RUN pip install --no-cache-dir apache-beam[gcp,dill]
  • If there is a dill version mismatch between submission and runtime environments you might encounter unpickling errors like Can't get attribute '_create_code' on <module 'dill._dill' from....
  • If dill is not installed in the runtime environment you will see the error ImportError: Pipeline option pickle_library=dill is set, but dill is not installed...
  • Report any issues you encounter when using pickle_library=dill to the GitHub issue (#21298)
• (Python) Added a pickle_library=dill_unsafe pipeline option. This allows overriding dill==0.3.1.1 using dill as the pickle_library. Use with extreme caution. Other versions of dill has not been tested with Apache Beam (#21298).
• (Python) The deterministic fallback coder for complex types like NamedTuple, Enum, and dataclasses now normalizes filepaths for better determinism guarantees. This affects streaming pipelines updating from 2.68 to 2.69 that utilize this fallback coder. If your pipeline is affected, you may see a warning like: "Using fallback deterministic coder for type X...". To update safely sepcify the pipeline option --update_compatibility_version=2.68.0 (#36345).
• (Python) Fixed transform naming conflict when executing DataTransform on a dictionary of PColls (#30445). This may break update compatibility if you don't provide a --transform_name_mapping.
• Removed deprecated Hadoop versions (2.10.2 and 3.2.4) that are no longer supported for Iceberg from IcebergIO (#36282).
• (Go) Coder construction on SDK side is more faithful to the specs from runners without stripping length-prefix. This may break streaming pipeline update as the underlying coder could be changed (#36387).
• Minimum Go version for Beam Go updated to 1.25.2 (#36461).
• (Java) DoFn OutputReceiver now requires implementing a builder method as part of extended metadata support for elements (#34902).
• (Java) Removed ProcessContext outputWindowedValue introduced in 2.68 that allowed setting offset and record Id. Use OutputReceiver's builder to set those field ([#36523]apache/beam#36523).

... (truncated)

Commits

• 2ca7d21 Set version for 2.69.0 RC3
• 976cbc3 Revert "Add GRPC experiments to Python dockerfile (#36525)" (#36573)
• ecdc49b Fix dependency version (#36570)
• 5729522 Merge pull request #36567 from Abacn/cp-36564
• 7d85f18 Fix BigQueryIO File load validate runtime value provider (#36564)
• c6e2df0 CP: Fix publishing of ml/distroless images (#36565)
• b48d832 [release-2.69] Cherrypick #36518 to the branch. (#36551)
• 8739e50 Set Dataflow container to release version.
• ed39503 Skip TestTimers_ProcessingTime_Unbounded for spark. (#36527)
• abf1904 Merge pull request #36523: revert outputWindowedValue changes from KafkaIO as...
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)