run ruff on codebase

[Mbeaulne]

TL;DR

Modernized Python code by adopting PEP 585 union syntax, consolidating imports, and applying various code style improvements.

What changed?

• Replaced typing.Union and typing.Optional with modern union syntax (| and | None)
• Consolidated multiple import statements from the same module into single lines
• Removed unused imports (Optional, OrderedDict, etc.)
• Updated type annotations to use built-in generics (list, dict, tuple) instead of typing equivalents
• Replaced f-string formatting with direct string literals where variables weren't used
• Simplified conditional logic in some cases (combined elif conditions)
• Updated exception handling to remove unused exception variables
• Modernized file opening calls by removing redundant mode specifications
• Fixed import ordering to follow PEP 8 conventions

How to test?

Run the existing test suite to ensure all functionality remains intact. The changes are purely stylistic and should not affect runtime behavior.

Why make this change?

These changes improve code readability and maintainability while adopting modern Python conventions. The new union syntax is more concise and readable, consolidated imports reduce clutter, and removing unused imports helps with code clarity and potentially reduces import overhead.

[Mbeaulne]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• Fixes typing issues using mypy #122
• run ruff on codebase #121 👈 (View in Graphite)
• Adds ruff and mypy linting and type checking #120
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

[morgan-wowk]

Seems pretty good to me. Can you ask an agent if any of the changes made in the most recent commit (assuming you are checked out on this branch) are breaking changes? I'd be interested to have that peace of mind, especially given my own Python knowledge is limited and I won't be able to properly judge line by line if this is safe.

If you think it's safe, and an agent thinks it's safe, and a regression test of the API and orchestrators passes locally, then it's good with me!

[Ark-kun]

Thank you for your work on improving the codebase.

There couple of notes:

run ruff

We use Black for code formatting.

Our style guide is based on Google Style guide: https://google.github.io/styleguide/pyguide.html with certain exception like using relative imports.

adopting PEP 585 union syntax

Note that the type union syntax is https://peps.python.org/pep-0604/ which was only introduced in Python 3.10.
Since Python 3.9 is EOL, I guess we can drop support. But usually we try to support users unless there is some compelling feature that we want.

Let's make sure the resulting changes work on Python 3.10 minimum version. There are some delayed type annotations that I think are only supported in later Python versions (for example, referring to the function's enclosing class).

We also use https://www.conventionalcommits.org/ . This will make it easy to generate changelogs.

[Ark-kun]

We usually put cloud_pipelines import in a group adjacent to cloud_pipelines_backend since these are closely related projects.

[Ark-kun]

In which versions of Python this will work?

[morgan-wowk]

I left a comment and trust you can do the necessary safety checks before merging / after you have addressed Alexey's comments