New: Add support for Cloud Run API v2

[japerry911]

New: Add support for Cloud Run API v2

Overview

• Add support for Cloud Run API v2
  • more features are available, for example longer timeouts (24 hours in beta now)
• Added inheritance for Cloud Run classes, since there are 2 now that have a lot of similarities
• Closes Upgrade Cloud Run to use API v2 #199

Example

• Anonymous Block example:

CloudRunJobV2(
    credentials=gcp_credentials_block,  # GCPCredentials Block
    image=(
        f"us-central1-docker.pkg.dev/{variables.GCP_PROJECT_ID}/prefect-flows/flows:"
        f"{variables.IMAGE_TAG}"
    ),
    region="us-central1",
    cpu=1,
    memory=512,
    memory_unit="Mi",
    env=variables.GCP_BLOCK_ENV,  # environment variables object
    keep_job=False,
    timeout=86400,  # 24 hours
    launch_stage="BETA",
    max_retries=1,
)

Screenshots

• Screenshot of Job being queued up in Cloud Run Jobs

• Screenshot of some Job Configurations, showing 24 hour timeout (Cloud Run API v2 only feature)

• Screenshot of added Documentation

Checklist

• References any related issue by including "Closes #" or "Closes ".
  • If no issue exists and your change is not a small fix, please create an issue first.
• Includes tests or only affects documentation.
• Passes pre-commit checks.
  • Run pre-commit install && pre-commit run --all locally for formatting and linting.
• Includes screenshots of documentation updates.
  • Run mkdocs serve view documentation locally.
• Summarizes PR's changes in CHANGELOG.md

[zzstoatzz]

awesome @japerry911 - thank you for contributing! this is looking pretty good to me so far

I'm going to take it for a spin as soon as i can

[japerry911]

awesome @japerry911 - thank you for contributing! this is looking pretty good to me so far

I'm going to take it for a spin as soon as i can

Thanks @zzstoatzz , I appreciate the opportunity. Let me know if you have any questions or comments, or ideas to expand on, looking forward to hearing back!

Also, for the tests, I noticed they were failing for a 500 error, and was unsure how to deal with that, but can look more into it later this evening.

[japerry911]

Just fixed Pytests, I did have to add components to the base class, and I am curious if those attributes to BaseCloudRunJob are align with what you're thinking @zzstoatzz

thanks!

[japerry911]

just bumping this, any chance this will be reviewed this week, or in near future? I have a Worker for Cloud Run V2 ready/will just need to make a few adjustments based off of any changes to this PR.

Thanks

[zzstoatzz]

hi @japerry911 - apologies on the wait here. Just did some QA and things are looking pretty solid to me.

I'm sure you have more context on the v2 api, but do you think itd be worthwhile to allow users to config the service account to use as a field on the infra block?

also, looks like we have some tests failing due to python < 3.8 annotations. I think we should eventually remove the 3.7 tests as we've dropped support for 3.7 in core, but i suspect 3.7 and 3.8 are failing for the same reason here

[japerry911]

@zzstoatzz no worries at all.

Thanks, that's a great suggestion, so I implemented and tested with Service Account. I also fixed the tests.

Lastly, I will try to review this more/test it a few more times before Monday.

[japerry911]

@zzstoatzz , it appears to be working overall, just wanted to note, which might be obvious/wasn't obvious to me haha, but it only currently works with Prefect Agent, until I finish the Cloud Run V2 Worker (which is mostly built, just need this to merge/then open another PR, etc...).

[japerry911]

Going to close this based off of our discussion. Will work on a branch with Worker for Cloud Run V2 along with the block and open that in future. Thanks @zzstoatzz !