gcloud.app.deploy no longer accepts python37.

[nguyendon]

Python 3.7 is end of support and Gcloud is no longer accepting it as a runtime. While running the butler.py script as prescribed in the production setup documentation, I am running into an error

| ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: Error(s) encountered validating runtime. Runtime python37 is end of support and no longer allowed. Please use the latest Python runtime for App Engine Standard..
| Return code is non-zero (1).
gcloud deployment failed, retrying...
Running: gcloud app deploy --no-stop-previous-version --quiet --project=foobar  src/appengine/index.yaml src/appengine/app.yaml src/appengine/cron.yaml src/appengine/cron-service.yaml
| Services to deploy:
|
| descriptor:                  [/clusterfuzz/src/appengine/app.yaml]
| source:                      [/clusterfuzz/src/appengine]
| target project:              [foobar]
| target service:              [default]
| target version:              [20240227t222323]
| target url:                  [https://foobaruc.r.appspot.com]
| target service account:      [foobar@appspot.gserviceaccount.com]
|
|
| descriptor:                  [/clusterfuzz/src/appengine/cron-service.yaml]
| source:                      [/clusterfuzz/src/appengine]
| target project:              [foobar]
| target service:              [cron-service]
| target version:              [20240227t222323]
| target url:                  [https://cron-service-dot-foobar.uc.r.appspot.com]
| target service account:      [foobar@appspot.gserviceaccount.com]
|
|
| Configurations to update:
|
| descriptor:      [/clusterfuzz/src/appengine/index.yaml]
| type:            [datastore indexes]
| target project:  [foobar]
|
|
| descriptor:      [/clusterfuzz/src/appengine/cron.yaml]
| type:            [cron jobs]
| target project:  [foobar]
|
|
| Beginning deployment of service [default]...
| ╔════════════════════════════════════════════════════════════╗
| ╠═ Uploading 0 files to Google Cloud Storage                ═╣
| ╚════════════════════════════════════════════════════════════╝
| File upload done.
| ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: Error(s) encountered validating runtime. Runtime python37 is end of support and no longer allowed. Please use the latest Python runtime for App Engine Standard..
| Return code is non-zero (1).
Failed to deploy after 3 retries.

The error is throw by this line:

gcloud app deploy --no-stop-previous-version --quiet --project=foobar  src/appengine/index.yaml src/appengine/app.yaml src/appengine/cron.yaml src/appengine/cron-service.yaml

Where it looks like there are some autogenerated files src/appengine/app.yaml and src/appengine/cron-service.yaml that are requesting runtime: python37.

[jonathanmetzman]

Yes sorry for this, we're currently using an exception to get around this issue. Is there a process for non-Google users to get this exception? We'd like to upgrade to 3.11 but I can't say it's so easy for us to do so.

[jonathanmetzman]

ACtually, you can really help us move to 3.11 if youre running ClusterFuzz on Windows. Are you?

[nguyendon]

Is there a process for non-Google users to get this exception?

Was this a question for me or someone else?

ACtually, you can really help us move to 3.11 if youre running ClusterFuzz on Windows. Are you?

I am not running Windows, though could try to get something up on a Windows machine. Is support on Windows the last piece or is there another reason Windows is needed?

[jonathanmetzman]

It was the last piece when I last attempted this (August). I don't yet have a timeline for when I will do the upgrade sadly, but I'll discuss this internally.

[jonathanmetzman]

Was this a question for me or someone else?

For you. I don't know if you can get an exception, but the email cloud sent me brought me to a Google-only process. Did you get a similar email about the deprecation of 3.7?

[nguyendon]

It was the last piece when I last attempted this (August). I don't yet have a timeline for when I will do the upgrade sadly, but I'll discuss this internally.

Do you have a branch with your attempted changes? I can look when I get some cycles - hoping there isn't too much needed in addition for Windows. I think even partial support for 3.11 would be valuable because 3.7 is hard-blocked right now.

For you. I don't know if you can get an exception, but the email cloud sent me brought me to a Google-only process. Did you get a similar email about the deprecation of 3.7?

I did not get an email though it doesn't mean it didn't go out. We are picking up our CF instance that was deployed a few years ago by a different implementer so I wouldn't have been associated in any system with ClusterFuzz or GCP. Is there any way you could check with the cloud team if there is a public process?

[oliverchang]

#3809 is related to this, although it's possible we can support some kind of appengine deployment on new Pythons prior to that because the requirement of bundling all dependencies does not exist there.

[nguyendon]

@jonathanmetzman would you have that branch handy where you started the upgrade?

[bkosciarz]

#3809 is related to this, although it's possible we can support some kind of appengine deployment on new Pythons prior to that because the requirement of bundling all dependencies does not exist there.

I can confirm that appengine deployment on python3.8 seems to work for release v2.6.0
Wasn't able to get master working due to issues with grpc library

[jonathanmetzman]

@jonathanmetzman would you have that branch handy where you started the upgrade?

It was a few PRs, I'd say the state before this PR reverted everything was the most complete, it worked everywhere (as far as I could tell, it's possible that Mac wasn't working at all for me to notice breakages at the time), but windows.

[quanyang]

Hey team, we're facing the same issue here as well and support for python >3.7 would be greatly appreciated.