Error when setting up Cloud Scheduler Email

[BillPetti]

I am following the process via cr_setup() and I am running into an error when I reach the step for setting up Cloud Scheduler email.

To this point, the setup has found an environmental variable for GCE_DEFAULT_PROJECT_ID, GCE_AUTH_FILE, and GCS_DEFAULT_BUCKET.

I select 'Yes' when asked if I want to set up an email and then Yes (recommended) when it asks if I want to use the email from my JSON service account auth key. I then select my pre-authorized account and the result is the following:

ℹ 2020-09-10 16:59:26 > Request Status Code: 400 Error: API returned: Service account service-[my service clientid]@gcp-sa-cloudscheduler.iam.gserviceaccount.com does not exist.

Not sure where the issue may be.

[MarkEdmondson1234]

Could you try this with the latest GitHub version (on CRAN this week)

It checks for missing service keys. If not then I can add it, it's usually when it's an older GCP project that hasn't had the service activated before, so you could also try adding it yourself via the IAM section in the web console.

[MarkEdmondson1234]

Its now on CRAN v0.3.0 - can you update and see if that solves the issue?

#73

[BillPetti]

Thanks, I did update but was still running into issues. I decided to create a brand new project, but when I get to the step for creating a client ID I cannot select "Other" from the drop down:

[MarkEdmondson1234]

It’s updated recently to “Desktop app” from previously “Other”

…

________________________________ From: Bill Petti <notifications@github.com> Sent: Friday, September 11, 2020 3:32:29 PM To: MarkEdmondson1234/googleCloudRunner <googleCloudRunner@noreply.github.com> Cc: Mark <github@markedmondson.me>; Comment <comment@noreply.github.com> Subject: Re: [MarkEdmondson1234/googleCloudRunner] Error when setting up Cloud Scheduler Email (#89) Thanks, I did update but was still running into issues. I decided to create a brand new project, but when I get to the step for creating a client ID I cannot select "Other" from the drop down: [Screen Shot 2020-09-11 at 9 30 37 AM]<https://user-images.githubusercontent.com/1923057/92931559-a0588a80-f411-11ea-8415-0006d2f9fe36.png> — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#89 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAYCPLETMLKGRRAG2FZESADSFIRG3ANCNFSM4RF3VREQ>.

[MarkEdmondson1234]

In your old GCP account, it should also fix it if you add service-[my service clientid]@gcp-sa-cloudscheduler.iam.gserviceaccount.com as a "Cloud Scheduler Service Account" role in your IAM

[BillPetti]

Thanks. So I get to the same step, and here's what happens:

I select 2: Yes (Recommended)

Then I get this:

[MarkEdmondson1234]

If you check the IAM console is the service key in question there? I wonder if it takes a few minutes to register. Should be at this link: https://console.cloud.google.com/iam-admin/iam

[MarkEdmondson1234]

Could you also confirm your googleAuthR version via sessionInfo() - should be 1.3.0

[MarkEdmondson1234]

Ok I think I found it, it was not checking to make the service email before assigning it a role. This should be fixed if you manually create the email in the console, but should also be fixed in the GitHub version you can install via remotes::install_github("MarkEdmondson1234/googleCloudRunner")

The issue should only affect projects where Cloud Scheduler has not been activated before and the service key is missing.

[BillPetti]

If you check the IAM console is the service key in question there? I wonder if it takes a few minutes to register. Should be at this link: https://console.cloud.google.com/iam-admin/iam

Interesting, so the service account isn't there. I created it through cr_setup, but looking at my Credentials it was listed under OAuth 2.0 Client IDs and now it isn't even there. Something is definitely off.

When I created my own service account via the console I have this (plumber@plumber-take-3.iam.gserviceaccount.com). But it doesn't seem to be associating that service account.

[BillPetti]

Could you also confirm your googleAuthR version via sessionInfo() - should be 1.3.0

Yes, that is the version I have installed

[MarkEdmondson1234]

As well as the service key you create for the library, the Cloud Scheduler service key ( service-[my service clientid]@gcp-sa-cloudscheduler.iam.gserviceaccount.com) needs to exist in your IAM, even though you will not directly use it. If you download the latest GitHub version it should ensure this is present during setup and not fail with the 400 error you saw in first post.

[lgnbhl]

Hi,

Thanks a lot for this package!

I have a different error message at the Cloud Scheduler email step after successfully passing the previous steps.

i No environment argument detected: CR_BUILD_EMAIL
Do you want to setup a Cloud Scheduler email?

1: Nope
2: Yup
3: No way

Selection: 2
Do you want to use the email from your JSON service account auth key?

1: No
2: Yes (Recommended)

Selection: 2
Erreur : parse error: premature EOF
                                       
                     (right here) ------^

Any idea where the issue could be?

Thank you very much in advance for your help!

[MarkEdmondson1234]

Hi @lgnbhl - did the issue still persist after an R session restart? Or does it still happen if you choose "Configure Cloud Scheduler build email" in the wizard setup screen?

May I also see your sessionInfo() to check what version of the packages you are using, and the value of Sys.getenv("GCE_AUTH_FILE") which should be a valid location of the json auth file - it may be that the auth key you have is invalid, in which case the tests will not complete (cr_setup_tests()).

[MarkEdmondson1234]

I added some more messaging around this workflow which may help pinpoint what the issue is, but its most likely an invalid auth json file.

[MarkEdmondson1234]

Looks related to #94

It seems not possible to check for the existence of the scheduler email (its not in the list of IAM API calls) so the end user will need to set it up in the console (e.g. activate App Engine and choose a region). Once the email is present, then the roles should be already set.

[MarkEdmondson1234]

@BillPetti this should work now, which was caused by a failure when checking for the scheduler email. Google has changed the format of the file/email and its too much for me to keep up so I removed it.