Ubuntu can't download public gcs file when created in daisy

[helen-fornazier]

Hello,

I get the following error when executing this simple workflow: https://paste.ee/p/HFEoD

startup-script: INFO Found startup-script-url in metadata.
startup-script: INFO Downloading url from gs://main-nucleus-128012-daisy-bkt/daisy-startup-script-linux-20181212-19:58:21-n5hrn/sources/startup_file_public.ps1 to /startup-ddyqfbac/tmpcg39bi_g using gsutil.
chronyd[1620]: Selected source 169.254.169.254
google_metadata_script_runner[1680]: Failure: Could not reach metadata service: Not Found.
startup-script: WARNING Could not download gs://main-nucleus-128012-daisy-bkt/daisy-startup-script-linux-20181212-19:58:21-n5hrn/sources/startup_file_public.ps1 using gsutil. Command '['gsutil', 'cp', 'gs://main-nucleus-128012-daisy-bkt/daisy-startup-script-linux-20181212-19:58:21-n5hrn/sources/startup_file_public.ps1', '/startup-ddyqfbac/tmpcg39bi_g']' returned non-zero exit status 1..

But I don't have this issue when I use gcloud with --no-scopes and --no-service-account:

gcloud compute instances create koike-u18-2 \
 --metadata=startup-script-url=gs://main-nucleus-128012-daisy-bkt/daisy-startup-script-linux-20181212-19:58:21-n5hrn/sources/startup_file_public.ps1 \
--image=projects/ubuntu-os-cloud/global/images/family/ubuntu-1804-lts \
--no-service-account --no-scopes

where the file startup_file_public.ps1 is public.

I am not entirely sure what is the difference between the instances created by daisy and gcloud, I am still investigating, please let me know if you have already seen this before.

Thanks

[helen-fornazier]

If I add the scopes (with the code below) in the daisy workflow it works.

          "Scopes": [
            "https://www.googleapis.com/auth/devstorage.read_only"
          ]

But when creating the instance in gcloud using --no-scopes it works fine, confusing.

[adjackura]

I have a feeling this may be a bug in gcloud, would have to do some digging, you can check the instance after its created to see if --no-scopes actually works. That workflow looks like its working correctly, what you have there will create an instance with no scopes, maybe try the UI as well?
To work with no scopes the GCS file needs to be set to public read, so set allUsers as reader.

[helen-fornazier]

@adjackura when using gcloud with --no-scopes, no scopes are listed with gcloud compute instances describe koike-u18-2 (and I confirmed they are listed when the --no-scopes is omitted)
I also tested with the UI (no service account and no scopes), and the script runs fine.
The GCS file is public, the interesting thing is that we just get this error with Ubuntu.

[adjackura]

Can you give me the startup-script metadata value for the daisy instance and the gcloud instance, I think this is a bug with the guest environment that has since been fixed.

[helen-fornazier]

@adjackura you can use the same as the example above:
startup-script-url=gs://main-nucleus-128012-daisy-bkt/daisy-startup-script-linux-20181212-19:58:21-n5hrn/sources/startup_file_public.ps1
This file is public for now, you should be able to access it.

[adjackura]

I meant is there a difference when you start the instance? Does gcloud compute instances describe show any differences with metadata.

[helen-fornazier]

I don't think is the difference in the metadata, but one interesting thing is that daisy adds a service account even if no scopes are provided, which doesn't happen with gcloud, even if I provide --service-account=email but with --no-scopes, it doesn't add the service account (the same happens in the UI).

If I comment out this line (populateScopes) it works on Daisy: https://github.com/GoogleCloudPlatform/compute-image-tools/blob/master/daisy/instance.go#L111

I am still digging into the code, but I see that i.Scopes here is not nil, but i.ServiceAccounts is nil so it gets the default SA in the if statement.

Full instance description: Daisy: https://paste.ee/p/kltg4, gcloud: https://paste.ee/p/y1voA

[helen-fornazier]

Adding an empty ServiceAccounts in the workflow fixes the bug [1]. But I was wondering if the best solution wouldn't be to check if i.Scopes is an empty list in this line [2].
I mean:

-       if i.ServiceAccounts == nil {
+       if len(*i.Scopes) > 0 && i.ServiceAccounts == nil {

@adjackura what do you think?

[1] https://github.com/GoogleCloudPlatform/compute-image-tools/compare/master...collabora-gce:fix-metadata-script-test?expand=1
[2] https://github.com/GoogleCloudPlatform/compute-image-tools/blob/master/daisy/instance.go#L230