GCE cloudinit and ubuntu keys from metadata to ubuntu authorized_keys #3208

ubuntu-server-builder · 2023-05-11T14:59:01Z

This bug was originally filed in Launchpad as LP: #1781039

Launchpad details

affected_projects = ['cloud-init (Ubuntu)', 'cloud-init (Ubuntu Trusty)']
assignee = shaner
assignee_name = Shane Peters
date_closed = 2018-07-11T16:39:36.027484+00:00
date_created = 2018-07-10T19:21:44.235303+00:00
date_fix_committed = 2018-07-11T16:39:36.027484+00:00
date_fix_released = 2018-07-11T16:39:36.027484+00:00
id = 1781039
importance = low
is_complete = True
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1781039
milestone = None
owner = shaner
owner_name = Shane Peters
private = False
status = fix_released
submitter = shaner
submitter_name = Shane Peters
tags = ['sts', 'verification-done', 'verification-done-trusty']
duplicates = []

Launchpad user Shane Peters(shaner) wrote on 2018-07-10T19:21:44.235303+00:00

[Impact]

Per documentation at https://wiki.ubuntu.com/GoogleComputeEngineSSHKeys ssh keys for cloudinit and ubuntu users should both be added to the 'ubuntu' users authorized_keys file.
This works fine in Xenial (16.04) and higher, but doesn't work for Trusty (14.04).

[Test Case]

Create a file that contains ssh public keys

$ cat googlekeys
test:ssh-rsa test@example.com
ubuntu:ssh-rsa test@example.com
cloudinit:ssh-rsa test@example.com
Create an ubuntu 14.04 instance

gcloud compute instances create ubuntu1404cloudinit --image-family ubuntu-1404-lts --image-project ubuntu-os-cloud --metadata-from-file=ssh-keys=googlekeys --metadata=block-project-ssh-keys=True
Create an ubuntu 16.04 instance

gcloud compute instances create ubuntu1604cloudinit --image-family ubuntu-1604-lts --image-project ubuntu-os-cloud --metadata-from-file=ssh-keys=googlekeys --metadata=block-project-ssh-keys=True
Notice that the ubuntu user in the ubuntu 14.04 instance contains no keys from cloud-init (the keys there are added by the google daemon):

$ sudo cat /home/ubuntu/.ssh/authorized_keys

Added by Google

ssh-rsa test@example.com
However, in 16.04,

$ sudo cat /home/ubuntu/.ssh/authorized_keys
ssh-rsa test@example.com
ssh-rsa test@example.com

Added by Google

ssh-rsa test@example.com

[Regression Potential]

DatasourceGCE.py is heavily modified to fix this behavior in 14.04. That said, there is a medium amount of regression potential when using the GCE datasource. More specificallly, there is now stricter checking of the metadata source when used(platform_check=True).
Significant testing has been completed via the Google Compute platform as well as other none-GCE datasources (lxd) to confirm functionality and to test for possible regressions.

ubuntu-server-builder · 2023-05-11T14:59:03Z

Launchpad user Shane Peters(shaner) wrote on 2018-08-06T20:50:20.625111+00:00

Launchpad attachments: lp-1781039-gce-datasource-update.patch

ubuntu-server-builder · 2023-05-11T14:59:05Z

Launchpad user Shane Peters(shaner) wrote on 2018-08-06T20:55:39.256855+00:00

I've tested a "first boot" scenario using a customised image with this patched cloud-init and it works as expected (you can see both cloudinit and ubuntu keys above the 'Added by google' comment).

$ cat googlekeys
carbon:ssh-rsa AAAAB....ZCRI2M shaner@carbon
ubuntu:ssh-rsa AAAAB....65Otq/ shaner@ubuntu
cloudinit:ssh-rsa AAAA..6s3oSv shaner@cloudinit

USING CUSTOM IMAGE WITH UPGRADED CLOUD-INIT

#############################################
$ gcloud compute instances create trusty1 --image custom-cloud-init --image-project firm-lacing-209716 --metadata-from-file=ssh-keys=googlekeys --metadata=block-project-ssh-keys=True
$ ssh ubuntu@x.x.x.x cat .ssh/authorized_keys
ssh-rsa AAAAB....65Otq/ shaner@ubuntu
ssh-rsa AAAA..6s3oSv shaner@cloudinit

Added by Google

ssh-rsa AAAAB....65Otq/ shaner@ubuntu

USING EXISTING IMAGE

################################
$ gcloud compute instances create trust2 --image-family ubuntu-1404-lts --image-project ubuntu-os-cloud --metadata-from-file=ssh-keys=googlekeys --metadata=block-project-ssh-keys=True

$ ssh ubuntu@${IP} cat .ssh/authorized_keys

Added by Google

ssh-rsa AAAAB.....z65Otq/ shaner@ubuntu

$ scp cloud-init_0.7.5-0ubuntu1.23_all.deb ubuntu@${IP}:~/
$ ssh ubuntu@${IP} sudo dpkg -i cloud-init_0.7.5-0ubuntu1.23_all.deb
$ ssh ubuntu@${IP} sudo rm -rf /var/lib/cloud/instances
$ ssh ubuntu@${IP} sudo reboot
$ ssh ubuntu@${IP} cat .ssh/authorized_keys
ssh-rsa AAAA.....6s3oSv shaner@cloudinit

Added by Google

ssh-rsa AAAAB....65Otq/ shaner@ubuntu

You'll notice in this second example, the 'ubuntu' key from my googlekeys file isn't added on top like you would expect if the image had an upgraded cloud-init. This is because the code is duplicate checking and won't add a key if it already exists.

ubuntu-server-builder · 2023-05-11T14:59:07Z

Launchpad user Scott Moser(smoser) wrote on 2018-08-07T14:29:04.997617+00:00

Hi Shane,
Could you put this into a git branch and propose it for merge into
ubuntu/trusty branch of upstream?
Similar to

please feel free to ping me if what i'm asking isn't clear.

Scott.

ubuntu-server-builder · 2023-05-11T14:59:09Z

Launchpad user Scott Moser(smoser) wrote on 2018-08-07T14:30:42.469362+00:00

Bah. similar to described in
http://cloudinit.readthedocs.io/en/latest/topics/hacking.html
but instead of 'master' you'll propose merge into ubuntu/trusty.

Also, you will need to do the change as a "quilt 3.0" format...
that is to say your change will add a file to debian/patches/
and will update debian/patches/series.

ubuntu-server-builder · 2023-05-11T14:59:11Z

Launchpad user Shane Peters(shaner) wrote on 2018-08-07T16:08:00.000679+00:00

Hi Scott,
Thanks for the guidance! I think I got it now, let me know if you see the merge proposal.

ubuntu-server-builder · 2023-05-11T14:59:13Z

Launchpad user Scott Moser(smoser) wrote on 2018-09-10T22:25:49.536502+00:00

uploaded. 0.7.5-0ubuntu1.23
Currently in the SRU queue.

Thanks Shane.

ubuntu-server-builder · 2023-05-11T14:59:15Z

Launchpad user Robie Basak(racb) wrote on 2018-09-12T11:38:22.327428+00:00

12:36 smoser: could you add SRU information to bug 1781039 please?

12:36 smoser: in particular Regression Potential. Looks like the entire cloudinit/sources/DataSourceGCE.py file has been rewritten or wholesale backported?

Or are you intending to follow the full process documented at https://wiki.ubuntu.com/CloudinitUpdates?

ubuntu-server-builder · 2023-05-11T14:59:16Z

Launchpad user Scott Moser(smoser) wrote on 2018-09-12T12:56:07.559341+00:00

Shane,
Could you pleaes fill in a Stable Release Updates template on this bug?

https://wiki.ubuntu.com/StableReleaseUpdates

Thanks.

ubuntu-server-builder · 2023-05-11T14:59:18Z

Launchpad user Shane Peters(shaner) wrote on 2018-09-12T19:47:08.735220+00:00

Updated SRU template. Let me know if there's anything I'm missing.

ubuntu-server-builder · 2023-05-11T14:59:20Z

Launchpad user Brian Murray(brian-murray) wrote on 2018-09-27T19:04:03.521077+00:00

Hello Shane, or anyone else affected,

Accepted cloud-init into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.5-0ubuntu1.23 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-trusty to verification-done-trusty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-trusty. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

ubuntu-server-builder · 2023-05-11T14:59:22Z

Launchpad user David Coronel(davecore) wrote on 2018-10-24T15:20:27.306365+00:00

I tested the package cloud-init 0.7.5-0ubuntu1.23 from trusty-proposed and confirm it works as expected.

TEST KEYS

$ cat googlekeys
test:ssh-rsa AAAA[...]+1LRl test@example.com
ubuntu:ssh-rsa AAAA[...]+2LRl test@example.com
cloudinit:ssh-rsa AAAA[...]+3LRl test

CREATE THE INSTANCE

$ gcloud compute instances create ubuntu1404cloudinittest --image-family ubuntu-1404-lts --image-project ubuntu-os-cloud --metadata-from-file=ssh-keys=googlekeys --metadata=block-project-ssh-keys=True --zone us-central1-a

BEFORE THE UPDATE

$ gcloud compute --project "ubuntu-os-support" ssh --zone "us-central1-a" ubuntu1404cloudinittest

$ sudo cat /home/ubuntu/.ssh/authorized_keys
ssh-rsa AAAA[...]+fVtBKqZ fubar@famine

Added by Google

ssh-rsa AAAA[...]+2LRl test@example.com

AFTER THE UPDATE

$ sudo sh -c 'echo deb http://us-central1.gce.archive.ubuntu.com/ubuntu/ trusty-proposed main restricted universe multiverse >> /etc/apt/sources.list'

$ sudo apt update

$ sudo apt install cloud-init
[...]
Setting up cloud-init (0.7.5-0ubuntu1.23) ...
[...]

$ sudo sh -c 'cat /dev/null > /home/ubuntu/.ssh/authorized_keys'

$ sudo rm -rf /var/lib/cloud/instance*

$ sudo reboot

$ gcloud compute --project "ubuntu-os-support" ssh --zone "us-central1-a" ubuntu1404cloudinittest

$ sudo cat /home/ubuntu/.ssh/authorized_keys
ssh-rsa AAAA[...]+2LRl test@example.com
ssh-rsa AAAA[...]+3LRl test

Added by Google

ssh-rsa AAAA[...]+2LRl test@example.com

cloud-init now works the same way as Ubuntu 16.04 does. Thanks!

Bonus test: I tested that oslogin also works well:

$ gcloud compute instances add-metadata ubuntu1404cloudinittest --metadata enable-oslogin=TRUE --zone "us-central1-a"
Updated [https://www.googleapis.com/compute/v1/projects/ubuntu-os-support/zones/us-central1-a/instances/ubuntu1404cloudinittest].

$ gcloud compute --project "ubuntu-os-support" ssh --zone "us-central1-a" ubuntu1404cloudinittest
[...]
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-137-generic x86_64)
[...]

ubuntu-server-builder · 2023-05-11T14:59:24Z

Launchpad user Łukasz Zemczak(sil2100) wrote on 2018-10-29T09:45:24.259578+00:00

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

ubuntu-server-builder · 2023-05-11T14:59:26Z

Launchpad user Launchpad Janitor(janitor) wrote on 2018-10-29T09:55:28.428004+00:00

This bug was fixed in the package cloud-init - 0.7.5-0ubuntu1.23

cloud-init (0.7.5-0ubuntu1.23) trusty; urgency=medium

debian/control: added python-six dependency.
debian/patches/lp-1781039-gce-datasource-update.patch:
Backport GCE datasource functionality from Xenial (LP: #1781039).

-- Shane Peters shane.peters@canonical.com Tue, 06 Sep 2018 17:57:23 -0400

ubuntu-server-builder added the launchpad Migrated from Launchpad label May 11, 2023

ubuntu-server-builder closed this as completed May 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GCE cloudinit and ubuntu keys from metadata to ubuntu authorized_keys #3208

GCE cloudinit and ubuntu keys from metadata to ubuntu authorized_keys #3208

ubuntu-server-builder commented May 11, 2023

Added by Google

Added by Google

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

GCE cloudinit and ubuntu keys from metadata to ubuntu authorized_keys #3208

GCE cloudinit and ubuntu keys from metadata to ubuntu authorized_keys #3208

Comments

ubuntu-server-builder commented May 11, 2023

Added by Google

Added by Google

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

USING CUSTOM IMAGE WITH UPGRADED CLOUD-INIT

Added by Google

USING EXISTING IMAGE

Added by Google

Added by Google

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023

TEST KEYS

CREATE THE INSTANCE

BEFORE THE UPDATE

Added by Google

AFTER THE UPDATE

Added by Google

ubuntu-server-builder commented May 11, 2023

ubuntu-server-builder commented May 11, 2023