read_seeded broken

[ubuntu-server-builder]

This bug was originally filed in Launchpad as LP: #1455233

Launchpad details

affected_projects = ['cloud-init (Ubuntu)', 'cloud-init (Ubuntu Vivid)']
assignee = None
assignee_name = None
date_closed = 2016-08-10T14:47:36.795825+00:00
date_created = 2015-05-14T20:51:55.559184+00:00
date_fix_committed = 2015-05-14T21:08:46.170275+00:00
date_fix_released = 2016-08-10T14:47:36.795825+00:00
id = 1455233
importance = medium
is_complete = True
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1455233
milestone = None
owner = smoser
owner_name = Scott Moser
private = False
status = fix_released
submitter = smoser
submitter_name = Scott Moser
tags = ['cloud-images', 'verification-done']
duplicates = []

Launchpad user Scott Moser(smoser) wrote on 2015-05-14T20:51:55.559184+00:00

=== Begin SRU Information ===
[Impact]
cloud-init cannot read a "seed". Many of the data-sources allow a seed from a url. This is useful in testing, especially with the NoCloud seed from the command line.

One of the functions in cloud-init simply does not work with python-3, and the patch makes that work. Patch is applied to upstream and present in wily.

[Test Case]
Run the attached test-bug-1455233. Without the patch applied, the system will boot and show error like in the serial.log:
  util.py[WARNING]: Getting data from <class 'cloudinit.sources.DataSourceNoCloud.DataSourceNoCloudNet'> failed

Additionally, a simple test case can be done more directly by simply running
 $ echo "my-userdata" > user-data
 $ echo "instance-id: FOO" > meta-data
 $ python -m SimpleHTTPServer 8999 &
$ python3 -c 'from cloudinit import util; print(util.read_seeded("http://localhost:8999/", retries=0))'
127.0.0.1 - - [09/Jun/2015 01:24:05] "GET /meta-data HTTP/1.1" 200 -
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 842, in read_seeded
if md_resp.ok():
AttributeError: 'str' object has no attribute 'ok'

The added test case in the build process pushes the code through this function.

[Regression Potential]
This code was 100% broken in python3, so likelyhood of regression is very low.
=== End SRU Information ===

util.read_seeded uses load_tfile_or_url, but then treats the return value as if it was a response.

this regressed in revno 1067.

[ubuntu-server-builder]

Launchpad user Scott Moser(smoser) wrote on 2015-05-14T21:08:54.094435+00:00

fixed in trunk at 1102

[ubuntu-server-builder]

Launchpad user Launchpad Janitor(janitor) wrote on 2015-05-15T21:35:54.855943+00:00

This bug was fixed in the package cloud-init - 0.7.7~bzr1106-0ubuntu1

---

cloud-init (0.7.7~bzr1106-0ubuntu1) wily; urgency=medium

• New upstream snapshot.
  • Azure: remove strict dependency on walinux-agent, but still utilize
    it for the time being.
  • fix read_seeded that is used in seeding user-data and meta-data
    from additional locations (LP: #1455233)
  • fix bug preventing partitioning of disks in many cases. (LP: #1311463)
  • Azure: do not override hostname if user has set it (LP: #1375252)
  • Fix GCE datasource not handling per-instance SSH keys (LP: #1403617)
  • Allow specifying of uid in user/group config.

-- Scott Moser smoser@ubuntu.com Fri, 15 May 2015 17:04:19 -0400

[ubuntu-server-builder]

Launchpad user Scott Moser(smoser) wrote on 2015-06-09T01:27:42.341291+00:00

running this will:
a.) download amd64 vivid cloud iamge
b.) convert it to qcow2 (removing compression)
c.) create a datasource seed and put make that available on port 32999
d.) boot a kvm guest with kernel command line pointing at the seed .

running it with PATCH_IMAGE=cloud-inig.deb will patch the image with the provided cloud-init.deb and then run.
Launchpad attachments: test case showing recreate

[ubuntu-server-builder]

Launchpad user Chris J Arges(arges) wrote on 2015-06-10T17:54:28.509275+00:00

Hello Scott, or anyone else affected,

Accepted cloud-init into vivid-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.7~bzr1091-0ubuntu3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[ubuntu-server-builder]

Launchpad user Scott Moser(smoser) wrote on 2015-06-10T20:18:33.412343+00:00

verified.

• downloaded test-bug-1455233 from attachment

• ran it, it downloaded daily/server/vivid/20150609/vivid-server-cloudimg-amd64-disk1.img

• it booted vm, and on console i see failure:
  [ 13.659828] cloud-init[729]: 2015-06-10 18:13:05,254 - util.py[WARNING]: Getting data from <class cloudinit.sources.DataSourceNoCloud.DataSourceNoCloudNet'> failed
  and in serial.log, it shows
  util.py[WARNING]: Getting data from <class 'cloudinit.sources.DataSourceNoCloud.DataSourceNoCloudNet'> failed

• updated the image to -proposed cloud image:

  work around 1463963 to get us some space

  $ sudo mount-image-callback --system-resolvconf --system-mounts
  vivid-server-cloudimg-amd64-disk1.qcow2 -- sh -ec
  'dev=$(awk "$2 == mp { print $1 }" "mp=$MOUNTPOINT" /proc/mounts);
  resize2fs $dev'

  $ sudo mount-image-callback --system-resolvconf --system-mounts
  vivid-server-cloudimg-amd64-disk1.qcow2 --
  chroot MOUNTPOINT sh -c '
  rel=$(lsb_release -sc);
  echo "deb http://archive.ubuntu.com/ubuntu $rel-proposed main" |
  tee /etc/apt/sources.list.d/proposed.list &&
  apt-get update -qy && apt-get install cloud-init grub-legacy-ec2 &&
  apt-cache policy cloud-init'
  ...
  cloud-init:
  Installed: 0.7.7bzr1091-0ubuntu3
  Candidate: 0.7.7bzr1091-0ubuntu3
  Version table:
  *** 0.7.7bzr1091-0ubuntu3 0
  500 http://archive.ubuntu.com/ubuntu/ vivid-proposed/main amd64 Packages
  100 /var/lib/dpkg/status
  0.7.7bzr1091-0ubuntu2 0
  500 http://archive.ubuntu.com/ubuntu/ vivid-updates/main amd64 Packages
  0.7.7~bzr1091-0ubuntu1 0
  500 http://archive.ubuntu.com/ubuntu/ vivid/main amd64 Packages

• ran ./test-bug-1455233
  logged in at login prompt with 'ubuntu' and 'passw0rd'

[ubuntu-server-builder]

Launchpad user Scott Moser(smoser) wrote on 2015-06-10T20:20:13.001139+00:00

Then, just for fun inside the booted system I verified additionally with:
$ python3 -c 'from cloudinit import util; print(util.read_seeded("http://localhost:8999/", retries=0))'
({'instance-id': 'FOO'}, b'my-userdata\n')

[ubuntu-server-builder]

Launchpad user Launchpad Janitor(janitor) wrote on 2015-06-18T16:15:44.427501+00:00

This bug was fixed in the package cloud-init - 0.7.7~bzr1091-0ubuntu3

---

cloud-init (0.7.7~bzr1091-0ubuntu3) vivid-proposed; urgency=medium

• d/README.source, debian/cherry-pick-rev: improve packaging tool
• fix read_seeded method needed for seeding cloud-init via kernel
  command line (LP: #1455233).

-- Scott Moser smoser@ubuntu.com Mon, 08 Jun 2015 15:39:32 -0400

[ubuntu-server-builder]

Launchpad user Brian Murray(brian-murray) wrote on 2015-06-18T16:15:49.364107+00:00

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[ubuntu-server-builder]

Launchpad user Scott Moser(smoser) wrote on 2016-08-10T14:47:50.406994+00:00

This is fixed in cloud-init 0.7.7