cloud-init does not use interfaces.d in trusty

[ubuntu-server-builder]

This bug was originally filed in Launchpad as LP: #1315501

Launchpad details

affected_projects = ['cloud-init (Ubuntu)']
assignee = None
assignee_name = None
date_closed = 2016-08-10T14:44:27.455468+00:00
date_created = 2014-05-02T19:42:47.968871+00:00
date_fix_committed = 2016-06-10T16:49:58.977760+00:00
date_fix_released = 2016-08-10T14:44:27.455468+00:00
id = 1315501
importance = high
is_complete = True
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1315501
milestone = None
owner = dmsimard
owner_name = David Moreau Simard
private = False
status = fix_released
submitter = dmsimard
submitter_name = David Moreau Simard
tags = []
duplicates = []

Launchpad user David Moreau Simard(dmsimard) wrote on 2014-05-02T19:42:47.968871+00:00

Hi,

Reference/context: https://ask.openstack.org/en/question/28297/cloud-init-nonet-waiting-and-fails/

The trusty image provided by http://cloud-images.ubuntu.com/trusty/ contains an eth0 interface configured as dhcp in /etc/network/interfaces.d/eth0.cfg.
When I boot this image in an Openstack non-dhcp networking environment, cloud-init configures the static IP provided by Neutron directly in /etc/network/interfaces (not interfaces.d).

This means I now have two eth0 devices configured, in two different files.
Booting 20 VMs with the same image yields around 50-60% of VMs that are not reachable by network.

Soft rebooting a VM in this state or doing and "ifdown eth0 && ifup eth0" will make it ping.

I removed the the eth0 interface file in /etc/network/interfaces.d/eth0.cfg from the image, booted another round of VMs and all of them worked fine.

Now, I see three possible outcomes:

• If eth0 is present in /etc/network/interfaces.d, cloud-init configures/re-configures that interface
• If eth0 is present in /etc/network/interfaces.d, cloud-init deletes it and configures /etc/network/interfaces
• Ubuntu cloud images ships without eth0 being configured by default

[ubuntu-server-builder]

Launchpad user Scott Moser(smoser) wrote on 2014-05-13T16:39:56.004143+00:00

Your 'ask' comment at
https://ask.openstack.org/en/question/28297/cloud-init-nonet-waiting-and-fails/

/etc/network/interfaces shows:

Injected by Nova on instance boot

If that entry was "injected" by nova, then there really isnt much or anything cloud-init can do about this.
This is a good example about why host "injection" is inherently flawed. The right fix for your problem is then to have nova realize that 'eth0' already existed, and remove /etc/interfaces.d/. That is clearly brittle and requires updating your hypervisor/cloud which is quite unreasonable.

if cloud-init reads network-interfaces from config drive, then it should handle eth0 correctl (ie, we need to fix that).

[ubuntu-server-builder]

Launchpad user David Moreau Simard(dmsimard) wrote on 2014-05-13T16:55:12.044418+00:00

Hi Scott,

You're correct, we use config drive in our implementation (using_config_drive=True for nova-compute).
The template used by nova is located here: https://github.com/openstack/nova/blob/master/nova/virt/interfaces.template

I also see your point that Nova could also implement a fix for this. I will check with them and refer to this bug.

[ubuntu-server-builder]

Launchpad user David Moreau Simard(dmsimard) wrote on 2014-05-13T17:01:01.349006+00:00

Adding the openstack nova bug for cross reference: https://bugs.launchpad.net/nova/+bug/1319117

[ubuntu-server-builder]

Launchpad user Scott Moser(smoser) wrote on 2014-05-13T18:13:47.426129+00:00

config drive != injection.

if nova "injected" the file (placed it in /etc/network/interfaces) then there isn't anything I can do. Essentially, "nova broke your image".

If nova placed the /etc/network/interfaces file on to a config drive, and cloud-init read it and wrote /etc/network/interfaces then cloud-init was at fault.

Both paths are possible. I generally believe the first one to be simply wrong and don't care if it is broken, the answer is "don't inject files into an image".

We'll fix the second path in cloud-init.

[ubuntu-server-builder]

Launchpad user Zoltan Arnold Nagy(zoltan) wrote on 2014-05-30T13:23:25.114930+00:00

Any ETA on a fix for cloud-init?

[ubuntu-server-builder]

Launchpad user Tobias(tobik) wrote on 2014-09-11T19:51:41.531954+00:00

Any update on that issue? Is there anybody who can suggest a work around?
Thank you!

[ubuntu-server-builder]

Launchpad user Pradeep Chandrasekar(pradeepcsekar) wrote on 2014-10-20T18:26:55.155538+00:00

Any workaround or ETA please?

[ubuntu-server-builder]

Launchpad user Vikram Hosakote(vhosakot) wrote on 2014-12-30T00:59:57.006590+00:00

Is there any workaround for this bug ?

eth0 does not get its IP and the VM cannot be pinged or SSH'ed into.

cloud-init-nonet[13.74]: waiting 120 seconds for network device
cloud-init-nonet[133.74]: gave up waiting for a network device.
Cloud-init v. 0.7.5 running 'init' at Mon, 29 Dec 2014 17:33:38 +0000. Up 133.89 seconds.
ci-info: +++++++++++++++++++++++Net device info++++++++++++++++++++++++
ci-info: +--------+-------+-----------+-----------+-------------------+
ci-info: | Device | Up | Address | Mask | Hw-Address |
ci-info: +--------+-------+-----------+-----------+-------------------+
ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | . |
ci-info: | eth0 | True | . | . | fa:16:3e:e9:35:31 |
ci-info: +--------+-------+-----------+-----------+-------------------+
ci-info: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Route info failed!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

[ubuntu-server-builder]

Launchpad user Diego Carrera Gallego(diegocarrera2000) wrote on 2015-03-25T18:12:53.182671+00:00

i got same error on new instalation with Ubuntu 14.04.

Any ETA on a fix for cloud-init?

[ubuntu-server-builder]

Launchpad user Joshua Holmes(ethode) wrote on 2015-04-06T21:41:10.802957+00:00

I've tried 12.04 and have the same issue

[ubuntu-server-builder]

Launchpad user David Moreau Simard(dmsimard) wrote on 2015-04-21T20:08:59.929023+00:00

Lots of people asking for workarounds, I figured I would post mine - it involves deleting the eth0 file from /etc/network/interfaces.d prior to using the image. So it goes a bit like this:

Install qemu-utils

apt-get install qemu-utils

Mount the Ubuntu cloud image

modprobe nbd max_part=63
qemu-nbd -c /dev/nbd0 trusty-server-cloudimg-amd64-disk1.img
mount /dev/nbd0p1 /mnt/
rm /mnt/etc/network/interaces.d/eth0
umount /mnt

Upload and use the edited image

[ubuntu-server-builder]

Launchpad user Bobby Yakovich(bgyako) wrote on 2015-04-22T18:44:26.825224+00:00

That would require access to instance, how do you get into instance?

[ubuntu-server-builder]

Launchpad user David Moreau Simard(dmsimard) wrote on 2015-04-23T04:04:49.199165+00:00

@bobby

My workaround is applied to the image the instance is spawned with. You must modify the image, upload that modified image and you will be able to successfully boot VMs without this issue.

If you do not have access to do that, the only other way I would see would be to spawn a VM using the image with this problem but pass user-data to cloud-init to grant you console access, a bit like this:

#cloud-config
users:

• name: root
  lock-passwd: false

chpasswd:
list: |
root:root
expire: false

This will allow you to login as root (with password 'root') in the VM console where you'll be able to either delete the extra eth0 file or diagnose the problem from there.

[ubuntu-server-builder]

Launchpad user mahmoh(mahmoh) wrote on 2016-03-01T01:46:16.780342+00:00

@smoser, hi any outlook on the fix for this? Thanks.

[ubuntu-server-builder]

Launchpad user Scott Moser(smoser) wrote on 2016-06-10T16:49:58.128882+00:00

marking fix-commited in cloud-init as revno 1225.
fix released in yakkety images and will make it back to xenial via sru.

[ubuntu-server-builder]

Launchpad user Debdiptaghosh(debdipta1078) wrote on 2016-08-09T11:26:22.364321+00:00

@scott

In which build this issue is fix?
In https://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-disk1.img
I have found same issue

[ubuntu-server-builder]

Launchpad user Scott Moser(smoser) wrote on 2016-08-10T14:44:25.984245+00:00

This is fixed in cloud-init 0.7.7

[ubuntu-server-builder]

Launchpad user Roy Zuo(roylez) wrote on 2017-01-06T01:58:08.884689+00:00

@scott,

Could you please backport the fix to trusty image? Trusty image is still widely used, and this still affects many until it is fixed in trusty cloud image.

[ubuntu-server-builder]

Launchpad user Scott Moser(smoser) wrote on 2017-01-20T18:44:23.325378+00:00

Roy,
I'm attaching a suggested patch for trusty. I've not tested it, and my only confidence in it is that the changes are down a very particular path (where openstack provides network config).

If you're interested in pushing this a bit further, please grab that patch and build and give it a try.

[ubuntu-server-builder]

Launchpad user Alex Gottschalk(alex-gottschalk) wrote on 2017-03-14T20:07:27.548817+00:00

This bug was originally reported for Trusty, and my organization is experiencing it with Trusty. I would note that 14.04 LTS still has two years left in its support cycle. Do you plan on releasing a fix for this?

[ubuntu-server-builder]

Launchpad user Scott Moser(smoser) wrote on 2018-09-14T19:36:41.990167+00:00

@alex,
sorry for the slow reply.

Servicing 14.04 could still be done, but doing so would really require some effort on the person wanting bug fixes in.

I had intended in comment 19 to point at my branch https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+ref/bug/1315501-trusty-openstack-interfaces but failed to do so.

Testing that would improve the potential of this getting backported to 14.04.

Launchpad attachments: git diff for posterity versus my branch.