On Debian host: Arch OpenStack started failing during cloud-init package creation

[hartwork]

Broken on host OS Debian.
Works on host OS Gentoo.

Is /dev/urandom is missing?:

[..]
# sudo -u nobody makepkg
==> Making package: cloud-init 9999-1 (Wed Jul 19 14:23:21 UTC 2017)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
  -> Downloading cloud-init-0.7.6.tar.gz...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  503k  100  503k    0     0   901k      0 --:--:-- --:--:-- --:--:--  901k
==> Validating source files with sha1sums...
    cloud-init-0.7.6.tar.gz ... Passed
==> Extracting sources...
  -> Extracting cloud-init-0.7.6.tar.gz with bsdtar
==> Starting prepare()...
==> Entering fakeroot environment...
==> Starting package()...
Traceback (most recent call last):
  File "./setup.py", line 27, in <module>
    import setuptools
  File "/usr/lib/python2.7/site-packages/setuptools/__init__.py", line 10, in <module>
    from setuptools.extern.six.moves import filter, map
  File "/usr/lib/python2.7/site-packages/setuptools/extern/__init__.py", line 1, in <module>
    from pkg_resources.extern import VendorImporter
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 36, in <module>
    import email.parser
  File "/usr/lib/python2.7/email/parser.py", line 12, in <module>
    from email.feedparser import FeedParser
  File "/usr/lib/python2.7/email/feedparser.py", line 27, in <module>
    from email import message
  File "/usr/lib/python2.7/email/message.py", line 16, in <module>
    import email.charset
  File "/usr/lib/python2.7/email/charset.py", line 13, in <module>
    import email.base64mime
  File "/usr/lib/python2.7/email/base64mime.py", line 40, in <module>
    from email.utils import fix_eols
  File "/usr/lib/python2.7/email/utils.py", line 27, in <module>
    import random
  File "/usr/lib/python2.7/random.py", line 885, in <module>
    _inst = Random()
  File "/usr/lib/python2.7/random.py", line 97, in __init__
    self.seed(x)
  File "/usr/lib/python2.7/random.py", line 113, in seed
    a = long(_hexlify(_urandom(2500)), 16)
OSError: [Errno 38] Function not implemented
==> ERROR: A failure occurred in package().
    Aborting...
[..]

[hartwork]

Related link:

[python2] OSError: [Errno 38] Function not implemented
https://archlinuxarm.org/forum/viewtopic.php?f=15&t=11647

[hartwork]

@jonhoo
This seems to be a bug in Arch. Can you help getting that fixed upstream?

[jonhoo]

Hmm, cloud-init 0.7.6 is really old, and has many known bugs. You should probably upgrade to 0.7.9 first.

[hartwork]

I would love to, but it's not trivial (I tried) and may not even work in the end. (Arch is not the only distro where post-0.7.6 causes trouble.) If that's something you can help with, that would be welcome.

[jonhoo]

What exactly are the problems you've run into? My cloud-arch works fine with 0.7.9, though did require some tweaks. See in particular jonhoo/cloud-arch@ff53f5f, jonhoo/cloud-arch@0aca753, and jonhoo/cloud-arch@419f3b3.

[hartwork]

There is bug #35 for clound-init >=0.7.7, I worked around the nameservers issue in 6f2a670 earlier, but there also is some Arch-specific frontend to networkd (I think it was Netctl) that cloud-init started using on Arch that I couldn't get to work. So all that combined made me gave up last time.

[hartwork]

PS: Is there work ongoing getting these patches upstream?
PPS: Maybe we need a new ticket for upgrading cloud-init.

[jonhoo]

Ah, on my VMs the network setup is always either static or DHCP, so I've never had to deal much with the cloud-init network setup. I'm surprised that cloud-init has opted for using netctl instead of just systemd-networkd on Arch, given that both are supported, but the latter is way more common. If you know more specifically what the error you observed was, I might be able to help. I also found another upstream bug with a couple of fixes for Arch Linux that may help in this particular case: https://bugs.launchpad.net/cloud-init/+bug/1555605.

As for #35, 'HostnameConf' object has no attribute 'encode' seems to be the python error you're getting. I haven't seen that before, but I believe I found the bug. Filed upstream at https://bugs.launchpad.net/cloud-init/+bug/1705306.

A new ticket is probably a good idea. https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1636912 is marked as "fix released", which strikes me as weird given that the problem is still there on recent releases. https://bugs.launchpad.net/cloud-init/+bug/1663045 doesn't seem to have garnered any attention, so a follow-up comment there might help.

[hartwork]

Broken on host OS Debian.
Works on host OS Gentoo.

Fixed by upgrading build host from jessie to stretch. Should have though of that earlier...