Cloud-init on Xenial won't deploy compressed content in "write_files" - says DecompressionError: Not a gzipped file

[ubuntu-server-builder]

This bug was originally filed in Launchpad as LP: #1565638

Launchpad details

affected_projects = ['cloud-init (Ubuntu)']
assignee = None
assignee_name = None
date_closed = 2016-08-10T14:52:05.858301+00:00
date_created = 2016-04-04T07:10:29.655318+00:00
date_fix_committed = 2016-04-06T00:45:04.836363+00:00
date_fix_released = 2016-08-10T14:52:05.858301+00:00
id = 1565638
importance = medium
is_complete = True
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1565638
milestone = None
owner = oded-geek
owner_name = Oded Arbel
private = False
status = fix_released
submitter = oded-geek
submitter_name = Oded Arbel
tags = []
duplicates = []

Launchpad user Oded Arbel(oded-geek) wrote on 2016-04-04T07:10:29.655318+00:00

I'm trying to deploy an Amazon EC2 instance using cloud-init.

When I'm compressing the files in the write_files section, the cloud-init process doesn't write the files, and I get the following error in the cloud-init.log :

Apr 3 16:27:16 ubuntu [CLOUDINIT] handlers.py[DEBUG]: finish: init-network/config-write-files: FAIL: running config-write-files with frequency once-per-instance
Apr 3 16:27:16 ubuntu [CLOUDINIT] util.py[WARNING]: Running module write-files (<module 'cloudinit.config.cc_write_files' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py'>)
failed
Apr 3 16:27:16 ubuntu [CLOUDINIT] util.py[DEBUG]: Running module write-files (<module 'cloudinit.config.cc_write_files' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py'>) failed
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 393, in decomp_gzip
return decode_binary(gh.read())
File "/usr/lib/python3.5/gzip.py", line 274, in read
return self._buffer.read(size)
File "/usr/lib/python3.5/gzip.py", line 461, in read
if not self._read_gzip_header():
File "/usr/lib/python3.5/gzip.py", line 409, in _read_gzip_header
raise OSError('Not a gzipped file (%r)' % magic)
OSError: Not a gzipped file (b"b'")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 735, in _run_modules
freq=freq)
File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 70, in run
return self._runners.run(name, functor, args, freq, clear_on_fail)
File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 199, in run
results = functor(*args)
File "/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py", line 39, in handle
write_files(name, files, log)
File "/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py", line 74, in write_files
contents = extract_contents(f_info.get('content', ''), extractions)
File "/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py", line 98, in extract_contents
result = util.decomp_gzip(result, quiet=False)
File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 400, in decomp_gzip
raise DecompressionError(six.text_type(e))
cloudinit.util.DecompressionError: Not a gzipped file (b"b'")

I've verified that the cloud init I'm submitting does encode the files correctly by running this ruby code on the cloud-init YAML file:

File.write("test.gz", YAML.load(File.read("test.init"))["write_files"].first["content"])

Then:

$ file test.gz
test.gz: gzip compressed data, last modified: Mon Apr 4 06:55:53 2016, max compression, from Unix
$ python
Python 2.7.11+ (default, Mar 30 2016, 21:00:42)
[GCC 5.3.1 20160330] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import gzip
with gzip.open('test.gz', 'rb') as f:
... file_content = f.read()
... print file_content
...

I've never implemented this with trusty, so I'm not sure how cloud-init on trusty handles that.

[ubuntu-server-builder]

Launchpad user Oded Arbel(oded-geek) wrote on 2016-04-04T10:36:55.850690+00:00

After disabling compression of write_files it works fine - just compressed file don't work.

[ubuntu-server-builder]

Launchpad user Scott Moser(smoser) wrote on 2016-04-06T00:45:03.463510+00:00

fixed in trunk at 1199.

[ubuntu-server-builder]

Launchpad user Launchpad Janitor(janitor) wrote on 2016-04-06T21:02:33.615327+00:00

This bug was fixed in the package cloud-init - 0.7.7~bzr1200-0ubuntu1

---

cloud-init (0.7.7~bzr1200-0ubuntu1) xenial; urgency=medium

• New upstream snapshot.
  • fix adding of users when no group is specified (LP: #1562918)
  • fix write_files with compressed content (LP: #1565638)
  • systemd: do not specify After of obsolete syslog.target (LP: #1536964)
  • chef: fix chef installation from gems (LP: #1553345)
  • disk_setup: correctly send force to mkfs on block devices (LP: #1548772)
  • locale: list unsupported environment settings in warning (LP: #1558069)
  • fix NoCloud and other datasources if no datasource config (LP: #1514407)

-- Scott Moser smoser@ubuntu.com Wed, 06 Apr 2016 13:07:27 -0400

[ubuntu-server-builder]

Launchpad user Scott Moser(smoser) wrote on 2016-08-10T14:52:04.433452+00:00

This is fixed in cloud-init 0.7.7