cloud-init devel net-convert crash when --debug is enabled

[ubuntu-server-builder]

This bug was originally filed in Launchpad as LP: #1975907

Launchpad details

affected_projects = []
assignee = chad.smith
assignee_name = Chad Smith
date_closed = 2022-08-19T16:37:17.133437+00:00
date_created = 2022-05-27T07:53:30.880247+00:00
date_fix_committed = 2022-06-01T16:22:34.416199+00:00
date_fix_released = 2022-08-19T16:37:17.133437+00:00
id = 1975907
importance = high
is_complete = True
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1975907
milestone = None
owner = ben-hesmans
owner_name = Benjamin Hesmans
private = False
status = fix_released
submitter = ben-hesmans
submitter_name = Benjamin Hesmans
tags = []
duplicates = []

Launchpad user Benjamin Hesmans(ben-hesmans) wrote on 2022-05-27T07:53:30.880247+00:00

Since 22.2 enabling "--debug" for "cloud-init devel net-convert" will make cloud-init crash.

Probably linked to 3e5938c and the use of safe dumper.

Stack trace shows:
Traceback (most recent call last):
File "/xyz/git/cloud-init/bin/cloud-init", line 8, in
sys.exit(main())
File "/xyz/git/cloud-init/lib/python3.8/site-packages/cloudinit/cmd/main.py", line 1059, in main
retval = util.log_time(
File "/xyz/git/cloud-init/lib/python3.8/site-packages/cloudinit/util.py", line 2637, in log_time
ret = func(*args, **kwargs)
File "/xyz/git/cloud-init/lib/python3.8/site-packages/cloudinit/cmd/devel/net_convert.py", line 136, in handle_args
"\n".join(["", "Internal State", safeyaml.dumps(ns, noalias=True), ""])
File "/xyz/git/cloud-init/lib/python3.8/site-packages/cloudinit/safeyaml.py", line 161, in dumps
return yaml.dump(
File "/xyz/git/cloud-init/lib/python3.8/site-packages/yaml/init.py", line 253, in dump
return dump_all([data], stream, Dumper=Dumper, **kwds)
File "/xyz/git/cloud-init/lib/python3.8/site-packages/yaml/init.py", line 241, in dump_all
dumper.represent(data)
File "/xyz/git/cloud-init/lib/python3.8/site-packages/yaml/representer.py", line 27, in represent
node = self.represent_data(data)
File "/xyz/git/cloud-init/lib/python3.8/site-packages/yaml/representer.py", line 58, in represent_data
node = self.yaml_representers[None](self, data)
File "/xyz/git/cloud-init/lib/python3.8/site-packages/yaml/representer.py", line 231, in represent_undefined
raise RepresenterError("cannot represent an object", data)
yaml.representer.RepresenterError: ('cannot represent an object', <cloudinit.net.network_state.NetworkState object at 0x7fa7979ec340>)

i tried to replace to dumper with the unsafe version and it was working again

[ubuntu-server-builder]

Launchpad user Chad Smith(chad.smith) wrote on 2022-05-28T03:18:18.060521+00:00

Thank you for this bug and making cloud-init better and for the additional commit reference as a guess for when regression was introduced.
You are correct that I reintroduced SafeDumper default and that's what broke the cloud-init devel net-convert command.

Because the original commit cf30836 had inadvertently switched to using yaml.Dumper when dropping the direct yaml.safe_dump() call we deemed it reasonable to switch that dumper back to the original SafeDumper as all runtime callsites provide only simple types and not python objects.

Turns out, the command cloud-init devel net-convert --debug does attempt to stuff in a python object as you noted NetworkState.

I have reviewed all runtime call-sites to cloudinit.safeyaml.dumps and all of the objects that cloud-init currently provides are dictionaries of simple dicts with the exception of this devel --debug command.

Here is an upstream PR to fix this one callsite and add a unit test so we recognize this type of failure in the future. #1484

[ubuntu-server-builder]

Launchpad user Benjamin Hesmans(ben-hesmans) wrote on 2022-05-30T07:38:31.058396+00:00

Thank you for the quick answer! I tried with your patch, and it looks good. Thx!

(quick test for info)
$ git logo -1
610d113 (HEAD, chad/net-convert) net-convert: use yaml.dump for debugging python NetworkState obj
[May 30 09:34AM] 0m0s
~/git/cloud-init ((HEAD detached at chad/net-convert)) :)
$ cloud-init devel net-convert --debug -p/tmp/network-config -k yaml -D centos && echo ok
(...)
ok

[ubuntu-server-builder]

Launchpad user Chad Smith(chad.smith) wrote on 2022-06-01T16:22:05.735071+00:00

Thanks for the test here Benjamin upstream commit landed with this fix. It will be included in cloud-init upstream version 22.3.

153a7ea

[ubuntu-server-builder]

Launchpad user Brett Holman(holmanb) wrote on 2022-08-19T16:37:18.102674+00:00

This bug is believed to be fixed in cloud-init in version 22.3. If this is still a problem for you, please make a comment and set the state back to New

Thank you.