refresh_facts.rb does not expand aliases

[smortex]

After adding a new module to the control repo, we experienced failures from the code of https://github.com/choria-io/mcorpc-ruby-support :

yaml_facts.rb:35:in `rescue in block in load_facts_from_source' Failed to load yaml facts from /etc/puppetlabs/mcollective/generated-facts.yaml: Psych::BadAlias: Unknown alias: 1

The module provide two facts that contain a reference to the same object. The generation of generated-facts.yaml detects this and produce an alias instead of outputing the same content twice.

But then, mcorpc-ruby-support cannot read this file because SafeYaml does not support alias expansion for safety reasons.

The script that build generated-facts.yaml should probably make sure no aliasing happen.

[ripienaar]

Nasty. Now that the ruby mco is gone we can probably make this script emit JSON which would make me much happier

[smortex]

From my findings digging this issue:

• A quick & dirty workaround is to convert the yaml to json and then back to yaml (ugly);
• There is not safe_dump method in Psych (the YAML module, bundled with Ruby if I recall correctly), so as of today, you cannot generate a YAML file that can be read with Psych.safe_load. I started hacking a safe_dump method, this involves quite a huge amount of code, so it's really not trivial.

So if the same could be done with JSON, maybe it would be the fastest cleanest way to proceed…

[ripienaar]

I see there is a :canonical option to dump, maybe that helps?

Go choria supports reading json or yaml - and anyway since this is run from cron not that frequently we can do the to json to yaml thing

[smortex]

I tried it but it does not solve the issue, we still have aliases…

non canonical:
---
a: &1
- 1
- 2
- 3
b: *1
canonical:
---
{
  ? "a"
  : &1 [
    ! "1",
    ! "2",
    ! "3",
  ],
  ? "b"
  : *1,
}

I just checked the choria code and feel like saving JSON to a filename that ends with "json" (at least not "yaml") and adjusting the hiera data to adjust the filename should be good. I'll try to take some time for this. Thanks!

[ripienaar]

Yeah json will work if you want to do that great, else we can do the hack I don’t mind

[smortex]

Busy day… switching to JSON is not as easy as I though, and we do have an ARM node with legacy mco server and legacy Puppet 5 in production that would be unhappy with such changes in the short term, so I am going the easy way with the hack solution.

For a long term solution, saving JSON indeed makes sense!

[ripienaar]

sweet, thank you