Ability to overwrite goss config/results

[taurus-forever]

Hi,

We are using goss to control our servers in production.
We have a list of tests which are executed on every server and some server specific checks.
We have separated them in a way:

cat goss.yaml 
gossfile:
  common.yaml: {}
  server_specific.yaml: {}

It works great.

Also we noticed that checks in the second file does overwrite checks in common which is very useful.
Sample: if "common.yaml" says A must NOT exist and "server_specific.yaml" redefines that saying A must exist. It does the job well. All servers are being verified to have lack of A and some specific server verifies A is really exist.

It is very useful for a users check, like "puppet" user must not exist on all servers while it must exist on puppet server itself. In this case we have nice huge common checks file and tune the checks in server_specific one redefining check results.

Also it is useful to temporary disable some check on some server for maintenance reason, etc.

The problem here: the results are not stable :-(
Sometimes (once in 10-20 checks) goss produces failed alarm.

I believe it is due to the fact that there is no real redefinition in code. Goss just loads both YML files common and server_specific and executes all tests in parallel. Having the fact that test A from common file started earlier -> it also finished earlier with failed test results, then test A from server_specific overwrites results with possitive state. While sometimes test A from common file finished later then test A from server_specific as a result goss reports failed here.

So, my question is: should we stop using that way or maybe it will be useful to make that somehow stable, at it looks very useful. Every next file loaded in goss.yaml can overwrite expected tests results. You can merge tests from all files and then executes them once. In this case extected tests results will come from the last file in goss.yaml.

Tnx!

[aelsabbahy]

If you run goss render multiple times on your goss file it will give you different results, because golang will randomize dictionary order. This behavior is confusing and non-deterministic, so I'm marking this as a bug.

One way to make this consistent is to sort the imported gossfiles, which would make imports more consistent and allow the user to specify order, ex:

gossfile:
  01_common.yaml: {}
  02_server_specific.yaml: {}

Later files would overwrite content from earlier files. Would this resolve your issue?

[aelsabbahy]

Haven't heard back from you.. I merged in this code change and will cut a release tomorrow. Hopefully it helps with your issue. If not, feel free to re-open.

[taurus-forever]

I am sorry for delay with reply here,
wrote reply on my way and for some reason didn't post it.

Yes, I believe it should be good enough for our needs!
Thank you!

[aelsabbahy]

Fixed in v0.2.4