RF[DE]: Drop the use of structs for the YAML config data

[griznog]

Was discussing this with @anderbubble yesterday and wanted to see how it might work. The idea is that instead of using a struct for the NodeInfo, that all the data in the nodes.conf would be left unstructured. A tiny example of this is in the gist https://gist.github.com/griznog/fab22104ed47ab66d512286609fa8caa (with mad props to my coding partner ChatGPT4 who did the heavy lifting there.)

The code there reads a sample nodes.conf pulled from a subset of my cluster config and uses a modified ifcfg.ww template to dump out the network config for the nodes in the nodes.conf. This approach would allow admins complete freedom in what they can add/use in the node config, including nesting any arbitrary tags as deeply as needed.

As this approach would mean that WW no longer enforces a structure on the node or cluster config, the WW and/or OpenHPC projects could then provide working "default" schemas and templates for cluster environments, the community can then vote with its feet on how to best organize a cluster config. Along with a repository for overlays, this would allow more easy exploration of cluster-management-space and development of more complex config-driven templates/overlays.

[anderbubble]

instead of using a struct for the NodeInfo

A small point, but I think you're talking about NodeConf, not NodeInfo.

[griznog]

Right, I mix those up. In my thought experiment it'd work such that the NodeConf gets pulled into the map[string]interface{} which could then be updated, written, re-read as needed. Then the current NodeInfo would just be a given node pulled out of that map[string]interface{} with the profiles it uses merged into it or have NodeInfo be just "real" nodes with all the profile merging done on them and regenerate that any time the upstream config is modified or re-read.

[griznog]

Another point that just occurred to me, there could/should be a special nodeprofile entry, call it NODESCHEMA, which defines what a node structure should look like (per what included "official" WW overlays expect from nodes/nodeprofiles). That'd allow verifying that node configs have the correct/expected structure while still leaving things completely open-ended for individual sites to replace/modify/override things as needed.

[jcsiadal]

1. This is a prime opportunity to move to the v3 YAML parser.
2. Then we switch to import everything as a map. You grab and modify the fields you need. When you marshal the data, all of the map gets pushed, even the stuff that was ignored, like user comments.

[griznog]

2. Then we switch to import everything as a map. You grab and modify the fields you need. When you marshal the data, all of the map gets pushed, even the stuff that was ignored, like user comments.

I couldn't wrap my head around how/when to manage writing new values out properly, but what you state here makes that a very simple process, any wwctl operation that changes the config just triggers marshal'ing the data out to disk. The map becomes a defacto in-memory database representation of the YAML config data and preserving everything for the round trip makes that trivial to keep those in sync. If warewulfd locked that file and offered a method for handing that lock to an editor and then re-reading the file on close (with syntax checking/rejecting edits as with visudo, vipw, ...) then this pretty elegantly handles keeping that file consistent while allowing direct edits via a user defined editor/tool. And maintaining comments is a huge bonus. With that file locked only for writes any locally developed tools/processes can still read from it at will with a pretty solid guarantee it's current.