Grow a MicroCloud cluster

[masnax]

Closes #31

Adds microcloud add which allows for rescanning for new cluster members. The process works similarly to init.

To facilitate communication with a system outside the cluster, microcloud now by default will listen on :9443 and will serve thee /1.0 and /cluster/1.0 endpoints. This listener will get replaced with a proper one once the peer has joined.

Additionally the token broadcast has been updated to send a struct, which additionally includes a LXDConfig for assigning MemberConfig to the join request made by the peer.

The /services/disks endpoint is now removed, to be replaced with the source.wipe config field for LXD storage pools.

As per the comments in #46, I've also reworked the join logic to be through the temporary network listener instead of mDNS.

[tomponline]

Do we need to add a version to the mdns advert/response so we can ignore responses from earlier/incompatible machines on the network (now that we are evolving the mdns advert format).

[masnax]

@tomponline I've made some changes now according to your feedback.

Do we need to add a version to the mdns advert/response so we can ignore responses from earlier/incompatible machines on the network (now that we are evolving the mdns advert format).

I think this is a really good idea. I've added a version flag that's appended to the keyword that tells us an mDNS record is from MicroCloud (_microcloud_v1.0). Adding it to the payload wouldn't have worked as older systems would fail the json.Unmarshal call in the first place.

[tomponline]

Adding it to the payload wouldn't have worked as older systems would fail the json.Unmarshal call in the first place.

I don't think thats true. json.Unmarshal will ignore unknown fields unless https://pkg.go.dev/encoding/json#Decoder.DisallowUnknownFields is enabled.

[masnax]

Adding it to the payload wouldn't have worked as older systems would fail the json.Unmarshal call in the first place.

I don't think thats true. json.Unmarshal will ignore unknown fields unless https://pkg.go.dev/encoding/json#Decoder.DisallowUnknownFields is enabled.

However in upstream MicroCloud, the response type is map[string]string which will fail to convert to map[string]struct{} with the error json: cannot unmarshal string into Go value of type <type>

[tomponline]

Adding it to the payload wouldn't have worked as older systems would fail the json.Unmarshal call in the first place.

I don't think thats true. json.Unmarshal will ignore unknown fields unless https://pkg.go.dev/encoding/json#Decoder.DisallowUnknownFields is enabled.

However in upstream MicroCloud, the response type is map[string]string which will fail to convert to map[string]struct{} with the error json: cannot unmarshal string into Go value of type <type>

Ah I see you've changed the type not just the fields. Is map[string]struct{} the old type?

[masnax]

Adding it to the payload wouldn't have worked as older systems would fail the json.Unmarshal call in the first place.

I don't think thats true. json.Unmarshal will ignore unknown fields unless https://pkg.go.dev/encoding/json#Decoder.DisallowUnknownFields is enabled.

However in upstream MicroCloud, the response type is map[string]string which will fail to convert to map[string]struct{} with the error json: cannot unmarshal string into Go value of type <type>

Ah I see you've changed the type not just the fields. Is map[string]struct{} the old type?

Oh actually forget everything I said, you're right. I thought I was still unmarshaling a whole map of structs (I was at some point but not anymore). It's actually one struct at a time now so I can totally just add Version to the ServerInfo struct.

[tomponline]

thanks for the changes @masnax

[stgraber]

@masnax oops, sorry, looks like I broke your branch with my gomod update

[masnax]

@stgraber Fixed now.

[stgraber]

@masnax needs a rebase post MicroOVN

[masnax]

@stgraber Fixed

EDIT: Just added one last commit that enables MicroOVN on the add command too.