Integrate into cloud-init upstream

[prologic]

Can we consider integration this datasource into upstream cloud-init? We would get much better support and more contributors.

[akutz]

Hi @prologic,

It has been considered. I even attended Cloud Con to discuss it. However, there is already a VMware data source (OVF) in upstream cloud-init, and at this time my plan is to try and move much of the functionality from this data source into that one. It will be the path of least resistance to try and achieve the same result.

[prologic]

Curious why there would be any resistance? 🤔

[akutz]

Because the current datasource is already present in the cloud-init repo, has a team of people supporting it (instead of just me with this one), and can perhaps work out how to also provide some of the functionality this one does without the dependency on netifaces (though I think the cloud-init folks are actually fine with this dependency, so maybe it stays?). Either way, I am still committed to seeing the functionality this DS provides in cloud-init proper -- it just may come as part of the OVF DS instead.

[prologic]

I don't see a problem actually. I've spoken with and debugged some really crazy problems with vSphere, open-vm-tools and cloud-init with them. We should just try to get this module into upstream pronto. The OVH datasource is a very different beast (is my understanding) and something we're not really using at all in our infra.

[jaymzmac]

I think the existing OVF datasource is going to need a ton of work in order to match the functionality that this datasource provides, and it seems like a more complex task compared to getting this datasource in upstream cloud-init.

Last time I played with the OVF datasource (about 6 months ago) there were numerous issues:

• it didn't support vendor-data, or meta-data being passed to cloud-init.
• it didn't maintain the same instance-id after reboots, which leads to network configuration reverting to dhcp. see https://bugs.launchpad.net/cloud-init/+bug/1835205/ and https://kb.vmware.com/s/article/71264
• the datasource was complex because it was trying to maintain support for customization via vmware-imc (guest os customization through vsphere)
• development for ovf datasource didn't seem particularly active with regards to fixing these issues.

Here was a comment (admittedly a couple of years old now) from a cloud-init engineer regarding the OVF datasource:
"The OVF datasource is going to need quite a bit of work to sort out which paths are working and which are not; having the platform provide a consistent set of metadata to the instance in a reliable form for the life-time of the instance is a requirement. This doesn't appear to be happening via the Imc path."

I believe there is a lot of interest and willingness to help get this datasource to cloud-init upstream (regardless of whether it's a new ds or it's provided through the existing ovf ds). So as long as someone can drive the work and organise a plan, I'm sure help would be offered from the community.

[toabctl]

@akutz could you just propose this datasource upstream to get a discussion started there? Having this 3rd party datasource is just annoying...

[akutz]

canonical/cloud-init#953

[akutz]

FYI, I'm renaming this DatasourceVMware as it merges into cloud-init. This is consistent with the pattern that the DS have a suffix of the vendor name.

[akutz]

Merged upstream into cloud-init

[prologic]

Merged upstream into cloud-init

Congrats! 🎉 And thank you for the time and effort you put into this! Now we can (hopefully soon) streamline our Packer images with VMWare Datasource baked into our images!

[toabctl]

Thanks a lot @akutz !!!

[akutz]

The thanks is to y'all and this awesome community I had a part in helping build. The DS would be nothing without your work and commitment, so thank y'all, truly.