Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nocloud/OVF seedfrom vendordata #570

Merged
merged 5 commits into from Sep 18, 2020

Conversation

jqueuniet
Copy link
Contributor

This PR modifies the util.seedfrom helper function to query optional vendor-data. I started with the idea of implementing for NoCloud, but as OVF is using the same helper, I figured it would be easier to do it for both.

try:
vd_resp = url_helper.read_file_or_url(vd_url, timeout=timeout,
retries=retries)
except url_helper.UrlError:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should log here that there was an url error when reading the vendor-data

except url_helper.UrlError:
pass
else:
if vd_resp.ok():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same thing here in case we receive a response that is not ok

@jqueuniet
Copy link
Contributor Author

I added some basic logging, but not sure if the messages are good enough, or if I can add something else.

@OddBloke OddBloke self-assigned this Sep 18, 2020
Copy link
Collaborator

@OddBloke OddBloke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@OddBloke OddBloke merged commit 6b5c306 into canonical:master Sep 18, 2020
TheRealFalcon added a commit to TheRealFalcon/cloud-init that referenced this pull request Dec 4, 2020
Test that we can add optional vendor-data to the seedfrom file in a
NoCloud environment
@TheRealFalcon TheRealFalcon mentioned this pull request Dec 4, 2020
3 tasks
TheRealFalcon added a commit to TheRealFalcon/cloud-init that referenced this pull request Dec 4, 2020
Test that we can add optional vendor-data to the seedfrom file in a
NoCloud environment
@jfgibbins
Copy link

As it doesn't seem nocloud can parse meta-data for network, would it be also possible to have NoCloud also pull the network-config file from url as well?

@jqueuniet
Copy link
Contributor Author

This is probably not the right place to talk about this as this PR is long closed, but a functional network is a prerequisite for URL fetching.

This was an issue for us as we use iPXE network boot on diskless bare-metal, and in the end the only feasible option in this scenario is passing the network-config as a kernel cmdline argument.

@jqueuniet jqueuniet deleted the nocloud_seedfrom_vendordata branch May 11, 2021 08:53
@jfgibbins
Copy link

jfgibbins commented May 11, 2021

Agreed regarding the place, but I couldn't find where to send an email to bring it up offline. :) Happy to talk about it elsewhere. I can be reached at yahoo, same name. Thanks

@smoser
Copy link
Collaborator

smoser commented May 11, 2021

This is probably not the right place to talk about this as this PR is long closed, but a functional network is a prerequisite for URL fetching.

This was an issue for us as we use iPXE network boot on diskless bare-metal, and in the end the only feasible option in this scenario is passing the network-config as a kernel cmdline argument.

well... other datasources can and do get networking information over the network.

They assume "dhcp on first network devcie" and then get the fuller network config.
The same could be done for nocloud.

@jfgibbins
Copy link

FYI, I'd love to have just put this in as an issue instead of tagging a relevant PR, but that tab seems to be missing on this github project.

@smoser Hey Scott. Haven't talked with you since working on lxc/lxd. Currently using vmware ovf to install a seedfrom. The method of using a cd passed with meta-data, etc is doable, but sloppy, even with bash/govc/cloud-init, and is certainly slower than over the net. Seedfrom does work decently for grabbing user and vendor, but networking has to resort to a write-files of the network config, as well as a network: config-disabled to remain persistent across reboots. Sadly VMware really hasn't put any effort into making cloud-init work well through vsphere. I've looked a lot, but any straight forward methods that don't require create an ISO and attaching has eluded me to date.

Current setup:
govc deploys ova and adds environment variables, including seedfrom (ex, "http://x.x.x.x/*hostname*/")
server boots, grabs user-data and vendor-data
writes previously mentioned files for network config and persists.
meta-data - no use
user-data - host specific config and network-config using write files
vendor-data - generic config
network-config - n/a

Ideal
meta-data - nocloud and ovf don't seem to make any use of usual data found in it (?)
user-data - host specific config
vendor-data - generic config
network-config - obviously network

@smoser
Copy link
Collaborator

smoser commented May 11, 2021

It would definitely be doable to have NoCloud operate entirely at "local" timeframe and just do the EphemeralDHPCv4 path.

it'd be neat.

@jfgibbins
Copy link

@smoser if I understand what you're saying correctly, its the feature of adding a static route through the dhcp.py(?) for an address such as 169.254.169.254 to reach a web server. I could certainly see this working. The things that come to mind are, in an environment like EC2, etc, the local router is actually a software router with additional services, that captures the requests, and by querying IPAM, knows who the client is supposed to be and forwards it to an instance that provides the host specific data. Unless running on a cloud-stack, or it looks like nsx-t might also provide that service, then the issue is how to know which device it is from vsphere(I include the host name as part of the seedfrom url, with mod_rewrite redirect to a generic user-data if not found) and route to the correct meta/user/vendor instance, and of course, the formatting for network config per the environment. My current method does work decently, it's just that hang up of write_files for config that seems so kludgy. My python is extremely limited, as I just started learning it for some basic scripting. (I used python to automate a remote-trigger blackhole with exaBGP, to update bogons, martians, known c&c networks etc from online sources like team cymru, etc).
I'd love to help where I can to improve the vmware cloud-init anyway I can. Certainly with lots of testing an my environment(I use a lot of Ubuntu, but a few others such as fedora, etc). Seems that if ovf could function like nocloud, and nocloud accepted network-config somewhere, the whole thing would be so much better.

@jqueuniet
Copy link
Contributor Author

FYI, I'd love to have just put this in as an issue instead of tagging a relevant PR, but that tab seems to be missing on this github project.

Issues for cloud-init are tracked on Launchpad

This was referenced May 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants