Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

openstack provider: feature request: option to disable floating IP allocation #855

Closed
darrylweaver opened this issue Mar 1, 2016 · 11 comments
Labels

Comments

@darrylweaver
Copy link

@darrylweaver darrylweaver commented Mar 1, 2016

Currently the Openstack provider will attempt to allocate a floating IP to all Virtual Machines on creation.
Some public clouds and many private cloud deployments will either be short of real floating IPs or have no need to use floating IPs at all, if networks are routed directly from the workstation running perfkit.

Therefore, there should be an option to disable the allocation of a floating IP and rely on only the internal IP only.

A flag such as "--openstack-disable-floating-ip=true" as an option for the openstack provider, which turns off the allocation of the floating IP and the only sets the internal IP address.

All benchmarks should then use the internal address only and only run the ip_address=internal benchmarks and disable the ip_address=external benchmark runs.

@voellm

This comment has been minimized.

Copy link
Collaborator

@voellm voellm commented Mar 1, 2016

You can force internal IP allocation today with --ip_addresses=INTERNAL for
all benchmark runs. Does that work for you?

On Tue, Mar 1, 2016 at 4:41 AM, Darryl Weaver notifications@github.com
wrote:

Currently the Openstack provider will attempt to allocate a floating IP to
all Virtual Machines on creation.
Some public clouds and many private cloud deployments will either be short
of real floating IPs or have no need to use floating IPs at all, if
networks are routed directly from the workstation running perfkit.

Therefore, there should be an option to disable the allocation of a
floating IP and rely on only the internal IP only.

A flag such as "--openstack-disable-floating-ip=true" as an option for the
openstack provider, which turns off the allocation of the floating IP and
the only sets the internal IP address.

All benchmarks should then use the internal address only and only run the
ip_address=internal benchmarks and disable the ip_address=external
benchmark runs.


Reply to this email directly or view it on GitHub
#855.

Anthony F. Voellm (aka Tony)
Google Voice: (650) 516-7382
https://www.google.com/voice/b/0?pli=1#phones
Blog: http://perfguy.blogspot.com/
Benchmark like a pro... PerfKitBenchmarker
https://github.com/GoogleCloudPlatform/PerfKitBenchmarker

@meteorfox

This comment has been minimized.

Copy link
Collaborator

@meteorfox meteorfox commented Mar 1, 2016

@darrylweaver Sorry for hijacking your post.

@voellm Looking at the code it seems it will try to allocate the floating IP address regardless, even when it later only uses the INTERNAL IP

The network allocation in the OpenStack provider implementation needs more work IMHO, you can actually have multiple networks, not just a public and/or private networks. Some of the networks might be allocated from a floating IP pool others give you a fixed IP.

We can work on a short-term solution to address the current issue but we also need to take a step back and look at the big picture to tackle the problems at the root.

I suggest that we need to approach OpenStack from a different perspective. I think we should avoid treating it like a public cloud service provider and assume that all OpenStack cloud deployments are the same. In the wild, there are different versions that have already reached EOL(end of life)[1] like Icehouse, Juno, and soon (Kilo), and not all deployments run all services. Part of the issues in #854 is also related to the mismatch of versions.

To give you an idea of this matrix. Here's list a few things (not exhaustive) that could be different on OpenStack Deployments

  • Custom flavors (aka. machine types).
  • Services, images, users, SSH keys, security groups only available within specific regions.
  • 1 or more regions and zones.
  • Not all services might run the latest versions of their API, and others might have extended APIs enabled.
  • Custom types available to choose from for remote block storage volumes. (LVM, Ceph, etc)
  • 1 or more networks, not necessarily just public vs service.
  • No support for floating IPs at all.
  • etc

Because of all this, I think we need as a community to decide which versions to support, and what exactly should we assume and/or not assume from an OpenStack cloud deployment.

Perhaps a good starting point is to look at the OpenStack DefCore and RefStack effort, which tries to define a minimum set of features that OpenStack cloud must be able to support to brand itself as an OpenStack Powered or OpenStack Compatible cloud.

More info here http://www.openstack.org/brand/interop/

Here are some charts from the October 2015 OpenStack Public User Survey Report[2] showing the current landscape.

screenshot from 2016-03-01 10-59-42

screenshot from 2016-03-01 11-00-05

[1] http://releases.openstack.org/
[2] https://www.openstack.org/assets/survey/Public-User-Survey-Report.pdf

@darrylweaver

This comment has been minimized.

Copy link
Author

@darrylweaver darrylweaver commented Mar 1, 2016

I tried the --ip_addresses=INTERNAL and I can confirm that it tries to allocate floating IPs for every single machine. This results in eventually running out of floating IPs and no more benchmarks being processed. I ended up commenting out the floating IP allocation code, so I could continue to get data.

With regards to the broader discussion of what should be supported, then it will be impossible to support every single possible deployment of Openstack, however, there are core components that should be supported, including neutron in several common configurations.

The built in Openstack provider should be useful for most private cloud deployments based on common deployment scenarios, such as using Nova/Cinder/Neutron and should be easily configurable per "Openstack" cloud, using the flags.
The custom flavors can already be specified using --machine_type
Regions/Zones is already being discussed elsewhere and I would advocate having an additional region parameter for all cloud providers and make zone=region for those that do not use a distinct separation.
Many Openstack deployments use regions, but not zones, surprisingly, and private clouds will often not set zones either, so it is the default: nova.

Most deployments will have a private and public network and floating IPs, but some have only 1 network and all IPs are public and others just don't issue you many public IPs by default. So, the most common use case is to have the ability to turn on/off floating IPs. Supporting many networks sounds like a complex scenario to me, that may be served better by customising the provider.

What I would prefer to see overall, is to make it easy to customise the existing Openstack provider to create a new provider based on the Openstack provider with any special customisations required, so that additional public cloud providers based on Openstack can add themselves as a provider with any special customisations required. For example, if it was as easy as copy the providers/openstack directory to new-provider directory and add a YAML file in to that directory that customises all the options for the Openstack Provider making the command line just a case of selecting that cloud provider and the defaults are all set correctly.

@meteorfox

This comment has been minimized.

Copy link
Collaborator

@meteorfox meteorfox commented Mar 1, 2016

@darrylweaver Here's what I'm envisioning to address your issue.

Make the flags --openstack_private_network and --openstack_public_network take a list of pre-created networks names. By default, it will try to attach and get a fixed IP from each network given. Both can be set but, at least, one of them has to be set. A public network could be any network that PKB client can reach. A private network could be a network that PKB might not necessarily reach but the instances it built can communicate through.

The PKB client will try to use an IP address from a network, if available, as a "public IP" in this order:

  1. Floating IP
  2. Public Network Fixed IP
  3. Private Network Fixed IP

Then introduce a new OPTIONAL flag, say something like --openstack_floating_ip_pool=<pool_name>, that when set it takes the pool name of an existing floating IP pool then it creates, if it needs to, and associates a floating IP from the pool an instance. When the flag is not set then it won't try to allocate the floating IPs at all, and default to next available IP address from either public network(s) or the private network(s).

Would that work in your case?

@darrylweaver

This comment has been minimized.

Copy link
Author

@darrylweaver darrylweaver commented Mar 1, 2016

Yes, that should address my issue.
Most, if not all, environments would be covered by that ordering, certainly the ones I am looking at.
Some private clouds and restricted public clouds would have to run PKB client inside the cloud anyway, so you should be able to have a private network only, so the last fall back should work for that case.

Not sure about making the --openstack_floating_ip_pool an optional flag as opposed to disabling floating IP allocation. The default common use case is that you would want an instance started on the only available network with a floating IP allocated from any available pool. In that use case it would be nicer if you didn't have to specify any options, but only if there are more complex requirements, i.e. more than 1 network to choose from and more than 1 floating IP pool.

@meteorfox

This comment has been minimized.

Copy link
Collaborator

@meteorfox meteorfox commented Mar 2, 2016

@darrylweaver I have the PR almost ready here master...meteorfox:openstack-opt-floating-ips

After experimenting with the flags against my OpenStack deployment, I can see what you meant. Adding the flag --openstack_floating_ip_pool doesn't really add much, since an external network would still need an internal network fixed IP address to associate the floating IP (at least for most private cloud deployments).

Also after thinking about it, having the flags take a list of networks just make things a lot more complex and can cause confusion to the user since most benchmarks rely on just 2 networks, internal and external. Probably, in the future, I'll have PKB create isolated networks and use those for some other benchmarks.

So taking your suggestion, and keeping things simple. I think what we could do is allocate a floating IP only when --openstack_public_network is set. In this case --openstack_private_network must always be set. Without it you cannot associate a floating IP, and if not using floating IP, you still need a network for PKB to talk through.

The commands would look something like this:

Note: PKB External IP is the IP address PKB uses to SSH into the instance and is equivalent to a Public IP address in your typical public cloud provider.

Case A:

  • External Network (w/ Floating IP)
  • Internal network
  • Router with ports to External and Internal Networks (NAT)
  • PKB client reaches instances through External Network
./pkb.py --cloud=OpenStack --machine_type=m1.small --image="Ubuntu 14.04" \
         --openstack_public_network=external --openstack_private_network=internal \
         --benchmarks=iperf

PKB External IP = floating IP
PKB Internal IP = IP from Internal network

Case B:

  • Internal Network (No Floating IP associated)
  • No Isolated Network
  • Router with ports to both Internal and External Network (NAT)
  • PKB client within Internal Network
./pkb.py --cloud=OpenStack --machine_type=m1.small --image="Ubuntu 14.04" \
         --openstack_private_network=internal \
         --benchmarks=iperf

PKB External IP = IP from Internal Network
PKB Internal IP = IP from Internal Network

One case this wouldn't cover, besides multiple isolated networks, is an external public network that does not use a floating IP. I don't think is a common case, though. To cover it, we could always use the flag you suggested for disabling the floating IP allocation, and then treat public_network like just another network except we will use it's IP for PKB's external IP.

@darrylweaver

This comment has been minimized.

Copy link
Author

@darrylweaver darrylweaver commented Mar 2, 2016

@meteorfox OK, Case A and B make sense to me.

One case this wouldn't cover, besides multiple isolated networks, is an external public network that does not use a floating IP.

Well, I guess it depends on how you define an openstack external network. Generally the only distinction in Openstack is that there are networks and external networks that floating IPs are allocated from. If we consider all networks that can attach an instance as an "internal" network and any "external" network to be a pool of floating IPs, then there is no such thing as an external network which does not use floating IPs and instances are always started on an "internal" network.
In that case a network in Openstack that has a route-able IP address without needing a floating IP would be considered an "internal" network and that use case is already addressed by case B.

This configuration would match the OVH Openstack deployment, i.e.:
Case C:

  • Single Flat style Network (No floating IP associated) of publicly routed IP addresses
  • No isolated Network
  • No neutron routers, single network, physical routers are outside of Openstack itself.
  • PKB client either internal or external to network.
./pkb.py --cloud=OpenStack --machine_type=m1.small --image="Ubuntu 14.04" \
         --openstack_private_network=internal \
         --benchmarks=iperf

PKB External IP = The fixed IP of the instance from the "internal" network
PKB Internal IP = PKB External IP (as above) same IP

Which I think works out exactly the same as Case B and is hence already addressed.

So, there may be no need to specifically flag disabling of floating IPs if we are clear what is an internal and external network? Maybe renaming of the flags would be appropriate?

Would it be clearer to have:
openstack_network = Any network to launch instances that obtain either an internal or external IP address, whichever is automatically allocated as fixed-IP. Initially one network, but could be expanded to include multiple networks later.
openstack_floating_ip_network = if defined, floating IPs are used from this network.

That could make it less confusing about the distinction between "internal" and "external" networks, which could be confusing if it is not referring to publicly routed or RFC1918 addresses, which is what most users would assume.
Of course, how this relates to PKB concepts of benchmarks running over "internal" and "external" networks should also be considered and if it directly maps to how tests are run, then that may be the overriding consideration.

@voellm

This comment has been minimized.

Copy link
Collaborator

@voellm voellm commented Mar 3, 2016

@darrylweaver - Do you want to take a pass at putting a pull request together? Just looking for the next step here based on the discussion with @meteorfox.

@meteorfox

This comment has been minimized.

Copy link
Collaborator

@meteorfox meteorfox commented Mar 8, 2016

@darrylweaver This PR #861 implements your suggestions.

@darrylweaver

This comment has been minimized.

Copy link
Author

@darrylweaver darrylweaver commented Mar 9, 2016

Yes, that seems to address the issue, thanks.

@meteorfox

This comment has been minimized.

Copy link
Collaborator

@meteorfox meteorfox commented Mar 21, 2016

Closing since #861 has been merged and PR has been confirmed to address this issue.

Feel free to re-open if the issue is still present.

@meteorfox meteorfox closed this Mar 21, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.