Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

CA-99179: Add other_config param for disabling default gateway advertising on host-internal-management network #1025

Merged
merged 2 commits into from

5 participants

@rdobson
Collaborator

Patches XAPI to allow clients to stop XAPI from generating a udhcpd
config file that includes advertising dom0's IP as the default gateway
for the network.

When using the host internal management network with Windows VMs
the network adapter will be allocated a high metric based on its speed,
and so traffic to unkown destinations may be routed via Dom0.

@xen-git
Owner

Can one of the admins verify this patch?

@djs55
Owner

Hi Rob,

Could you describe your scenario in a bit more detail? I'm a bit worried that setting this key to make your scenario work, will possibly break someone else's concurrent use where they expect the default route to be present-- so we may end up with features that conflict.

Perhaps we could always remove the route? If we can't I'd like to understand why :-)

Also, I'm worried that other_config keys tend to get forgotten about and get accidentally removed over refactoring. Assuming we're going with this one, could you write a small description in the style of this one:

https://github.com/djs55/xcp-idl/blob/master/xen-api-plugin/service.md

(I put that one in the wrong directory -- I was going to put it into an "apis" directory). My aim is to create and maintain a set of 'mini-API' specs each focused on a separate area.

Thanks!

@rdobson
Collaborator

Hi Dave,

I agree that this may prevent users from using both the KVP work and other virtual appliances which make use of the host internal management network at the same time. Which is less than ideal.

The context to this problem is that we have a KVP HTTP server running inside a Windows VM, and we plug a VIF into that VM which connects the VM to the Host Internal Management Network.

This extra VIF, is as much as possible meant to be 'unnoticed' in that it should not impact their networking, but rather provide an interface that only our KVP server uses, and listens on.

Due to the fact that we are using DHCP off of the udhcpd server running in Dom0, we run into problems when a default gateway is returned. This is due to the fact that an entry is added to the Windows routing table which is subsequently used to route packets down this added interface - something which shouldn't happen, and breaks the VMs traffic flows to unknown destinations.

It would be possible to pass the responsibility of removing this route onto the in guest service, however I have several concerns about doing this:

  • There is no way to intercept the route being added - so there is likely to be a window of opportunity for traffic to be mis-routed.
  • The guest service is designed to be able to have the VIF plugged/destroyed at will by Dom0 - so the service would potentially have to repeatedly remove the route
  • Relying on the service running inside the guest does not seem ideal - if there were a bug in the service (e.g. crashed/stopped/failing to see route being added), then we would impact the guests network traffic.

For those reasons I thought it would be safer to fix this issue by altering the configuration of the udhcpd server.

I chose to add an extra other_config parameter due to the fact other configuration options used this approach.

I suppose an alternative approach, would be to create and handle a separate network (mimicking the Host Internal Management Network in having a DHCP service, and an IP in Dom0 etc) which would obviously be more work.

@djs55 - What do you think?

If you were happy with this approach, would you like the API spec for all of the udhcpd other_config params? And where would you like me to commit it? (would you like me to create an apis directory?)

Thanks.

@djs55
Owner
@robhoes
Owner

@rdobson As an alternative, would it be possible for the DHCP client in the Windows guest to not request a default gateway from the DHCP server. I am not sure how practical this is in Windows, but the DHCP client in Linux lets you specify exactly what settings you want back from the server.

I believe the default gateway on the HIMN was made to be dom0's IP address, so that the guest knows which IP address to talk to. Does this particular appliance use a different way of identifying dom0's IP?

@rdobson
Collaborator

@robhoes - You are certainly able to configure an adapter in Windows to have certain properties which I believe would allow you to effectively disable listening for a default gateway (or at least crank the metric so high it will never be preferred). However the problem with this is that we do not have control over the initial properties of the device - and I would like to avoid inadvertently causing any traffic to disappear down the wrong route. In order to do it properly, we would likely need some functionality in the PV drivers I would suspect.

As for detecting Dom0's IP - we actually don't need to do that in this case. The server running in the guest communicates its IP via XenStore, and it talked to from outside, and in the guest. So although the VM obviously learns Dom0s IP, it does not need to initiate communication with it.

@robhoes robhoes was assigned
@jonludlam
Owner

ok to test

@xen-git
Owner

Can one of the admins verify this patch?

@rdobson
Collaborator

@robhoes - I've now created an API file for configuring the udhcp server. Could you see if your happy with it please?

Thanks.

@xen-git
Owner

Can one of the admins verify this patch?

@rdobson
Collaborator

@robhoes - can you please take a look at this?

Thanks.

@robhoes
Owner

ok to test

ocaml/xapi/xapi_udhcpd.ml
@@ -114,14 +115,21 @@ module Udhcpd_conf = struct
let string_of_lease l =
Printf.sprintf "static_lease\t%s\t%s # %s\n" l.mac (Ip.string_of l.ip) l.vif in
let leases = List.map string_of_lease t.leases in
- String.concat "\n" (skel :: interface :: subnet :: router :: leases)
+ let network = Helpers.get_guest_installer_network ~__context in
+ let other_config = Db.Network.get_other_config ~__context ~self:network in
+ let config_list =
+ if (List.mem_assoc ip_disable_gw_key other_config)
@robhoes Owner
robhoes added a note

Here you check only for the presence of the "ip_disable_gw" key, and not for whether it is set to "true". This means that if you set the key to "false", the gateway will also be disabled.

We normally use the following pattern:

if List.mem_assoc ip_disable_gw_key other_config && List.assoc ip_disable_gw_key other_config = "true" then .....
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@rdobson
Collaborator

@robhoes - Thanks - I've updated the patch to include the check for its value. Thanks for spotting that!

Are you happy to merge now?

ocaml/xapi/xapi_udhcpd.ml
@@ -114,14 +115,21 @@ module Udhcpd_conf = struct
let string_of_lease l =
Printf.sprintf "static_lease\t%s\t%s # %s\n" l.mac (Ip.string_of l.ip) l.vif in
let leases = List.map string_of_lease t.leases in
- String.concat "\n" (skel :: interface :: subnet :: router :: leases)
+ let network = Helpers.get_guest_installer_network ~__context in
+ let other_config = Db.Network.get_other_config ~__context ~self:network in
+ let config_list =
+ if (List.mem_assoc ip_disable_gw_key other_config && List.assoc ip_disable_gw_key other_config == "true")
@robhoes Owner
robhoes added a note

I'm afraid you need a '=' here, not a '==', which is something different.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@robhoes
Owner

Also, could you add "Signed-off-by" lines to your patches.

Apart from that, I think it all looks good and ready to merge. Thanks for taking the effort to write the docs!

rdobson added some commits
@rdobson rdobson CA-99179: Patches XAPI to allow clients to stop XAPI from generating
a udhcpd config file that includes advertising dom0's IP as
the default gateway for the network.

When using the host internal management network with Windows VMs
the network adapter will be allocated a high metric based on its speed,
and so traffic to unkown destinations may be routed via Dom0.

Signed-off-by: Rob Dobson <rob@rdobson.co.uk>
6522e6e
@rdobson rdobson CA-99179: Adding an API for configuring the udhcp server in Dom0.
Signed-off-by: Rob Dobson <rob@rdobson.co.uk>
f0de11c
@rdobson
Collaborator

Great thanks! I believe I have now made those required modifications.

Thanks.

@robhoes robhoes merged commit 399de85 into from
@robhoes
Owner

Cool, thanks! It is now merged. I'll duplicate your pull request for clearwater as well.

@rdobson rdobson deleted the branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Apr 15, 2013
  1. @rdobson

    CA-99179: Patches XAPI to allow clients to stop XAPI from generating

    rdobson authored rdobson committed
    a udhcpd config file that includes advertising dom0's IP as
    the default gateway for the network.
    
    When using the host internal management network with Windows VMs
    the network adapter will be allocated a high metric based on its speed,
    and so traffic to unkown destinations may be routed via Dom0.
    
    Signed-off-by: Rob Dobson <rob@rdobson.co.uk>
  2. @rdobson

    CA-99179: Adding an API for configuring the udhcp server in Dom0.

    rdobson authored
    Signed-off-by: Rob Dobson <rob@rdobson.co.uk>
This page is out of date. Refresh to see the latest.
Showing with 104 additions and 3 deletions.
  1. +93 −0 apis/udhcp.md
  2. +11 −3 ocaml/xapi/xapi_udhcpd.ml
View
93 apis/udhcp.md
@@ -0,0 +1,93 @@
+API for configuring the udhcp server in Dom0
+============================================
+
+Summary
+-------
+
+This API allows you to configure the DHCP service running on the Host
+Internal Management Network (HIMN). The API congigures a udhcp daemon
+residing in Dom0 and alters the service configuration for any VM using
+the network.
+
+It should be noted that for this reason, that callers who modify the
+default configuration should be aware that their changes may have an
+adverse affect on other consumers of the HIMN.
+
+Version history
+---------------
+
+ Date State
+ ---- ----
+ 2013-3-15 Stable
+
+_Stable_: this API is considered stable and unlikely to change between
+software version and between hotfixes.
+
+API description
+---------------
+
+The API for configuring the network is based on a series of other_config
+keys that can be set by the caller on the HIMN XAPI network object. Once
+any of the keys below have been set, the caller must ensure that any VIFs
+attached to the HIMN are removed, destroyed, created and plugged.
+
+ ip_begin
+
+The first IP address in the desired subnet that the caller wishes the
+DHCP service to use.
+
+ ip_end
+
+The last IP address in the desired subnet that the caller wishes the
+DHCP service to use.
+
+ netmask
+
+The subnet mask for each of the issues IP addresses.
+
+ ip_disable_gw
+
+A boolean key for disabling the DHCP server from returning a default
+gateway for VMs on the network. To disable returning the gateway address
+set the key to True.
+
+_Note_: By default, the DHCP server will issue a default gateway for
+those requesting an address. Setting this key may disrupt applications
+that require the default gateway for communicating with Dom0 and so
+so should be used with care.
+
+
+
+Example code
+------------
+
+An example python extract of setting the config for the network:
+
+ def get_himn_ref():
+ networks = session.xenapi.network.get_all_records()
+ for ref, rec in networks.iteritems():
+ if 'is_host_internal_management_network' \
+ in rec['other_config']:
+ return ref
+
+ raise Exception("Error: unable to find HIMN.")
+
+
+ himn_ref = get_himn_ref()
+ other_config = session.xenapi.network.get_other_config(himn_ref)
+
+ other_config['ip_begin'] = "169.254.0.1"
+ other_config['ip_end'] = "169.254.255.254"
+ other_config['netmask'] = "255.255.0.0"
+
+ session.xenapi.network.set_other_config(himn_ref, other_config)
+
+
+An example for how to disable the server returning a default gateway:
+
+ himn_ref = get_himn_ref()
+ other_config = session.xenapi.network.get_other_config(himn_ref)
+
+ other_config['ip_disable_gw'] = True
+
+ session.xenapi.network.set_other_config(himn_ref, other_config)
View
14 ocaml/xapi/xapi_udhcpd.ml
@@ -24,6 +24,7 @@ open Threadext
let ip_begin_key = "ip_begin"
let ip_end_key = "ip_end"
+let ip_disable_gw_key = "ip_disable_gw"
let udhcpd_conf = Filename.concat Fhs.vardir "udhcpd.conf"
let udhcpd_skel = Filename.concat Fhs.vardir "udhcpd.skel"
@@ -106,7 +107,7 @@ module Udhcpd_conf = struct
leases = leases
}
- let to_string t =
+ let to_string ~__context t =
let skel = Unixext.string_of_file udhcpd_skel in
let interface = Printf.sprintf "interface\t%s" t.interface in
let subnet = Printf.sprintf "option\tsubnet\t%s" t.subnet in
@@ -114,14 +115,21 @@ module Udhcpd_conf = struct
let string_of_lease l =
Printf.sprintf "static_lease\t%s\t%s # %s\n" l.mac (Ip.string_of l.ip) l.vif in
let leases = List.map string_of_lease t.leases in
- String.concat "\n" (skel :: interface :: subnet :: router :: leases)
+ let network = Helpers.get_guest_installer_network ~__context in
+ let other_config = Db.Network.get_other_config ~__context ~self:network in
+ let config_list =
+ if (List.mem_assoc ip_disable_gw_key other_config && List.assoc ip_disable_gw_key other_config = "true")
+ (*Check whether the default gateway should be defined*)
+ then (skel :: interface :: subnet :: leases)
+ else (skel :: interface :: subnet :: router :: leases) in
+ String.concat "\n" config_list
end
let write_config_nolock ~__context ip_router =
let config = Udhcpd_conf.make ~__context (!assigned) ip_router in
Unixext.unlink_safe udhcpd_conf;
- Unixext.write_string_to_file udhcpd_conf (Udhcpd_conf.to_string config)
+ Unixext.write_string_to_file udhcpd_conf (Udhcpd_conf.to_string ~__context config)
let command = Filename.concat Fhs.libexecdir "udhcpd"
Something went wrong with that request. Please try again.