Network barclamp

See all, Barclamp List

This is a Creole document

Overview

The Network barclamp provides two functions for the system. The first is a common role to instantiate network interfaces on the crowbar managed systems. The other function is address pool management.

The network interfaces are controlled by the network role that is applied by the barclamp as a node transition to "installed". Based upon assigned addresses, the network recipe will create the appropriate single, dual, or team mode interface sets.

The network assignment function is handled by the creation of an API extension of the base barclamp. The barclamp adds the allocate_ip REST API call. This function allocates an IP address from a requested network and updates the node's attributes and the network's data bag. The available networks (and their parameters) are defined in the configuration for the barclamp.

This page also describes the format of the config file and parameters in the UI, but here is an Annotated Network JSON File.

Roles

The Network Barclamp provides a single role: network. This role provides the recipe access to instantiate the specified interfaces for the allocated IP addresses.

Scripts

The barclamp provides a single network cookbook. The cookbook is used for interface manipulation.

The standard CLI script has been extended to allow for the allocation of IP addresses.

Parameters

The barclamp has the following parameters:

Name	Default	Description
start_up_delay	30	A number in seconds that the chef recipe should wait to let the networks settle.
mode	single	A string value of either single, dual, or team. This specifies the default network interface construction model.
teaming	map	A map of values specific to teaming
interface_map	list	A list of map of values to help order network interfaces
conduit_map	map	A map of values to define conduits into the box
networks	map	A map of networks that this barclamp should manage

The mode specifies a base line goal of the system. Single means to use the first interface (usually eth0) as the only interface for all networks. VLANs and bridges will be added on top of this single interface. Dual means that the first interface should be used as the admin interface and the second interface should be used for the other interfaces. Team means that the first two interfaces should be bonded together and all VLANs run over it.

With the addition of conduit and interface maps, the mode can be custom generated to meet the environments needs. More on this below.

The teaming sub-parameters are:

Name	Default	Description
mode	6	The default teaming algorithm to use for the bonding driver in Linux

NOTE: Do not use mode 6 with Nova. Mode 6 and bridges do not work well.

The interface_map is a list of maps that have a common format. These maps help define the order of the interfaces and will build a list of interfaces where each interface is normalized to 1g1, 1g2, .... The naming format is 1Gb network 1. One day the system will enumerate 10Gb networks with 10g1, 10g2, and so on.

Each interface_map entry has the following parameters:

Name	Default	Description
pattern	None	The name of the dmidecode system product name name value. This can be a regex expression.
bus_order	list	A list of bus ids that can have network devices. These are strings of the form: "0000:00/0000:00:1c"

When the system needs to configure an interface, the interface_map is searched in list order to find a pattern that matches. If no pattern matches, the default enumeration is used. If a pattern patches, the devices are ordered according to the bus they are part of. This handles the case were the bios/system will enumerate devices differently based upon how many adapters are in the box. For most systems, the ordering will be:

1g1	eth0
1g2	eth1
1g3	eth2

This can change. For example, a Dell PEC2100 with two 2-port 1Gb nics will have this enumeration.

1g1	eth4	This is the first port on the motherboard
1g2	eth5	This is the second port on the motherboard
1g3	eth0	The first port on the top adapter (top left)
1g4	eth1
1g5	eth2
1g6	eth3	The last port on the bottom adapter (bottom right)

Once the interfaces are mapped to their normalized name, the conduit_map defines the paths the box should have configured. This is the step that allows for generating bonds of interfaces to create a single logical interface (or conduit) for a network to be configured on.

The conduit_map is a list of maps that define the logical interfaces to expose to the networks map. Each entry in list defines the conduits for the matching pattern.

Each entry has the following parts:

Name	Default	Description
pattern	None	A regex expression around the triple of mode, number of 1Gb networks, and node role
conduit_list	map	A containing logical interfaces and who to map them to the physical interfaces

The pattern looks like this: mode/number of 1Gb interfaces/role the node has Since this is a regex string, the following matches all nodes: .*/.*/.* The default configuration file contains basic mappings like described under the mode variable above. You can do all sorts of combinations.

Here are some examples:

Pattern	What it would match
.*/4/nova-multi-controller	Regardless of mode, all nodes with 4 1Gb interfaces and the nova-multi-controller role
team/2/.*	Regardless of the roles on the node, all nodes with 2 1Gb interfaces and the global mode is team
single/.*/.*	When in single mode, all nodes.

The pattern list is processed in order and the first match is applied.

The other field is the conduit_list. It is a map with the following fields.

Name	Default	Description
if_list	list	List of normalized interface names for this conduit.
team_mode	optional	The team mode value for this bond if it is more than one interface. This overrides the global team mode

The networks map contains names of networks associated with a map of network parameters. Each name is unique and can be used by other barclamps or users when allocating addresses. The network contains a reference to the conduit it should run on.

Each network has the following parameters:

Name	Default	Description
conduit	String	The conduit name from the conduit map to run this network on
vlan	Integer	The vlan to use on the switch and interfaces for this network
use_vlan	true	A value of true indicates that the vlan should applied to the interface. A value of false assumes that the node will receive untagged traffic for this network.
add_bridge	false	indicates if the network should have a bridge built on top of it. The bridge will be br<VLAN ID>. This is mostly for Nova compute.
subnet	IP Address	The subnet for this network
netmask	Netmask	The netmask for this network
router	IP Address	The default router for this network
broadcast	IP Address	The default broadcast address for this network
ranges	map	This contains a map of strings to start and stop values for network. This allows for sub-ranges with the network for specific uses. e.g. dhcp, admin, bmc, hosts.
mtu	integer	specifies an optional MTU to set on the interface backing this network (on 1g/10g nics). The expected use is to support Jumbo frames on networks where performance can be helped by this (NOTE:this might not work correctly with bonded interfaces!)

The range map has a string key that is the name and map defining the range.

Name	Type	Description
start	IP Address	First address in the range, inclusive
end	IP Address	Last address in the range, inclusive

The system provides the following default networks.

Name	Usage	Notes
admin	Private network for node to node communication	A router, if wanted, is external to the system. This network must be owned by the crowbar system to run DHCP on.
bmc	Private network for bmc communication	This can be the same as the admin network by using the ranges to limit what IP goes where. A router, if wanted, is external to the system.
bmc_vlan	Private network for admin nodes on the bmc network	This must be the same as the bmc network and have the same vlan. This will be used to generate a vlan tagged interface on the admin nodes that can access the bmc lan.
storage	Private network for storage traffic	A router, if wanted, is external to the system
public	Public network for crowbar and other components	A router, if wanted, is external to the system.
nova_fixed	Public network for nova Virtual Machines	The nova-network node acts as a router. This must be completely owned by the nova system.
nova_floating	Broken	deprecated - most likely to be replaced by nova config.

The defaults for the barclamp assume independent networks for all of them except the bmc and admin networks. They are assumed to be the same subnet.

Operations

At create proposal time, the barclamp will create and initialize the IP address allocation data bags for each network from the default config package. This is WRONG and needs to move to the new pre-chef call. DE10.

The barclamp has a transition function that listens for the "installed" state. The function makes sure that the node has the network role associated with it.

The barclamp also uses the pre-chef apply role call out to make sure that all the allocated addresses match the appropriate mode per the configuration. This will reconfigure the node's interfaces if the mode changes.

The API extension is as follows:

Name	URL	Method	Description
Allocate IP	/crowbar/network/<version>/allocate_ip/<barclamp-instance-name>	PUT	Putting a json document will allocate an IP address to the node
Enable Interface	/crowbar/network/<version>/enable_interface/<barclamp-instance-name>	PUT	Putting a json document will enable the interface for that network without an IP (unless one is already assigned).
Deallocate IP	/crowbar/network/<version>/deallocate_ip/<barclamp-instance-name>	PUT	Putting a json document will deallocate an IP address from the node
Disable Interface	/crowbar/network/<version>/disable_interface/<barclamp-instance-name>	PUT	Putting a json document will disable the interface for that network without an IP (unless one is already assigned).

The required JSON document for allocate_ip looks like this:

{
  "name": "FQDN.node.net",
  "network": "name of network",
  "range": "range inside of network to allocate from",
  "suggestion": "Optional parameter providing a suggestion for the address as a string" 
}

The required JSON document for enable_interface, disable_interface, and deallocate_ip looks like this:

{
  "name": "FQDN.node.net",
  "network": "name of network"
}

Useful Commands and Information

The chef-server can be used to look at the network data bags to see allocated addresses and their owners.

The network barclamp uses a file-based lock to prevent multiple writers to the network data bags. This would have to change if multiple framework servers where to server barclamp transition and allocation requests.

Limitations and/or Futures

Nova will not use this for address allocation. Nova assumes a range and it seems silly to have the same range managed by two components. This will cause some headaches for the FlatManager and FlatDHCPmanager cases, but should be covered by documentation. Nova now uses the network barclamp to "reserve" addresses from the DHCP pool section of a network for use by Nova.

The Network Barclamp does not handle IPv6.

Prior running the install script the network json needs to be edited

Editing the Network json