diff --git a/doc/src/docbkx/openstack-compute-admin/computenetworking.xml b/doc/src/docbkx/openstack-compute-admin/computenetworking.xml index 9e579ee2080..cb448a33803 100644 --- a/doc/src/docbkx/openstack-compute-admin/computenetworking.xml +++ b/doc/src/docbkx/openstack-compute-admin/computenetworking.xml @@ -12,46 +12,48 @@ Networking Options This section offers a brief overview of each concept in networking for Compute. - In Compute, users organize their cloud resources in - projects. A Compute project consists of a number of VM - instances created by a user. For each VM instance, Compute - assigns to it a private IP address. (Currently, Nova only - supports Linux bridge networking that allows the virtual - interfaces to connect to the outside network through the - physical interface.) + In Compute, users organize their cloud resources in projects. A Compute project + consists of a number of VM instances created by a user. For each VM instance, Compute + assigns to it a private IP address. (Currently, Compute only supports Linux bridge + networking that allows the virtual interfaces to connect to the outside network through + the physical interface.) The Network Controller provides virtual networks to enable compute servers to interact with each other and with the public network. - Currently, Nova supports three kinds of networks, - implemented in three “Network Manager” types respectively: - Flat Network Manager, Flat DHCP Network Manager, and VLAN - Network Manager. The three kinds of networks can co-exist - in a cloud system. However, since you can't yet select the - type of network for a given project, you cannot configure - more than one type of network in a given Compute - installation. - Nova has a concept of Fixed IPs and Floating IPs. Fixed - IPs are assigned to an instance on creation and stay the - same until the instance is explicitly terminated. Floating - IPs are IP addresses that can be dynamically associated - with an instance. This address can be disassociated and - associated with another instance at any time. A user can - reserve a floating IP for their project. - In Flat Mode, a network administrator specifies a - subnet. The IP addresses for VM instances are grabbed from - the subnet, and then injected into the image on launch. - Each instance receives a fixed IP address from the pool of - available addresses. A network administrator must - configure the Linux networking bridge (named br100) both - on the network controller hosting the network and on the - cloud controllers hosting the instances. All instances of - the system are attached to the same bridge, configured - manually by the network administrator. + Currently, Compute supports three kinds of networks, implemented in three “Network + Manager” types: + + Flat Network Manager + + + Flat DHCP Network Manager + + + VLAN Network Manager + + The three kinds of networks can co-exist in a cloud system. However, + since you can't yet select the type of network for a given project, you cannot configure + more than one type of network in a given Compute installation. + Compute makes a distinction between fixed IPs and + floating IPs for VM instances. Fixed IPs are IP + addresses that are assigned to an instance on creation and stay the same until the + instance is explicitly terminated. By contrast, floating IPs are addresses that can be + dynamically associated with an instance. A floating IP address can be disassociated and + associated with another instance at any time. A user can reserve a floating IP for their + project. + In Flat Mode, a network administrator specifies a subnet. The IP addresses for VM + instances are grabbed from the subnet, and then injected into the image on launch. Each + instance receives a fixed IP address from the pool of available addresses. A network + administrator must configure the Linux networking bridge (typically named + br100, although this configurable) both on the network controller + hosting the network and on the cloud controllers hosting the instances. All instances of + the system are attached to the same bridge, configured manually by the network + administrator. - The configuration injection currently only works - on Linux-style systems that keep networking - configuration in /etc/network/interfaces. + The configuration injection currently only works on Linux-style systems that + keep networking configuration in + /etc/network/interfaces. In Flat DHCP Mode, you start a DHCP server to pass out @@ -344,15 +346,12 @@ public_interface=eth0 xml:id="outbound-traffic-flow-with-any-flat-networking"> Outbound Traffic Flow with Any Flat Networking - In any set up with FlatNetworking, the host with - nova-network on it is responsible for forwarding - traffic from the private network configured with the - --fixed_range=... directive in - nova.conf. This host needs to have br100 configured - and talking to any other nodes that are hosting VMs. - With either of the Flat Networking options, the - default gateway for the virtual machines is set to the - host which is running nova-network. + In any set up with FlatNetworking, the host with nova-network on it is + responsible for forwarding traffic from the private network configured with the + fixed_range=... directive in nova.conf. + This host needs to have a bridge interface (e.g., br100) configured and talking to any other + nodes that are hosting VMs. With either of the Flat Networking options, the default + gateway for the virtual machines is set to the host which is running nova-network. When a virtual machine sends traffic out to the public networks, it sends it first to its default gateway, which is where nova-network is configured. @@ -390,189 +389,209 @@ public_interface=eth0
Configuring VLAN Networking - In some networking environments, you may have a - large IP space which is cut up into smaller subnets. - The smaller subnets are then trunked together at the - switch level (dividing layer 3 by layer 2) so that all - machines in the larger IP space can communicate. The - purpose of this is generally to control the size of - broadcast domains. - Using projects as a way to logically separate each - VLAN, we can setup our cloud in this environment. - Please note that you must have IP forwarding enabled - for this network mode to work. - - Obtain the parameters for each network. You may need - to ask a network administrator for this information, - including netmask, broadcast, gateway, ethernet device - and VLAN ID. - You need to have networking hardware that supports - VLAN tagging. - Please note that currently eth0 is hardcoded as the - vlan_interface in the default options. If you need to - attach your bridges to a device other than eth0, you - will need to add following option to - /etc/nova/nova.conf: - - -vlan_interface=eth1 - - In VLAN mode, the setting for --network_size is the - number of IPs per project as opposed to the FlatDHCP - mode where --network_size indicates number of IPs in - the entire fixed range. For VLAN, the settings in - nova.conf that affect networking are also - --fixed_range, where the space is divided up into - subnets of --network_size. - VLAN is the default networking mode for Compute, so - if you have no --network_manager entry in your - nova.conf file, you are set up for VLAN. To set your - nova.conf file to VLAN, use this option in - /etc/nova/nova.conf: - -network_manager=nova.network.manager.VlanManager - - For the purposes of this example walk-through, we - will use the following settings. These are - intentionally complex in an attempt to cover most - situations: - - - - - VLANs: 171, 172, 173 and 174 - - - IP Blocks: 10.1.171.0/24, 10.1.172.0/24, - 10.1.173.0/24 and 10.1.174.0/24 - - - Each VLAN maps to its corresponding /24 (171 - = 10.1.171.0/24, etc) - - - Each VLAN will get its own bridge device, - which is in the format br_$VLANID - - - Each /24 has an upstream default gateway on - .1 - + Compute can be configured so that the virtual machine instances of different + projects (tenants) are in different subnets, with each subnet having a different + VLAN tag. This can be useful in networking environments where you have a large IP + space which is cut up into smaller subnets. The smaller subnets are then trunked + together at the switch level (dividing layer 3 by layer 2) so that all machines in + the larger IP space can communicate. The purpose of this is generally to control the + size of broadcast domains. It can also be useful to provide an additional layer of + isolation in a multi-tenant environment. + + The terms network and subnet are often used interchangeably in + discussions of VLAN mode. In all cases, we are referring to a range of IP + addresses specified by a subnet (e.g., + 172.16.20.0/24) that are on the same VLAN (layer 2 + network). + + Running in VLAN mode is more complex than the other network modes. In particular: + + IP forwarding must be enabled + - The first 6 IPs in each /24 are - reserved + The hosts running nova-network and nova-compute must have the + 8021q kernel module loaded - - First, create the networks that Compute can pull - from using nova-manage commands: - -$ nova-manage --flagfile=/etc/nova/nova.conf network create private 10.1.171.0/24 1 256 -$ nova-manage --flagfile=/etc/nova/nova.conf network create private 10.1.172.0/24 1 256 -$ nova-manage --flagfile=/etc/nova/nova.conf network create private 10.1.173.0/24 1 256 -$ nova-manage --flagfile=/etc/nova/nova.conf network create private 10.1.174.0/24 1 256 - - - Log in to the nova database to determine the network - ID assigned to each VLAN: - -$select id,cidr from networks; - - Update the DB to match your network settings. The - following script will generate SQL based on the - predetermined settings for this example. You - will need to modify this database update to fit - your environment. - -if [ -z $1 ]; then - echo "You need to specify the vlan to modify" -fi - -if [ -z $2 ]; then - echo "You need to specify a network id number (check the DB for the network you want to update)" -fi + + Your networking switches must support VLAN tagging + + + Your networking switches must be configured to enable the specific + VLAN tags you specify in your Compute setup + + + You will need information about your networking setup from your + network administrator to configure Compute properly (e.g., netmask, + broadcast, gateway, ethernet device, VLAN IDs) + + + To configure your nodes to support VLAN tagging, install the + vlan package and load the 8021q kernel + module, as + root:# apt-get install vlan +# modprobe 8021q + To have this kernel module loaded on boot, add the following line to + /etc/modules:8021q + Here is an example of settings from /etc/nova/nova.conf for a + host configured to run nova-network in VLAN mode -VLAN=$1 -ID=$2 + network_manager=nova.network.manager.VlanManager +vlan_interface=eth0 +fixed_range=172.16.0.0/12 +network_size=256 +The network_manager=nova.network.manager.VlanManager option specifies VLAN + mode, which happens to be the default networking mode. + The bridges that are created by the network manager will be attached to the + interface specified by vlan_interface, the example above uses the + eth0 interface, which is the default. + The fixed_range option is a CIDR block which describes the IP + address space for all of the instances: this space will be divided up into subnets. + This range is typically a private + network. The example above uses the private range + 172.16.0.0/12. + The network_size option refers to the default number of IP + addresses in each network, although this can be overriden at network creation time . + The example above uses a network size of 256, whicih corresponds + to a /24 network. + Networks are created with the nova-manage network create + command. Here is an example of how to create a network consistent with the above + example configuration options, as + root:# nova-manage network create --label=example-net --fixed_range_v4=172.16.169.0/24 --vlan=169 --bridge=br169 --project_id=a421ae28356b4cc3a25e1429a0b02e98 + This creates a network called example-net associated with + tenant a421ae28356b4cc3a25e1429a0b02e98. The subnet is + 172.16.169.0/24 with a VLAN tag of 169 + (the VLAN tag does not need to match the third byte of the address, though it is a + useful convention to remember the association). This will create a bridge interface + device called br169 on the host running the nova-network service. + This device will appear in the output of an ifconfig + command. + Each network must be associated with one tenant, specified using the + --project_id flag which corresponds to the tenant ID. Use the + keystone tenant-list command to list the tenants and + corresponding IDs that you have already created. + The nova-manage network create command supports many + configuration options, which are displayed when called with the + --help flag: + Usage: nova-manage network create <args> [options] -cat > vlan.sql << __EOF_ -update networks set vlan = '$VLAN' where id = $ID; -update networks set bridge = 'br_$VLAN' where id = $ID; -update networks set gateway = '10.1.$VLAN.7' where id = $ID; -update networks set dhcp_start = '10.1.$VLAN.8' where id = $ID; -update fixed_ips set reserved = 1 where address in ('10.1.$VLAN.1','10.1.$VLAN.2','10.1.$VLAN.3','10.1.$VLAN.4','10.1.$VLAN.5','10.1.$VLAN.6','10.1.$VLAN.7'); -__EOF_ +Options: + -h, --help show this help message and exit + --label=<label> Label for network (ex: public) + --fixed_range_v4=<x.x.x.x/yy> + IPv4 subnet (ex: 10.0.0.0/8) + --num_networks=<number> + Number of networks to create + --network_size=<number> + Number of IPs per network + --vlan=<vlan id> vlan id + --vpn=VPN_START vpn start + --fixed_range_v6=FIXED_RANGE_V6 + IPv6 subnet (ex: fe80::/64 + --gateway=GATEWAY gateway + --gateway_v6=GATEWAY_V6 + ipv6 gateway + --bridge=<bridge> VIFs on this network are connected to this bridge + --bridge_interface=<bridge interface> + the bridge is connected to this interface + --multi_host=<'T'|'F'> + Multi host + --dns1=<DNS Address> First DNS + --dns2=<DNS Address> Second DNS + --uuid=<network uuid> + Network UUID + --fixed_cidr=<x.x.x.x/yy> + IPv4 subnet for fixed IPS (ex: 10.20.0.0/16) + --project_id=<project id> + Project id + --priority=<number> Network interface priority - After verifying that the above SQL will work for - your environment, run it against the nova database, - once for every VLAN you have in the - environment. + In particular, flags to the nova-mange network create command + can be used to override settings from nova.conf: + + --network_size + + Overrides the network_size configuration + option + + + + --bridge_interface + + Overrides the vlan_interface configuration + option + + + + To view a list of the networks that have been created, as + root:# nova-manage network list + To modify an existing network, use the nova-manage network + modify command, as + root:# nova-manage network modify --help +Usage: nova-manage network modify <args> [options] - Next, create a project manager for the Compute - project: +Options: + -h, --help show this help message and exit + --fixed_range=<x.x.x.x/yy> + Network to modify + --project=<project name> + Project name to associate + --host=<host> Host to associate + --disassociate-project + Disassociate Network from Project + --disassociate-host Disassociate Host from Project + To delete a network, use nova-manage network delete, as + root:# nova-manage network delete --help +Usage: nova-manage network delete <args> [options] - -$ nova-manage --flagfile=/etc/nova/nova.conf user admin $username - - - Then create a project and assign that user as the - admin user: - - -$ nova-manage --flagfile=/etc/nova/nova.conf project create $projectname $username - - - Finally, get the credentials for the user just - created, which also assigns one of the networks to - this project:) - - -$ nova-manage --flagfile=/etc/nova/nova.conf project zipfile $projectname $username - - - When you start nova-network, the bridge devices and - associated VLAN tags will be created. When you create - a new VM you must determine (either manually or - programatically) which VLAN it should be a part of, - and start the VM in the corresponding project. +Options: + -h, --help show this help message and exit + --fixed_range=<x.x.x.x/yy> + Network to delete + --uuid=<uuid> UUID of network to delete + Note that a network must first be disassociated from a project using the + nova-manage network modify command before it can be + deleted. + Creating a network will automatically cause the Compute database to populate with + a list of available fixed IP addresses. You can view the list of fixed IP addresses + and their associations with active virtual machines by doing, as + root:# nova-manage fix list + + + Due to Compute bug #754900, deleting a network with the + nova-manage network delete command does not delete + the associated fixed IP addresses. As a workaround, these fixed IP addresses + can be deleted by connecting to the nova database and + issuing the SQL query (this example assumes the deleted network id is + 1DELETE from fixed_ips where network_id=1; + + - In certain cases, the network manager may not - properly tear down bridges and VLANs when it is - stopped. If you attempt to restart the network manager - and it does not start, check the logs for errors - indicating that a bridge device already exists. If - this is the case, you will likely need to tear down - the bridge and VLAN devices manually. + In certain cases, the network manager may not properly tear down bridges and VLANs + when it is stopped. If you attempt to restart the network manager and it does not + start, check the logs for errors indicating that a bridge device already exists. If + this is the case, you will likely need to tear down the bridge and VLAN devices + manually. It is also advisable to kill any remaining dnsmasq processes. These + commands would stop the service, manually tear down the bridge and VLAN from the + previous example, kill any remaining dnsmasq processes, and start the service up + again, as root: - -$ vconfig rem vlanNNN -$ ifconfig br_NNN down -$ brctl delbr br_NNN - + # stop nova-network +# vconfig rem vlan169 +# ip link set br169 down +# brctl delbr br169 +# killall dnsmasq +# start nova-network - Also, if users need to access the instances in their - project across a VPN, a special VPN instance (code - named cloudpipe) needs to be created as described. You - can create the cloudpipe instance. The image is - basically just a Linux instance with openvpn - installed. It needs a simple script to grab user data - from the metadata server, b64 decode it into a zip - file, and run the autorun.sh script from inside the - zip. The autorun script should configure and run - openvpn to run using the data from Compute. - For certificate management, it is also useful to - have a cron script that will periodically download the - metadata and copy the new Certificate Revocation List - (CRL). This will keep revoked users from connecting - and disconnects any users that are connected with - revoked certificates when their connection is - re-negotiated (every hour). You set the - use_project_ca option in - nova.conf for cloudpipes to - work securely so that each project has its own - Certificate Authority (CA). + If users need to access the instances in their project across a VPN, a special VPN + instance (code named cloudpipe) needs to be created as described in the section + titled Cloudpipe — Per Project + VPNs.
- Cloudpipe — Per Project Vpns + Cloudpipe — Per Project VPNs Cloudpipe is a method for connecting end users to their project instances in VLAN networking mode. @@ -600,7 +619,7 @@ __EOF_
Creating a Cloudpipe Image - Making a cloudpipe image is relatively easy. + To make a cloudpipe image: install openvpn on a base ubuntu image. @@ -706,16 +725,19 @@ force_dhcp_release=True
Certificates and Revocation - If the use_project_ca config - option is set (required to for cloudpipes to work - securely), then each project has its own ca. This - ca is used to sign the certificate for the vpn, - and is also passed to the user for bundling - images. When a certificate is revoked using - nova-manage, a new Certificate Revocation List - (crl) is generated. As long as cloudpipe has an - updated crl, it will block revoked users from - connecting to the vpn. + For certificate management, it is also useful to have a cron script that will + periodically download the metadata and copy the new Certificate Revocation List + (CRL). This will keep revoked users from connecting and disconnects any users + that are connected with revoked certificates when their connection is + re-negotiated (every hour). You set the use_project_ca option in nova.conf for + cloudpipes to work securely so that each project has its own Certificate + Authority (CA). + If the use_project_ca config option is set (required to for + cloudpipes to work securely), then each project has its own CA. This CA is used + to sign the certificate for the vpn, and is also passed to the user for bundling + images. When a certificate is revoked using nova-manage, a new Certificate + Revocation List (crl) is generated. As long as cloudpipe has an updated crl, it + will block revoked users from connecting to the vpn. The userdata for cloudpipe isn't currently updated when certs are revoked, so it is necessary to restart the cloudpipe instance if a user's