router's gateway to the external network / the status of the external port are down

[rubber-ant]

Ext_net :
http://img827.imageshack.us/img827/6711/screenshotfrom201305141.png

quantum net-show ext_net : http://paste.ubuntu.com/5663984/

quantum subnet-show id-sub-ext : http://paste.ubuntu.com/5663991/

quantum port-show id-port : http://paste.ubuntu.com/5664006/
after this command : "quantum router-gateway-set $put_router_proj_one_id_here $put_id_of_ext_net_here"
notice - the tenant_id is EMPTY ! this is seems a bug ?

Internal net working fine all the port are ACTIVE and ping each other.

from /var/log/quantum/ all fine instead :
/var/log/quantum/openvswitch-agent.log in network node :
ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'add-port', 'br-tun', 'gre-2']. Exception:
Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'add-port', 'br-tun', 'gre-2']
Exit code: 1
Stdout: ''
Stderr: 'ovs-vsctl: cannot create a port named gre-2 because a port named gre-2 already exists on bridge br-tun\n'
2013-05-14 10:34:37 ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'add-port', 'br-tun', 'gre-2']. Exception:
Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'add-port', 'br-tun', 'gre-2']
Exit code: 1
Stdout: ''
Stderr: 'ovs-vsctl: cannot create a port named gre-2 because a port named gre-2 already exists on bridge br-tun\n'

and in compute node is :
ERROR [quantum.agent.linux.ovs_lib] Unable to execute ['ovs-vsctl', '--timeout=2', 'add-port', 'br-tun', 'gre-1']. Exception:
Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'add-port', 'br-tun', 'gre-1']
Exit code: 1
Stdout: ''
Stderr: 'ovs-vsctl: cannot create a port named gre-1 because a port named gre-1 already exists on bridge br-tun\n'

and controller node seems fine :
/var/log/quantum/server.log
2013-05-14 10:33:54 WARNING [quantum.api.extensions] Extension port-security not supported by any of loaded plugins
2013-05-14 10:33:54 WARNING [quantum.api.extensions] Extension service-type not supported by any of loaded plugins
2013-05-14 10:33:54 WARNING [quantum.api.extensions] Extension lbaas not supported by any of loaded plugins
2013-05-14 10:33:54 WARNING [quantum.api.extensions] Extension routed-service-insertion not supported by any of loaded plugins
2013-05-14 10:33:54 WARNING [quantum.api.extensions] Extension flavor not supported by any of loaded plugins
2013-05-14 10:33:54 WARNING [quantum.api.extensions] Extension router-service-type not supported by any of loaded plugins
2013-05-14 10:33:54 WARNING [quantum.api.extensions] Extension security-group not supported by any of loaded plugins

any one have idea how turn on or why the status are not ACTIVE the ports in ext-net ?

thanks a lot

[sgstreet]

I'm seeing the same thing. There is a discussion with no resolution at https://answers.launchpad.net/quantum/+question/223204. Did you make any progress? A hard reboot of the instance will seems to work around the issue, but that is not really a solution.

[hrushig]

Irrespective of this status, I have seen basic network connectivity working through external network or on floating IP.

From: Stephen Street [mailto:notifications@github.com]
Sent: Tuesday, May 14, 2013 9:54 AM
To: mseknibilel/OpenStack-Grizzly-Install-Guide
Subject: Re: [OpenStack-Grizzly-Install-Guide] router's gateway to the external network / the status of the external port are down (#46)

I'm seeing the same thing. There is a discussion with no resolution at https://answers.launchpad.net/quantum/+question/223204. Did you make any progress? A hard reboot of the instance will seems to work around the issue, but that is not really a solution.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/46#issuecomment-17889421.

[bilelmsekni]

Delete your bridges and re-create them ! that will fix your problem.

[rubber-ant]

hi mseknibile ,
do you think is possible because after a restart of the network node or compute node the bridges are down ?

[cgirda]

mseknibilel,
Yes I have external network "Floating-IP" interfaces are showing DOWN. If I understand correct, I should remove the br-ex on the controller and re-create it, correct?

Thank you
Chakri

[rubber-ant]

the ports in the gui "Network Detail: ext_net" still down , but after bring up the bridge and the br-ex in the /etc/network/interface, I can see all my vm from external

[vnetworks]

Hi mseknibilel,

I have deleted all the bridges (br-ex, br-int and br-tun) and recreated them as you suggested and it's still not working as seen below:

 Fixed IPs          Attached Device           Status    Admin State     
192.168.3.102   network:router_gateway    DOWN  UP  
192.168.3.104   network:floatingip        DOWN  UP

After recreating the bridges, do I need to restart any services?

Thanks.

[rubber-ant]

yes now I have the same of you thanks a lot

[vnetworks]

Hi claenjoy,

Are you saying that it's working for you? I have done all that was suggested and restarted all services but no success. Am I the only one having this issue?

[michaelgrifalconi]

i have the same problem, tried to reboot, to re-create bridges but the problem stills there..and errors are the same as the first post with 12.04 and 13.04

[worldofmanish]

openstack dashboard "Network Detail: ext_net" still down , i deleted bridge couple of times but its not working

[srimahesht]

Had a similar issues.

Deleted and re created the bridges on the network node and it worked for us.

ovs-vsctl list-br

ovs-vsctl del-br br-ex
ovs-vsctl del-br br-int
ovs-vsctl del-br br-tun

ovs-vsctl add-br br-int
ovs-vsctl add-br br-ex
ovs-vsctl add-port br-ex eth<?>

ovs-vsctl list-br

cd /etc/init.d/; for i in $( ls quantum-* ); do sudo service $i restart; done

ovs-vsctl list-br

[neumerance]

I have the same issue I tried to make an installation base on mseknibiel's Grizzly Installation with OVSPlugin.
https://ask.openstack.org/en/question/3749/openstack-external-network-ports-is-down/

And then I tried also to install Grizzly using LinuxBridge and I got this error as well.
root@iaas01:/etc/init.d# tail -f /var/log/quantum/*.log
==> /var/log/quantum/dhcp-agent.log <==

==> /var/log/quantum/l3-agent.log <==
2013-08-15 18:52:43 ERROR [quantum.agent.l3_agent] The external network bridge 'br-ex' does not exist

==> /var/log/quantum/linuxbridge-agent.log <==

==> /var/log/quantum/metadata-agent.log <==

==> /var/log/quantum/server.log <==
2013-08-15 18:52:39 WARNING [quantum.api.extensions] Extension routed-service-insertion not supported by any of loaded plugins
2013-08-15 18:52:39 WARNING [quantum.api.extensions] Extension flavor not supported by any of loaded plugins
2013-08-15 18:52:39 WARNING [quantum.api.extensions] Extension service-type not supported by any of loaded plugins
2013-08-15 18:52:39 WARNING [quantum.api.extensions] Extension security-group not supported by any of loaded plugins
2013-08-15 18:52:39 WARNING [quantum.api.extensions] Extension port-security not supported by any of loaded plugins
2013-08-15 18:52:39 WARNING [quantum.api.extensions] Extension lbaas not supported by any of loaded plugins
2013-08-15 18:52:39 WARNING [quantum.api.extensions] Extension router-service-type not supported by any of loaded plugins

[wrasm]

Like many of the above I am unable to get in from the outside (so to speak). My efforts are based on:
https://github.com/mseknibilel/OpenStack-Grizzly-Install-Guide/blob/OVS_MultiNode/OpenStack_Grizzly_Install_Guide.rst

What is even more of a cause for head scratching is:

Pinging 192.168.100.72 with 32 bytes of data:
Reply from 192.168.100.72: Destination host unreachable.
Reply from 192.168.100.72: Destination host unreachable.
Reply from 192.168.100.72: Destination host unreachable.
Reply from 192.168.100.72: Destination host unreachable.

where 192.168.100.72 is the FloatingIP associated with the VM. Notice, on face value, the host is telling me its unreachable! Which I am taking to mean that I got to the start of the tunnel only to find nothing (aka no light) at the other end (DNAT not functioning as expected?)

I am seeing (in the /var/log/quantum/openvswitch-agent.log on the Compute Node ).
Stderr: 'ovs-vsctl: cannot create a port named gre-1 because a port named gre-1 already exists on bridge br-tun\n'

I have tried, tearing down and rebuilding the Virtual network, adding load balancing support, deleting bridges and recreating (both on Compute and Network nodes). All still leave me with External Network as "DOWN".

Has anyone else found solutions other than those proposed above?

[worldofmanish]

Please Use ip of your Host machine as gateway when you define external
network

quantum subnet-create --tenant-id admin_tenant_id --allocation-pool
start=x.x.x.x,end=x.x.x.x --gateway IP_of_your_hostmachine ext_net
x.x.x.x/24 --enable_dhcp=false

On Tue, Aug 27, 2013 at 4:21 PM, wrasm notifications@github.com wrote:

Like many of the above I am unable to get in from the outside (so to
speak). My efforts are based on:

https://github.com/mseknibilel/OpenStack-Grizzly-Install-Guide/blob/OVS_MultiNode/OpenStack_Grizzly_Install_Guide.rst

What is even more of a cause for head scratching is:

Pinging 192.168.100.72 with 32 bytes of data:
Reply from 192.168.100.72: Destination host unreachable.
Reply from 192.168.100.72: Destination host unreachable.
Reply from 192.168.100.72: Destination host unreachable.
Reply from 192.168.100.72: Destination host unreachable.

where 192.168.100.72 is the FloatingIP associated with the VM. Notice, on
face value, the host is telling me its unreachable! Which I am taking to
mean that I got to the start of the tunnel only to find nothing (aka no
light) at the other end (DNAT not functioning as expected?)

I am seeing (in the /var/log/quantum/openvswitch-agent.log on the Compute
Node ).
Stderr: 'ovs-vsctl: cannot create a port named gre-1 because a port named
gre-1 already exists on bridge br-tun\n'

I have tried, tearing down and rebuilding the Virtual network, adding load
balancing support, deleting bridges and recreating (both on Compute and
Network nodes). All still leave me with External Network as "DOWN".

Has anyone else found solutions other than those proposed above?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/46#issuecomment-23323004
.

Warm Regards,

Manish Kumar Abhishek
Project Engineer C-DAC(PUNE)
Mob +91-7350661889
Mail- manish@worldofmanish.co.in,amanish@cdac.in
www.worldofmanish.co.in

Let us all plant a tree, else, print sensibly.

[hrushig]

Ext NIC settings – no ip on the logical and might like to assign ip on br-ex. Typically, looks like:

VM Internet Access

auto eth2

iface eth2 inet manual

up ifconfig $IFACE 0.0.0.0 up

up ip link set $IFACE promisc on

down ip link set $IFACE promisc off

down ifconfig $IFACE down

auto br-ex

iface br-ex inet static

address 10.1.56.12

netmask 255.255.255.0

All bridges on all the nodes (br-ex, br-int, br-tun) needs to up and running (must add in interfaces to ensure it is UP on reboot)

If using vmware as virtualized environment, promiscuous setting to Accept on the vSwitch. Or, must be set on hardware switch.

Regards~Hrushi

From: wrasm [mailto:notifications@github.com]
Sent: Tuesday, August 27, 2013 4:35 AM
To: mseknibilel/OpenStack-Grizzly-Install-Guide
Cc: Gangur, Hrushikesh (HP Converged Cloud - R&D - Sunnyvale)
Subject: Re: [OpenStack-Grizzly-Install-Guide] router's gateway to the external network / the status of the external port are down (#46)

Thanks worldofmanish, however, no joy. From within the VM I can ping external machines, appropriately run NSLOOKUP but currently still cannot ping reliably from outside - I have, on occasions, received the expected response but this does not last long and I have never had an RDP connection nor a VNC (tunnelled through SSH) . Router status shows External Gateway DOWN.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/46#issuecomment-23329113.

[wrasm]

I now have a working environment. Red hearing on my part: needed to enable firewall passing of ICMP on Windows 2012 Server Evaluation (missed the fine print). Add TCP port 3389 to Security Group. And I believe, changing /etc/quantum/l3_agent.ini external_network_bridge=br-ex played a part - (had already assigned an IP to br-ex). I still have External Gateway "DOWN" status but, so far, working as "hoped".

[hrushig]

External Gateway DOWN seems misleading (ignore it). I guess there is an open defect targeted for Havana.

From: wrasm [mailto:notifications@github.com]
Sent: Wednesday, August 28, 2013 1:23 AM
To: mseknibilel/OpenStack-Grizzly-Install-Guide
Cc: Gangur, Hrushikesh (HP Converged Cloud - R&D - Sunnyvale)
Subject: Re: [OpenStack-Grizzly-Install-Guide] router's gateway to the external network / the status of the external port are down (#46)

I now have a working environment. Red hearing on my part: needed to enable firewall passing of ICMP on Windows 2012 Server Evaluation (missed the fine print). Add TCP port 3389 to Security Group. And I believe, changing /etc/quantum/l3_agent.ini external_network_bridge=br-ex played a part - (had already assigned an IP to br-ex). I still have External Gateway "DOWN" status but, so far, working as "hoped".

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/46#issuecomment-23398411.

[wrasm]

I decided to expand my network (Ubuntu 13.04 1xCtrl, 1xNet, 1xComp) configured based on https://github.com/mseknibilel/OpenStack-Grizzly-Install-Guide/blob/OVS_MultiNode/OpenStack_Grizzly_Install_Guide.rst by adding a second Compute node. The object was to test migration in particular within a mixed architecture (AMD for Comp1 and Intel for Comp2) environment. FYI single tenant setup at present; ufw status is inactive on all systems. The issue that arose was that the VM on the second system (Comp2 - Windows 7 Pro) could not be reliably accessed while the VM on Comp1 behaved as expected. The ping success rate (Comp2) was about 44%, bursts of <1ms intermixed with time outs. There did not appear to be a problem accessing via VNC console in Horizon. I used tcpdump to look at the flow which provided no indication other than ARP messages were not consistently being responded to (corresponding to timeouts).
I looked at the interfaces (VMs and hosts) and tunnels stats and found no significant errors/dropouts. I deleted the bridges (br-int, br-ex, br-tun as appropriate) on the network node and both the Compute nodes, shutdown all systems and then restarted; and still accessibility issues remained. What I did notice was for some reason both the network node and Comp1 had two gre tunnels open to Comp2 i.e. under br-tun there where ports gre-1 through gre-4 (albeit three specifically shown one implied); and both gre-3 and gre-4 had the remote_ip of Comp2.
I deleted the gre-3 port (gre-4 was implied on Comp2 hence the gre-3 choice) from all nodes and as of writing (running IE on Windows 7 Pro VM on Comp2 via external RDP sessions, pings to VMs from external network running) all appears well leading me to believe the symptom are due to iptables dropping packets resulting from the dual channel routing.
So my questions:

1. Is there a known issue with respect to superfluous tunnel connections leading to dropped packets due to iptables rules (OVSHybridIptablesFirewallDriver context)? Does Havana make (mult-ihost flag?) it all better?
2. Should I be adding additional routing information? Basically I simply built another Comp Server as per install guide replacing ip addresses as appropriate. FYI Comp1 differs also in that it is also a storage server (cinder configured).
3. If you shutdown all the nodes, does the order they are bought back up impact behaviour (excluding timeouts in logs)? I had a proclivity of Net->Ctrl->Comp but on reflection Ctrl->Net->Comp may be a better option (at least to reduce timeout messages in logs).

Regards and thanks