Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

End-to-End MPLS Not Working On Docker Container #4872

Closed
anspectrum opened this issue Aug 23, 2019 · 5 comments
Closed

End-to-End MPLS Not Working On Docker Container #4872

anspectrum opened this issue Aug 23, 2019 · 5 comments
Labels

Comments

@anspectrum
Copy link

@anspectrum anspectrum commented Aug 23, 2019

Hello,
I've following setup:
Host machine is Ubuntu 16.04 LTS (Linux OS 4.15.0-58-generic #64~16.04.1-Ubuntu SMP Wed Aug 7 14:10:35 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux). Installed docker on it ( Version: 18.09.7). Also have GNS3 simulator (2.1.21).

Pulled FRR image from Docker (docker pull frrouting/frr)
MPLS modules (mpls_iptunnel mpls_router) are enabled on host machine. Also enabled following in /etc/sysctl.conf on host machine:

# Enable MPLS Label processing on all interfaces
net.mpls.platform_labels=100000
net.mpls.conf.lo.input=1
net.mpls.conf.enp0s25.input=1
net.mpls.conf.docker0.input = 1

A simple MPLS 3 x router topology

frr

Everything works fine like OSPF, ISIS, BGP. However, when I enable MPLS LDP on interconnecting interfaces, frr-1 can not ping frr-3 loopback and vide versa. I've done packer captures and it seems that frr-1 and frr-3 both send the correct MPLS labelled packets but frr-2 keeps dropping them. Don't know why its like that. Here is the configs:

hostname frr-1
!
!
interface eth0
 ip address 10.1.1.1/30
 ip ospf network point-to-point
!
interface lo
 ip address 1.1.1.1/32
!
!
 !
router ospf
 network 0.0.0.0/0 area 0
!
mpls ldp
 router-id 1.1.1.1
 !       
 address-family ipv4
  discovery transport-address 1.1.1.1
  !
  interface eth0
  !
  interface lo
  !
 exit-address-family
 !
!
line vty
 exec-timeout 0 0
!
end


hostname frr-2
!
interface eth0
 ip address 10.1.1.2/30
 ip ospf network point-to-point
!
interface eth1
 ip address 10.1.1.5/30
 ip ospf network point-to-point
!
interface lo
 ip address 1.1.1.2/32
!
router ospf
 network 0.0.0.0/0 area 0
!
mpls ldp
 router-id 1.1.1.2
 !       
 address-family ipv4
  discovery transport-address 1.1.1.2
  !
  interface eth1
  !
  interface eth0
  !
  interface lo
  !
 exit-address-family
 !
line vty
!
end


hostname frr-3
!
!
interface eth1
 ip address 10.1.1.6/30
 ip ospf network point-to-point
!
interface lo
 ip address 1.1.1.3/32
!
 !
!
router ospf
 network 0.0.0.0/0 area 0
!
mpls ldp
 router-id 1.1.1.3
 !       
 address-family ipv4
  discovery transport-address 1.1.1.3
  !
  interface eth1
  !
  interface lo
  !
 exit-address-family
 !
!
line vty
 exec-timeout 0 0
!
end

Please suggest where can I look to fix it.

@anspectrum anspectrum added the triage label Aug 23, 2019
@anspectrum

This comment has been minimized.

Copy link
Author

@anspectrum anspectrum commented Aug 28, 2019

I've also checked this issue on latest released Alpine Linux and the behavior remains the same i.e., labelled packets can't get through.

@mkbt

This comment has been minimized.

Copy link

@mkbt mkbt commented Aug 28, 2019

@mkbt

This comment has been minimized.

Copy link

@mkbt mkbt commented Aug 28, 2019

@anspectrum

This comment has been minimized.

Copy link
Author

@anspectrum anspectrum commented Aug 28, 2019

@mkbt Thanks for reading and replying.
During my testing I've also created containers with "--privileged" flag and once I spawn them I can see that new vethxxxxxxxx interfaces are created on host machine. Then I enabled mpls processing in container (/proc/sys/net/mpls/conf/vethxxxxx/input) as we all on the host machine but still its not working. Although when I capture traffic on those vethxxxxx interfaces on host machine using tcpdump, I can see MPLS labeled packets.
Really stuck in here :((

((NOTE: Removed BGP config from OP as it was not required))

@anspectrum

This comment has been minimized.

Copy link
Author

@anspectrum anspectrum commented Aug 29, 2019

After much hit and trial and searching information, the FRR is working perfectly now. Here is what I did (for others to be benefited)

Need to enable MPLS processing in Docker container Kernel:

# cat /etc/sysctl.conf 
# content of this file will override /etc/sysctl.d/*
net.mpls.conf.lo.input = 1
net.mpls.conf.eth0.input = 1
net.mpls.conf.eth1.input = 1
net.mpls.conf.eth2.input = 1
net.mpls.platform_labels = 100000

After enabling it execute sysctl -p

VRF Interface Addition on FRR CLI

ip link add ABC type vrf table 10
ip link set dev ABC up
ip link set eth0 master ABC

FIB Entries in Linux for proper working of VRF
((Got this from Andrea Dainese from http://www.routereflector.com/2016/11/working-with-vrf-on-linux/))

ip rule add iif ABC table 10
ip rule add oif ABC table 10

To confirm rules have been added

ip rule show
0:	from all lookup local
32764:	from all oif ABC lookup 10
32765:	from all iif ABC lookup 10
32766:	from all lookup main
32767:	from all lookup default

Sample working config of frr-1

hostname frrr-1
!
enable password zebra
password zebra
!
interface eth1 vrf ABC
 ip address 20.1.1.1/30
!
interface eth0
 ip address 10.1.1.1/30
!
interface lo
 ip address 1.1.1.1/32
!
router bgp 1
 bgp router-id 1.1.1.1
 neighbor 1.1.1.3 remote-as 1
 neighbor 1.1.1.3 update-source lo
 !
 address-family ipv4 unicast
  neighbor 1.1.1.3 next-hop-self
 exit-address-family
 !
 address-family ipv4 vpn
  neighbor 1.1.1.3 activate
  neighbor 1.1.1.3 next-hop-self
 exit-address-family
!
router bgp 1 vrf ABC
 !
 address-family ipv4 unicast
  redistribute connected
  label vpn export auto
  rd vpn export 1:1
  rt vpn both 1:1
  export vpn
  import vpn
 exit-address-family
!
router ospf
 network 0.0.0.0/0 area 0
!
mpls ldp
 router-id 1.1.1.1
 !
 address-family ipv4
  discovery transport-address 1.1.1.1
  !
  interface eth0
  !
  interface lo
  !
 exit-address-family
 !
!
line vty
!
end
@anspectrum anspectrum closed this Aug 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.