Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bhyve tap not added to bridge #301

Closed
awrede opened this issue Jul 3, 2018 · 7 comments
Assignees

Comments

@awrede
Copy link

@awrede awrede commented Jul 3, 2018

FreeBSD 11.1-RELEASE-p10
cbsd version 11.1.16

The tap interface for the bhyve instance is not added to the bridge if the underlying Ethernet interface has an mtu other than 1500.

  • parent interface is em0:
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1420
	options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
	ether 38:2c:4a:af:fb:3f
	hwaddr 38:2c:4a:af:fb:3f
	inet 192.168.10.92 netmask 0xffffff00 broadcast 192.168.10.255 
	inet 192.168.10.71 netmask 0xffffff00 broadcast 192.168.10.255 
	inet6 fe80::3a2c:4aff:feaf:fb3f%em0 prefixlen 64 scopeid 0x1 
	inet6 2607:f2c0:a000:1d5:3::c0 prefixlen 64 
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
  • bridge is correctly configured with the mtu from the parent, with only em0 as a member:
 bridge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1420
	description: em0
	ether 02:96:87:4c:28:01
	nd6 options=1<PERFORMNUD>
	groups: bridge 
	id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
	maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
	member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 1 priority 128 path cost 20000

  • during bstart the message ifconfig: BRDGADD tap2: Invalid argument is displayed. The tap interface shows an mtu of 1500.

If I set mtu in bconfig->bhyvenic->nic1->nic_mtu to 1420, the interface is still not added to the bridge (I get the ifconfig: BRDGADD tap2: Invalid argument message) but now the tap interface shows the correct mtu:

tap2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1420
	description: collabra-nic0
	options=80000<LINKSTATE>
	ether 00:bd:51:cd:2b:02
	hwaddr 00:bd:51:cd:2b:02
	inet6 fe80::2bd:51ff:fecd:2b02%tap2 prefixlen 64 scopeid 0x5 
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	media: Ethernet autoselect
	status: active
	groups: tap 
	Opened by PID 19630

running 'ifconfig bridge1 addm tap2' manually adds the tap2 to the bridge:

bridge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1420
	description: em0
	ether 02:96:87:4c:28:01
	nd6 options=1<PERFORMNUD>
	groups: bridge 
	id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
	maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
	member: tap2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 5 priority 128 path cost 2000000
	member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 1 priority 128 path cost 20000

I suspect that the ifconfig addm is performed before the mtu of the tap interface is set. Ideally the mtu for the tap interface would be taken from the bridge

@olevole olevole self-assigned this Jul 3, 2018
@olevole

This comment has been minimized.

Copy link
Collaborator

@olevole olevole commented Jul 4, 2018

Hmm, interesting

CBSD ( at least the latest versions ) inherits parent NIC MTU by default:

https://github.com/cbsd/cbsd/blob/develop/bhyve.subr#L399-L406

I'll now try to find an opportunity to reproduce your cases ( as I understand it, just change the physical interface MTU is enough). But before - can you update CBSD to latest version ( 11.2.0, e.g from the ports ) and after cbsd initenv сonfirm that the problem remains ? Thanks!

@olevole

This comment has been minimized.

Copy link
Collaborator

@olevole olevole commented Jul 4, 2018

Yes, you are right, adding tap interface to bridge performed before setting MTU. It's amazing that on my FreeBSD-CURRENT (12) it works without errors

@olevole

This comment has been minimized.

Copy link
Collaborator

@olevole olevole commented Jul 4, 2018

@awrede We can test your assumptions by moving MTU settings block before 'addm' in /usr/local/cbsd/bhyve.subr (and re-run 'cbsd initenv' after this change)

E.g:

e49f76d

Or you can download and apply this modification via:

cd /usr/local/cbsd
mv bhyve.subr bhyve.subr-orig
fetch -o bhyve.subr https://raw.githubusercontent.com/cbsd/cbsd/e49f76d530ebe68fbb8cb2c8d037dedfb701f3aa/bhyve.subr
cbsd initenv

!!! This is for CBSD 11.2.0 !!!

olevole added a commit that referenced this issue Jul 15, 2018
Inherits parent MTU to stop "ifconfig: BRDGADD XXX: Invalid argument" when
parent MTU less then tap. Fix args nic_info - I forgot the argument --mtu args
Issue #301
@olevole olevole closed this Jul 15, 2018
@awrede

This comment has been minimized.

Copy link
Author

@awrede awrede commented Jul 20, 2018

@mekanix

This comment has been minimized.

Copy link
Member

@mekanix mekanix commented Jul 20, 2018

If it's in the master, expect it to be fixed in the next version. We still don't have ETA for the next release, but thanx for confirmation of the fix!

@olevole

This comment has been minimized.

Copy link
Collaborator

@olevole olevole commented Jul 21, 2018

@awrede yes, it seems I have corrected one more place apart from bhyve.subr. I found these places after I suggested you try to fetch 'bhyve.subr'. Well, I think, slowly we will start preparing the release. CBSD must not contain known errors ;-)) Thx!

@olevole

This comment has been minimized.

Copy link
Collaborator

@olevole olevole commented Jul 28, 2018

commited to ports as 11.2.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.