DHCP-misbehaviour of LAN-LAN-connected nodes with anygw

[pony1k]

This happens in setups where multiple nodes with lime-proto-anygw in the same cloud are connected with cables.
When a client connects to an AP, it broadcasts a DHCP-Discover into the network.
Since the AP-interfaces and the LAN-interfaces are bridged together, the message will arrive at every cable-connected node. Every node will then send a DHCP-Offer to the client. There would not be a problem if dnsmasq was not to put the interfaces primary ip address into the Server-ID (54)-field of the DHCP-Offer. The anygw-IP is the same everywhere in the cloud. The client then gets a bunch of DHCP-Offers with idenctial Server-IDs. The client then broadcasts a DHCP-Request with the Server-ID they chose to accept the offer from. Usually the chosen DHCP-server would then reply with ACK and store the new lease in its file while the rest of the servers would reply with NAK. In this case however, all the servers create a lease in their file and reply with ACK. This can easily be verified by looking into the lease files of the nodes. They will contain mostly the same mac-addresses.
Attached is a tcpdump of the DHCP-traffic in a network with 8 cable-connected nodes. It countains 2 Discovers from the client, 16 Offers, 1 Request and then 8 ACKs from the servers.
dhcp tcpdump.txt

According to https://dnsmasq-discuss.thekelleys.org.narkive.com/M9Rwt2oy/cannot-override-dhcp-server-identifier-option-54 it is not possible to make dnsmasq send a different Server-ID.
However, I was able to solve the problem by adding an ebtables rule that prevents the DHCP-Discover and DHCP-Request messages from being forwarded.

ebtables -A FORWARD -j DROP -d Broadcast -p IPv4 --ip-dport 67 --ip-proto 17

The nft version would be (I think):

nft add rule bridge filter forward ether daddr ff:ff:ff:ff:ff:ff udp dport 67 counter drop

This effectively replicates the behaviour of batman-adv when setting gw_mode to client everywhere (which is what anygw does).
Maybe we could add this to the anygw-package? Are there any other ideas?

[ilario]

Thanks @pony1k !
A few days ago, also @pedro-nonfree saw an issue that sames like the same but with SSH:
he was connecting with the anygw IP (the same IP for all the nodes) like this:

ssh root@10.1.0.1 (the IP of anygw in our case, using thisnode.info should also work)

to a network made like this:

Pedro's laptop-ethernet port____cable____LAN port-LibreMesh router-LAN port____LAN port-LibreMesh router.

The SSH connection was frequently breaking because both routers were answering him on the same IP.

[ilario]

I am really not into firewalls, but if we add a firewall rule for avoiding the forwarding of packets destinated to the anygw IPs, also this issue should be solved, right?

[pony1k]

I am really not into firewalls, but if we add a firewall rule for avoiding the forwarding of packets destinated to the anygw IPs, also this issue should be solved, right?

There is a rule already in place that does just that, but on Layer 2:
RULE_DO_NOT_FORWARD_ANYGW_1="FORWARD -j DROP -d $ANYGW_MACS"
It doesn't help with this issue though, because the DHCP-Requests are not destined to anygw-address. They are broadcasted.

As for the ssh to anygw issue, I have no clue. It should not happen, as far as I'm concerned.

[G10h4ck]

Thanks @pony1k ! A few days ago, also @pedro-nonfree saw an issue that sames like the same but with SSH: he was connecting with the anygw IP (the same IP for all the nodes) like this:

ssh root@10.1.0.1 (the IP of anygw in our case, using thisnode.info should also work)

to a network made like this:

Pedro's laptop-ethernet port____cable____LAN port-LibreMesh router-LAN port____LAN port-LibreMesh router.

The SSH connection was frequently breaking because both routers were answering him on the same IP.

AnyGW IP is not meant to use for non any-castable stuff, and SSH is one of those non-anycast stuff so the thing reported by Pedro is to be expected, and not to be considered a issue

[G10h4ck]

Except for the fact that a client get multiple leases while using just one, that doesn't seems a big problem per se, what kind of misbehavior is caused by this?

We could add filters, but:

1. What happen in networks where anygw nodes are mixed with L2 non-anygw nodes? Currently nodes with anygw would do the job also for the others, adding this rule I am not sure this could still work (we eventually may decide to not support this use case anymore as it was meant mainly to 4MB flash routers)
2. The filtering would not work between non-DSA hardware switches port, so clients connecting via ethernet cable would keep getting multiple IPs

[pony1k]

Except for the fact that a client get multiple leases while using just one, that doesn't seems a big problem per se, what kind of misbehavior is caused by this?

I was thinking it could be problematic in small address spaces. But dnsmasq seems to give the same IP everywhere, so might not be a problem really.

We could add filters, but:

1. What happen in networks where anygw nodes are mixed with L2 non-anygw nodes? Currently nodes with anygw would do the job also for the others, adding this rule I am not sure this could still work (we eventually may decide to not support this use case anymore as it was meant mainly to 4MB flash routers)

How would that work? Even If the configuration suceeds, the client would have the anygw-address configured as default route. But when the client ist not connected via cable, they would not be able to send unicast packets there, because of the RULE_DO_NOT_FORWARD_ANYGW_1 rule I mentioned above, or am I mistaken?
A little bit offtopic: I think it would be nice to have it documented somewhere what kind of setups are 'supported' and which ones are not.

2. The filtering would not work between non-DSA hardware switches port, so clients connecting via ethernet cable would keep getting multiple IPs

Right! I totally forgot that. I think a saw it working when I tested my proposed rule on two DSA-devices, but now I am unsure on how that could even work. I don't think we even want to filter every packet that goes through the switch, because that would create a lot of overhead on the CPU and the path between CPU and the internal switch.

I will close this issue now, because I don't think anymore that it is really an issue, at least not one that can be fixed.