Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Android networking (except for DNS) does not work with nftables, but works with iptables #105

Open
benwaffle opened this issue Sep 27, 2021 · 30 comments

Comments

@benwaffle
Copy link

When running with the nftables networking, I saw in waydroid logcat that hosts were being resolved, but http/https connections were not connecting.

Switching LXC_USE_NFT to "false" in waydroid-net.sh fixed networking for me.

@xypd
Copy link

xypd commented Oct 5, 2021

How are you changing this? Or is this before compiling? I can't get network access at all.

Would something like this work? waydroid prop set LXC_USE_NFT false

@benwaffle
Copy link
Author

Not compiling, i installed it from the AUR and manually edited the waydroid-net.sh file wherever it was placed in /usr

@xypd
Copy link

xypd commented Oct 5, 2021

Okay cool thanks. Will give it a shot.

@xypd
Copy link

xypd commented Oct 5, 2021

Btw, found this on the postmarketOS wiki

Troubleshooting Networking

The container may not have access to the Internet out of the box. To access the network you need to allow access from and to the container via the firewall (otherwise DHCP will be blocked and the container will never get the IP) and configure a default gateway in the waydroid container (gateway IP is what your host device has on waydroid0 interface):

$ echo "ip route add default via 192.168.250.1" | sudo waydroid shell

Have not tried it yet, so not sure if this is PMOS specific

@benwaffle
Copy link
Author

Inside the waydroid shell you csn run ip a to check if the container got an IP via dhcp

@xypd
Copy link

xypd commented Oct 5, 2021

Sadly nothing is seeming to work. Will put it on pause till I have another block of time.

I probably need to work on my container knowledge, which would be helpful on this and another future project.

@trn1ty
Copy link

trn1ty commented Oct 20, 2021

Also having this issue on PMOS edge.

@erfanoabdi
Copy link
Member

22f671b
we disabled nft

@xypd
Copy link

xypd commented Oct 21, 2021

I tried this manually changing this config, but still not working. Should I remove and reinstall everything, @erfanoabdi ?

And if so, what is the best way to uninstall? There are pieces all over the place.

@erfanoabdi
Copy link
Member

not needed to uninstall
it should have worked with just a script change
i just disabled it as @benwaffle confirms it works without nft
i highly like to enable it again if it doesn’t change anything, because broke internet for other users

@SameExpert
Copy link

SameExpert commented Oct 25, 2021

To make internet on WayDroid working with nftables, an nftables rule in /etc/nftables.d is needed to allow waydroid0 (virtual) network interface to access internet. This is for good in terms of security, as I believe. I highly recommend to use nftables and create an nftables rule:

#!/usr/sbin/nft -f
table inet filter {
    chain input {
        iifname "waydroid0" accept comment "Allow incoming network traffic from WayDroid"
    }
    chain forward {
        iifname "waydroid0" accept comment "Allow outgoing network traffic from WayDroid"
        ct state {established, related} counter accept comment "accept established connections"
    }
}

@pizdjuk
Copy link

pizdjuk commented Oct 26, 2021

I add the snippet above to /etc/nftables.conf, restarted nftables.service but nothing happened. Waydroid still not has connetcion.

And if I change LXC_USE_NFT to false, the session doesnt start at all.

waydroid log says that "failed to attach "vethxxxxxx" to bridge "waydroid0", bridge interface doesn't exists

Distro: Manjaro ARM on Pinephone

@SameExpert
Copy link

SameExpert commented Oct 27, 2021

I add the snippet above to /etc/nftables.conf, restarted nftables.service but nothing happened. Waydroid still not has connetcion.

If you add it to /etc/nftables.d/51_waydroid.nft (name it whatever but with .nft extension) it should work. Alpine added it in this commit. Now internet works for me with LXC_USE_NFT="true", but I need to type this every time I start WayDroid:

$ echo "ip route add default via 192.168.250.1" | sudo waydroid shell

And if I change LXC_USE_NFT to false, the session doesnt start at all.

waydroid log says that "failed to attach "vethxxxxxx" to bridge "waydroid0", bridge interface doesn't exists

That happens for me too, because it falls back to iptables and demands ip_tables module which is missing in the kernel. Regarding this I opened #178. So using LXC_USE_NFT="true" works for me.

@erfanoabdi
Copy link
Member

To make internet on WayDroid working with nftables, an nftables rule in /etc/nftables.d is needed to allow waydroid0 (virtual) network interface to access internet. This is for good in terms of security, as I believe. I highly recommend to use nftables and create an nftables rule:

#!/usr/sbin/nft -f
table inet filter {
    chain input {
        iifname "waydroid0" accept comment "Allow incoming network traffic from WayDroid"
    }
    chain forward {
        iifname "waydroid0" accept comment "Allow outgoing network traffic from WayDroid"
        ct state {established, related} counter accept comment "accept established connections"
    }
}

thanks for the rule
i think it makes sense to include it on waydroid package and re-enable nft once and for all lol

@pizdjuk
Copy link

pizdjuk commented Oct 30, 2021

for some reaseon, andoid in the container doesnt pick up a gateway...

@SameExpert
Copy link

SameExpert commented Nov 2, 2021

for some reaseon, andoid in the container doesnt pick up a gateway...

Tried this too?

$ echo "ip route add default via 192.168.250.1" | sudo waydroid shell

@pizdjuk
Copy link

pizdjuk commented Nov 2, 2021

Clear, it works. But it is an ugly workaround, moreover -- not persistent. Restarting android within the container could (will?) break the setting.

The gateway should be picked up automatically -- why this is not the case is a magic. Because with iptables seems to work. With nft -- no. Meanwhile both filtering rules (iptables and nft) seem to be absolutely indentical.

@Lushioe
Copy link

Lushioe commented Mar 30, 2022

i tried by changing the nftables but nothing seems to work, if someone can help me pls im on arch (btw) basically the wifi turn on and automatically back off pls send help

@folliehiyuki
Copy link

To make internet on WayDroid working with nftables, an nftables rule in /etc/nftables.d is needed to allow waydroid0 (virtual) network interface to access internet. This is for good in terms of security, as I believe. I highly recommend to use nftables and create an nftables rule:

#!/usr/sbin/nft -f
table inet filter {
    chain input {
        iifname "waydroid0" accept comment "Allow incoming network traffic from WayDroid"
    }
    chain forward {
        iifname "waydroid0" accept comment "Allow outgoing network traffic from WayDroid"
        ct state {established, related} counter accept comment "accept established connections"
    }
}

These rules seem a bit loose to me. For context, here is how libvirt does network filtering. To translate to nft term it is something like this (libvirt NAT mode):

table inet filter {
  chain input {
    ...
    iifname "virbr0" udp dport { 53, 67 } accept
    iifname "virbr0" tcp dport { 53, 67 } accept
  }

  chain forward {
    ...
    oifname "virbr0" ip daddr 192.168.122.0/24 ct state { established, related } accept
    iifname "virbr0" ip saddr 192.168.122.0/24 accept
    iifname "virbr0" oifname "virbr0" accept
    oifname "virbr0" reject with icmpx type port-unreachable
    iifname "virbr0" reject with icmpx type port-unreachable
  }
}

table ip nat {
  chain postrouting {
    ...
    ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 masquerade
  }
}

I haven't tested these rules for waydroid though.

Also, I don't think Alpine should ship firewall rules for waydroid. It should be provided upstream (here) or manually configured by the users to their liking.

@folliehiyuki
Copy link

if [ -n "$LXC_IPV6_ARG" ] && [ "$LXC_IPV6_NAT" = "true" ]; then
NFT_RULESET="${NFT_RULESET}
add table ip6 lxc;
flush table ip6 lxc;
add chain ip6 lxc postrouting { type nat hook postrouting priority 100; };
add rule ip6 lxc postrouting ip saddr ${LXC_IPV6_NETWORK} ip daddr != ${LXC_IPV6_NETWORK} counter masquerade;
"

This is a bit of a rant, but the NAT firewall rule here is dangerous. Usually when you assign IPv6 for a bridge, shouldn't it use prefix delegation (so the bridge is like a subnet of the previous subnet)? AFAIK IPv6 source nat doesn't exist.

@v1nsai
Copy link

v1nsai commented Jul 3, 2022

I'm troubleshooting a problem that may be related, not getting internet after a clean delete/init of waydroid. I checked that nftables is disabled, but I notice that my iptables is completely blank, is this expected?

$ sudo waydroid shell
/system/bin/sh: No controlling tty: open /dev/tty: No such file or directory
/system/bin/sh: warning: won't have full job control
:/ # iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination  

@Idcrafter
Copy link

on Fedora 36 does this show
:/ # iptables -L
iptables v1.8.4 (legacy): can't initialize iptables table `filter': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

@v1nsai
Copy link

v1nsai commented Jul 19, 2022

On Arch this seems to be fixed now with a fresh install, try deleting all the folders, re-initializing and upgrading waydroid.

sudo rm -rf /var/lib/waydroid /home/.waydroid
rm -rf ~/waydroid ~/.share/waydroid ~/.local/share/applications/*aydroid* ~/.local/share/waydroid

sudo waydroid init -f
sudo waydroid upgrade

It's overkill to upgrade right after a fresh init, but it double checks you've got the latest.

More info on the arch wiki here.

@RpxdYTX
Copy link

RpxdYTX commented Oct 12, 2022

$ echo "ip route add default via 192.168.250.1" | sudo waydroid shell

When running this with or without nft, it gives RTNETLINK answers: Network is unreachable

@pizdjuk
Copy link

pizdjuk commented Oct 13, 2022

$ echo "ip route add default via 192.168.250.1" | sudo waydroid shell

When running this with or without nft, it gives RTNETLINK answers: Network is unreachable

It could be, that the waydroid container gets different address. go into it and see how the routing table and addresses look out:

waydroid shell

ip r
ip a

Ahm. In android would be

route
ifconfig

and then you should place an address of yout host adapter in place of 192.168.250.1 in the command below. It could be some 192.168.100.1 or something like this.

@RpxdYTX
Copy link

RpxdYTX commented Oct 25, 2022

Found mine "192.168.240.1" no errors on script but no wifi either

@oklopfer
Copy link

Found mine "192.168.240.1" no errors on script but no wifi either

same here except getting still getting errors on the script :(

@vrmaurice
Copy link

vrmaurice commented Feb 24, 2023

Not compiling, i installed it from the AUR and manually edited the waydroid-net.sh file wherever it was placed in /usr

I tried this on Fedora, now whenever I do waydroid session start I get this error:
[16:12:38] RuntimeError: Command failed: % /usr/lib/waydroid/data/scripts/waydroid-net.sh start

Even when I revert it back to false it still does that. How is this even possible??

Reinstalling fixed it tho...

@firefoxlover
Copy link

I think there also is a problem with systemd-resolved.

OpenSnitch only doesnt break it btw.

@chilledfrogs
Copy link

I actually had the same error message as @vrmaurice (so could be something else but hey) on postmarketOS edge recently, after trying a few things with the script individually it eventually occurred to me that strangely enough iproute2-minimal isn't listed as a dependency which made ip be handled by busybox which has different permission semantics, so I'll fix that on aports but it might be some equivalent issue with how Fedora handles that command...?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests