Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Captive Portal broken in 17.7.4 #1865

Closed
patcsy88 opened this issue Oct 5, 2017 · 44 comments
Closed

Captive Portal broken in 17.7.4 #1865

patcsy88 opened this issue Oct 5, 2017 · 44 comments
Assignees
Labels
bug Production bug
Milestone

Comments

@patcsy88
Copy link

patcsy88 commented Oct 5, 2017

All OPNsense folks,

I have configured Captive Portal to manage a Guest network (VLAN20 and 192.168.20.1/24. I have added the relevant DNS and port range 8000-10000. When i enable the Captive Portal, I am not presented with the login screen. Instead the browser is allowed to connect to all URLs and resolving. Ping from the Guest network is blocked when the Zone is enabled. Guest is on re2_vlan20. The captive_portal process is running in the background and when i browse to http://<firewall_vlan20_add>:8000, i am presented with the index.htm from the Zone template that i uploaded. Any help would be appreciated as i had high expectations moving from pfSense on an APU platform from pcengines.ch.

Regards,
Patrick

@fichtner
Copy link
Member

fichtner commented Oct 5, 2017

Hi Patrick,

Are you using Multi WAN?

Cheers,
Franco

@patcsy88
Copy link
Author

patcsy88 commented Oct 5, 2017

Hi Franco,

Yes I have multiwan setup using gateway groups.

@fichtner
Copy link
Member

fichtner commented Oct 5, 2017

Can you try to enable Firewall: Settings: Advanced: Shared Forwarding? :)

@fichtner fichtner added the support Community support label Oct 5, 2017
@patcsy88
Copy link
Author

patcsy88 commented Oct 5, 2017

As I was remotely accessing the fw via OpenVPN, as soon as i enabled the setting, my connection via my primary link dropped. Whilst i am now able to connect via the secondary link and can confirm that the login page now appears on the guest pc, i am unable to establish a VPN via my primary link. I have issued a reboot...will keep you posted.

@fichtner
Copy link
Member

fichtner commented Oct 5, 2017

Ok, that's a good start. Let me say why this is necessary / not the default:

In FreeBSD the firewalls ipfw and pf are used for different purposes and OPNsense is built around it to make use of both systems. But the problem is that ipfw and pf have trouble cooperating in the case of gateway group policies (or any policy based routing in pf) so we wrote the non-standard shared forwarding kernel patches. This changes the path of packets in pf so that ipfw can always see the packets as well instead of being skipped.

As this may affect NAT and is not in standard FreeBSD. A slight change in firewall behaviour is not unlikely, but it should be easy to fix your policies to allow OpenVPN to keep working.

@patcsy88
Copy link
Author

patcsy88 commented Oct 5, 2017

Ok thanks for the help and info. The Captive Portal documentation would need to updated to include the Shared Forwarding setting information. I will retest on the guest machine this evening via UI browser to confirm what i observed via lynx.

@patcsy88
Copy link
Author

patcsy88 commented Oct 5, 2017

Ok i have tested and the Captive Portal login screen now appears and i can authenticate against my authentication services and browse the Internet. However, i tested my OpenVPN set up via NAT and the secondary link does not work. Can you enlighten me on how i can fix my policies so that my OpenVPN access is available on both links?

Many thanks.
Patrick

@fichtner
Copy link
Member

fichtner commented Oct 5, 2017

Only if you can share how you set up NAT for OpenVPN in the first place... :)

@patcsy88
Copy link
Author

patcsy88 commented Oct 5, 2017

Coming from mostly a Linux environment, I can quickly capture the iptables -t nat output. I am not yet familiar with ipfw and pf but am giving it a shot so here you go:-

rdr on pppoe0 inet proto udp from any to (pppoe0) port = openvpn -> <host_alias> round-robin
rdr on pppoe1 inet proto udp from any to (pppoe1) port = openvpn -> <host_alias> round-robin

@patcsy88
Copy link
Author

patcsy88 commented Oct 5, 2017

Here are the rules:-

pass in quick on pppoe0 reply-to (pppoe0 1.2.3.4) inet proto udp from any to <host_alias> port = openvpn keep state label "USER_RULE: NAT "
pass in quick on pppoe1 reply-to (pppoe1 1.2.3.4) inet proto udp from any to <host_alias> port = openvpn keep state label "USER_RULE: NAT "

@fichtner
Copy link
Member

fichtner commented Oct 5, 2017

So OpenVPN is not on the OPNsense box? Which inbound connection is working, pppoe0 or pppoe1?

@patcsy88
Copy link
Author

patcsy88 commented Oct 5, 2017

pppoe0...pppoe1 is not

@fichtner
Copy link
Member

fichtner commented Oct 5, 2017

judging from your rules, do both pppoe connections have the same gateway "1.2.3.4" ? that would be a config that shouldn't work in FreeBSD at the moment.

The traffic should hit an internal VPN server via pppoe1 but the outgoing packets are probably moved out the other way.

Are you sure that everything is 100% correct (not counting the captive portal) when shared forwarding is off? UDP OpenVPN can have issues with multi WAN, see #952

CC @mimugmail

@patcsy88
Copy link
Author

patcsy88 commented Oct 5, 2017

Sorry 1.2.3.4 and 1.2.3.5. Both gateways have different IP addresses. Yes before enabling the Shared Forwarding both links work. I did a test by disabling and both links can connect.

@patcsy88
Copy link
Author

patcsy88 commented Oct 5, 2017

Whilst UDP is stateless, would not the return route go via the original gateway instead of coming back through the other?

@fichtner
Copy link
Member

fichtner commented Oct 5, 2017

It should, but it has side effects when OpenVPN is running on the OPNsense itself (#952)

Alright, so far so good. Last question would be: do you see the initial connection attempt on pppoe1 on the internal OpenVPN server?

@patcsy88
Copy link
Author

patcsy88 commented Oct 5, 2017

Yes i do and here is the tcpdump on pppoe1 (121.122.111.173) when i initiate the openvpn connection:-

22:35:54.839852 IP (tos 0x0, ttl 63, id 42153, offset 0, flags [DF], proto UDP (17), length 42)
x.x.x.x.openvpn > 113.210.193.25.50083: [udp sum ok] UDP, length 14
22:35:54.992075 IP (tos 0x0, ttl 248, id 64282, offset 0, flags [none], proto UDP (17), length 42)
113.210.193.25.50083 > 121.122.111.173.openvpn: [udp sum ok] UDP, length 14
22:35:54.992975 IP (tos 0x0, ttl 63, id 42195, offset 0, flags [DF], proto UDP (17), length 54)
x.x.x.x.openvpn > 113.210.193.25.50083: [udp sum ok] UDP, length 26
22:35:56.068936 IP (tos 0x0, ttl 63, id 42781, offset 0, flags [DF], proto UDP (17), length 42)
x.x.x.x.openvpn > 113.210.193.25.50083: [udp sum ok] UDP, length 14
22:35:56.212457 IP (tos 0x0, ttl 248, id 64809, offset 0, flags [none], proto UDP (17), length 42)
113.210.193.25.50083 > 121.122.111.173.openvpn: [udp sum ok] UDP, length 14
22:35:56.213012 IP (tos 0x0, ttl 63, id 42791, offset 0, flags [DF], proto UDP (17), length 50)
x.x.x.x.openvpn > 113.210.193.25.50083: [udp sum ok] UDP, length 22
22:36:00.519810 IP (tos 0x0, ttl 63, id 44637, offset 0, flags [DF], proto UDP (17), length 42)
x.x.x.x.openvpn > 113.210.193.25.50083: [udp sum ok] UDP, length 14

@patcsy88
Copy link
Author

patcsy88 commented Oct 5, 2017

Also I have another service on tcp with the same setup and i cannot access that service via pppoe1.

@mimugmail
Copy link
Member

Do you have Gateway switching enabled?

@patcsy88
Copy link
Author

patcsy88 commented Oct 5, 2017

nope

@mimugmail
Copy link
Member

Would you mind enabling it just for testing?

@patcsy88
Copy link
Author

patcsy88 commented Oct 5, 2017

Sure

@patcsy88
Copy link
Author

patcsy88 commented Oct 5, 2017

The behaviour is the same when Gateway Switching is enabled - no access via pppoe1

@patcsy88
Copy link
Author

patcsy88 commented Oct 5, 2017

i had the same setup on pfsense for years npb. As stated in the OPNSense pages, there is only a small % of overlap of code with pfsense but it may be useful to understand the approach that was taken by pfsense and the security risks, if any and how this can be adopted into OPNSense. This is a deal breaker for me and if it is not working (i manage a remote DC and it is imperative i have the backup link working) i may have to revert to pfsense until this is addressed.

@fichtner
Copy link
Member

fichtner commented Oct 5, 2017

pfSense uses a more modified FreeBSD kernel. If it works, why not. We've found it increasingly difficult to maintain a large set of patches with diminishing returns so we try to flatten out the modifications to the bare minimum.

I think this is one of the cases in shared forwarding where the default route in FreeBSD decides where the packet will be sent to as opposed to allowing pf to deal with it directly.

To confirm and assuming your default route is set to pppoe0, please switch it to pppoe1. pppoe0 won't work, but pppoe1 should.

If that's the case, that's something we can definitely start to write a kernel patch for given that we can test it in the correct setting.

fichtner added a commit to opnsense/src that referenced this issue Oct 5, 2017
@fichtner
Copy link
Member

fichtner commented Oct 5, 2017

I have a possible candidate here and will post a modified kernel for testing after internal test is ok: opnsense/src@6eaf07e05

@mimugmail
Copy link
Member

I tested this setup on multwan with one static wan and one dhcp. Works fine without shared forwarding and enable gateway switching.

@fichtner
Copy link
Member

fichtner commented Oct 5, 2017

@mimugmail I think this is a different setup :)

@mimugmail
Copy link
Member

Yep, but double pppoe and a test machine with CP is only at work, and off :)

fichtner added a commit to opnsense/src that referenced this issue Oct 5, 2017
fichtner added a commit to opnsense/src that referenced this issue Oct 5, 2017
fichtner added a commit to opnsense/src that referenced this issue Oct 5, 2017
@fichtner fichtner added bug Production bug and removed support Community support labels Oct 5, 2017
@fichtner fichtner self-assigned this Oct 5, 2017
@fichtner fichtner added this to the 18.1 milestone Oct 5, 2017
@fichtner
Copy link
Member

fichtner commented Oct 5, 2017

@patcsy88 can you try this kernel + shared forwarding?

# opnsense-update -kr 17.7.1-replyto
# /usr/local/etc/rc.reboot

@patcsy88
Copy link
Author

patcsy88 commented Oct 6, 2017

I have introduced the new kernel and enabled Shared Forwarding but the behaviour remains the same:

OpenVPN access (via a host behind the firewall via NAT) on pppoe0 works
OpenVPN access (via a host behind the firewall via NAT) on pppoe1 doesn't

Guest PC on the Guest network gets redirected correctly to the login page via :8000.

@patcsy88
Copy link
Author

patcsy88 commented Oct 6, 2017

Here is output of uname -a to confirm the patched kernel is the running one when i tested the above

FreeBSD OPNsense.ch30ng.org 11.0-RELEASE-p12 FreeBSD 11.0-RELEASE-p12 #0 6eaf07e05(fix_replyto): Thu Oct 5 19:29:06 CEST 2017 root@sensey64:/usr/obj/usr/src/sys/SMP amd64

@patcsy88
Copy link
Author

patcsy88 commented Oct 6, 2017

I did the test "To confirm and assuming your default route is set to pppoe0, please switch it to pppoe1. pppoe0 won't work, but pppoe1 should." and while I did not check the Default Gateway for both links, selecting pppoe1 as the default allowed the remove VPN connection to work on pppoe1 but not on pppoe0 (even with the new kernel).

@patcsy88
Copy link
Author

patcsy88 commented Oct 6, 2017

I am upgrading to 17.7.5 to that i have the latest 'patch' level.

@patcsy88
Copy link
Author

patcsy88 commented Oct 6, 2017

Do i re-introduce the 6eaf07e kernel after i upgraded to 17.7.5? It did not alter the observed behaviour in 17.7.4 though...

@fichtner
Copy link
Member

fichtner commented Oct 6, 2017

The kernel will reinstall its known release version, so you will have to update the test kernel again.

I have one more idea here for how to address this. Thanks for the test with the default gateway, it indicates that reply-to does not work at all, probably due to the fact that it tries to alter the outgoing interface while already in the outgoing path.

@patcsy88
Copy link
Author

patcsy88 commented Oct 6, 2017

I will leave it with the default kernel and wait for the new kernel to test.

@patcsy88
Copy link
Author

patcsy88 commented Oct 7, 2017

Is there a new kernel with the 'one more idea' i can test as i have the weekend to progress this more promptly?

@fichtner
Copy link
Member

fichtner commented Oct 7, 2017

Yes, this one:

# opnsense-update -kr 17.7.1-replyto3
# /usr/local/etc/rc.reboot

@patcsy88
Copy link
Author

patcsy88 commented Oct 8, 2017

Hi Franco,

The patched 17.7.1-replyto3 kernel works! I will continue to test it till end of the weekend but will go ahead and close this. In summary with Shared Forwarding enabled, CP appears to now work correctly (i.e. redirected to Login page) AND the inbound NAT/Routing via dual links to services behind the FW now works properly.

Many thanks for this...you guys rock!

Regards,
Patrick

@patcsy88 patcsy88 closed this as completed Oct 8, 2017
@fichtner
Copy link
Member

fichtner commented Oct 8, 2017

Hi Patrick,

Thanks for this splendid feedback!

We're going to test this more extensively internally before we add it to 17.7.x. It was already added to the upcoming 18.1-BETA code so we get more extensive testing from this angle as well.

I'll drop a final note here when this ships in 17.7.x. Until then make sure the kernel is fixed up after a firmware update.

Cheers,
Franco

@fichtner fichtner mentioned this issue Oct 23, 2017
@patcsy88
Copy link
Author

Hi,

Has this fix been added to the default 17.7.8 as I am now unable to fetch the patched kernel with opnsense-update -kr 17.7.1-replyto3

Regards,
Patrick

@patcsy88
Copy link
Author

Hi,

Please ignore my previous comment as i just tested via both gateways and all appears to be working and hence am assuming this has been merged back to 'trunk'.

Patrick

@fichtner
Copy link
Member

Hi patrick, yes, 17.7.8 has your fix 😊

Cheers,
Franco

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Production bug
Development

No branches or pull requests

3 participants