Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NAT before IPsec: limitations and workarounds #1773

Closed
fraenki opened this issue Aug 10, 2017 · 34 comments
Closed

NAT before IPsec: limitations and workarounds #1773

fraenki opened this issue Aug 10, 2017 · 34 comments
Labels
help wanted Contributor missing / timeout

Comments

@fraenki
Copy link
Member

fraenki commented Aug 10, 2017

As described in #440 there are two limitations in our new implementation:

The first one is related to "My identifier":

root@opnsense:~ # /sbin/setkey -f /tmp/setkeyAsL4tW
libipsec: invalid IP address while parsing "host.example.com"
line 1: hostname nor servname provided, or not known at [ out ipsec esp/tunnel/host.example.com-1.2.3.4/require]
parse failed, line 1.

This means that it's required to use an IP address for "My identifier". Anything else, for example "Distinguished name", will not work. (Well, maybe it might work to get the IP address from the selected interface for those cases... but I'm not sure.)


Besides that I've found another case where "NAT before IPsec" is not yet working: when OPNsense is behind NAT. In this case there's a mismatch (as can be seen in the setkey output):

# working; created by strongSwan
  esp/tunnel/<localRouterIP>-<remoteIP>/require

# mismatch; created for "NAT before IPsec"
  esp/tunnel/10.0.0.254-<remoteIP>/unique:1

Apparently, the Router IP that is used for this tunnel is not configured on the OPNsense firewall. But instead there's a port forward on the Router.


As opposed to my original submission, I think it might be possible to workaround both limitations. The reason why it's not working is the same in both cases: the local entry is erroneous in the setkey output. We should add a new option to the GUI to allow the user to overwrite this value. I think in both cases the user should be able to enter the right value, it's hard to "guess" it because there are many corner cases. Additionally a bit of background information as to when this is required should be added to the help text.

@fraenki
Copy link
Member Author

fraenki commented Feb 2, 2018

I plan to work on this (later this year), please assign the issue to me.

@fichtner fichtner added this to the 18.7 milestone Feb 3, 2018
@fichtner fichtner added the feature Adding new functionality label Feb 3, 2018
@fichtner
Copy link
Member

fichtner commented Feb 3, 2018

(can always move milestone up if you want)

@ruffy91
Copy link

ruffy91 commented May 29, 2018

We are affected by both this issue and #2173
Our constellation is: two Site-2-Site Tunnels, on the second one the remote side is behind NAT for phase 1 (ESP-UDP) and we need to use BiNAT because we have colliding subnets.
@fraenki are you still planning to work on this soon?

We manually added the correct spdadd entries via setkey which works.

How can I add spdadd entries so they survive a reboot/upgrade?

@fraenki
Copy link
Member Author

fraenki commented May 29, 2018

@ruffy91 I still plan to work on this, but I had to postpone it to July/August (it will not be ready for OPNsense 18.7).

@fichtner fichtner added this to the Future milestone Jul 15, 2018
@fichtner fichtner removed this from the Future milestone Jul 30, 2018
@banym
Copy link

banym commented Oct 30, 2018

Hi, I had similar requirement for some BINAT requirements.

I run into the problem that autogenerated spd entries are not correctly generated when the Remote ID in phase 1 is not the external remote IP. For reasons the other side is not able to change their used ID to the external IP and so I am stuck to a workaround.

Manual set SPD entries fixed it for me temporarily but I would like to manage this entries persistent over api or gui that this configurations are backed up and documented.

Regards,
Dominik

@officebluesource
Copy link

@fraenki was there a change with OpnSense 18.7.10 ?
spdadd workaround isn't working anymore :(

regards,
Michael

AdSchellevis added a commit that referenced this issue Feb 5, 2019
…can't add a tunnel. switch to remote-gateway if that is a valid ip address.

It's not a complete fix, for #1773 but improves error logging and adds functionality when peer identifier is currently broken.
@AdSchellevis
Copy link
Member

@bluesource 8f45e0d should fix your issue.

@officebluesource
Copy link

officebluesource commented Feb 5, 2019

just for info: its working again after a call with deciso (business support).

The Issue was: Remote FW is only accepting a Cert as peer identifier and therefore we set "KeyID" with value "%any, although remote FW has a fix IP. Then the Manual SPD entries are not set.

Fix from deciso:
8f45e0d

and btw: great support from Deciso!

fichtner pushed a commit that referenced this issue Mar 3, 2019
…can't add a tunnel. switch to remote-gateway if that is a valid ip address.

It's not a complete fix, for #1773 but improves error logging and adds functionality when peer identifier is currently broken.

(cherry picked from commit 8f45e0d)
@jroehler23
Copy link

Hi, is this still open to fix in 19.1.x? I had to change my WAN configuration in several remote sites from PPPoE to a static IP behind a NAT router. The normal IPsec tunnels are all working normal, but the additional networks of the remote sides, which must be NATed before IPsec are not entering the tunnel any more.

Regards,
Jochen

@AdSchellevis
Copy link
Member

we don't have other fixed queued than 8f45e0d at the moment.

@fichtner
Copy link
Member

8f45e0d was shipped as fe38ca7 in 19.1.3

@jroehler23
Copy link

OK. Can you help me to find out where or why the packets are not finding the way into the tunnel? I really need this feature back working!

When I remove the manual SPD for the additional network in phase 2, the packets were send out over the standard gateway via WAN. So IPsec configuration is recognizing the additional network. But I can't see any SNAT transformation in the firewall log of the packets. The configuration for this is unchanged and made in the 1to1 NAT of the firewall section.

@jroehler23
Copy link

jroehler23 commented May 22, 2019

To make it short: I've found my mistake, NAT before IPsec is working when OPNsense is behind a NAT router.

Before I had to change the WAN configuration, I used 'my IP address' and 'peer IP address' in the phase 1 as identifiers at both ends. After I moved one router behind a NAT router, I had to use 'IP address' on one side for one identifier. I chose 'my identifier' in the OPNsense behind the NAT router and put the public WAN IP address of the NAT router in. Everything worked fine, except the additional networks behind this OPNsense.
So the router behind the NAT must have 'my IP address' set as 'my identifier' otherwise the NATed packets will not find the way into the IPsec tunnel! The change in the identifiers must be done at the other end of the tunnel: the 'peer identifier' must be set to the private WAN IP of the OPNsense behind the NAT router. Now this is obvious to me...

Maybe it is possible to put this hint into the IPsec BINAT wiki.

@fichtner fichtner added help wanted Contributor missing / timeout and removed feature Adding new functionality labels May 22, 2019
@mimugmail
Copy link
Member

@olivierIllogika
Copy link

Hi, we ran into this issue with the router behind a NAT. The phase 1 identifier is set to 'IP address' (with public IP in value field). The manual SPD entry was incorrectly using this IP address as the local tunnel endpoint instead of the opnsense IP from which the tunnel is initiated.

My fix was to modify /usr/local/etc/inc/plugins.inc.d/ipsec.inc with the following:

    // add manual added spd entries
    if (!empty($config['ipsec']['phase1']) && !empty($config['ipsec']['phase2'])) {
        foreach ($config['ipsec']['phase1'] as $ph1ent) {
            foreach ($config['ipsec']['phase2'] as $ph2ent) {
                if (!isset($ph2ent['disabled']) && $ph1ent['ikeid'] == $ph2ent['ikeid'] && !empty($ph2ent['spd'])) {
-                    $myid_data = ipsec_find_id($ph1ent, "local");
+                    $myid_data = ipsec_get_phase1_src($ph1ent);
                    $peerid_spec = ipsec_find_id($ph1ent, "peer");
                    if (!is_ipaddr($peerid_spec)) {
                        if (is_ipaddr($ph1ent['remote-gateway'])) {
                            $peerid_spec = $ph1ent['remote-gateway'];
                        } else {
                            log_error(sprintf(
                                "spdadd: unable to match remote network on %s or %s [skipped]",
                                $peerid_spec,
                                $ph1ent['remote-gateway']
                            ));
                            continue;
                        }
                    }
                    foreach (explode(',', $ph2ent['spd']) as $local_net) {

This is probably incorrect for a general solution, but it fixed my issue.

@AdSchellevis
Copy link
Member

@olivierIllogika good to hear this solved your issue, I'm always struggling a bit with the setkey (https://www.freebsd.org/cgi/man.cgi?query=setkey) documentation.

At a first glance it looks like we missed the correct policy source definition here, which resulted in a working setup when using the external address.

Looking at the code, it looks like it will only work at the moment if ipsec_find_id($ph1ent, "local") and ipsec_get_phase1_src($ph1ent) outcomes are equal, now we need to validate if that is indeed the case.

I've prepared a fix 30a5536, which incorporates your change. now we need to find some people to validate this on their functional setups.

@fraenki @mimugmail do you have the time and spirit to test this patch? Thanks in advance!

@olivierIllogika
Copy link

@AdSchellevis fair warning: I'm not at all familiar with the code or setkey. The setkey documentation could indeed be a little more specific, but as I understand it, the "tunnel/%s-%s/require" string needs to match the tunnel's endpoint:

[...] If mode is tunnel you must specify the end-point addresses of the SA as src and dst with `-' between these addresses, which is used to specify the SA to use. [...]

I'm not sure about the 'SA' part, but this (linux) documentation mentions the tunnel explicitly: https://www.mad-hacking.net/documentation/linux/networking/ipsec/spd-setkey.xml

If this assumption is correct, it might make sense to (1) remove the $peerid_spec is_ipaddr() check and just get the phase 1 destination from the start; and (2) rename the variables from $myid_data/$peerid_spec to $tunnel_src/$tunnel_dst

@AdSchellevis
Copy link
Member

@olivierIllogika I think you are right, we could do a bit more here to clarify things. let me refactor this a bit further and add some logging in case the end result varies from what it does on current stable.

@AdSchellevis
Copy link
Member

@olivierIllogika this should be it 1e8ebbf , it needs 30a5536 first as a starting point.

My setup still works with the patch applied, hopefully this removes the limitations, it will simplify the code as well (after removal of the extra temporary logging).

fichtner pushed a commit that referenced this issue Jul 23, 2019
…n. ref #1773

(cherry picked from commit 30a5536)
(cherry picked from commit 1e8ebbf)
@AdSchellevis
Copy link
Member

@fraenki I think we can close this issue, most limitations should be solved with 755849d, what do you think?

@Jesper-fr
Copy link

Hi,
is this NAT before IPSEC tunnel still an issue. I have just upgraded my Opnsense to ver. 19.7.2, but it still doens't work. Is Strongswan patch still required to get it to work ? Strongswan ver. 5.8.0

@AdSchellevis
Copy link
Member

@Jesper-fr works like a charm on my end, we don't need patches to Strongswan. Just don't forget to add manual spd entries (some documentation can be found here https://docs.opnsense.org/manual/how-tos/ipsec-s2s-binat.html)

@Jesper-fr
Copy link

My problem is that I only have access to one end of the tunnel. The opposite end is at a customer site.

@AdSchellevis
Copy link
Member

IPsec can be tricky to debug, certainly if you can't access both sides.

@Jesper-fr
Copy link

Jesper-fr commented Aug 15, 2019

I can't see that the NAT address is being sent through the tunnel. Encode counters are not increasing when I do a ping from a server in my end of the tunnel towards the customer server.

@mimugmail
Copy link
Member

Screenshots and logs please, best via Forum threas

@Jesper-fr
Copy link

I litle more explanation
It's an existing tunnel, My end 10.222.0.0/16 remote end 10.1.204.0/24
I have a server in my end coming from 10.220.2.72 which I translate to 10.222.8.4. If I ping from the local server and do a packet capture on WAN interface , then I can see the translated addr. towards the server I try to reach (10.1.204.108).

firewall interfaces:
VLAN 10 (10.220.0.9/22
VLAN 32 (10.222.8.4/22)
WAN

My local server is comming via VLAN 10 and is being translated to interface addr. on VLAN 32.

I will provide screenshots tomorrow.

@Jesper-fr
Copy link

Jesper-fr commented Aug 16, 2019

NAT rule

please find attached a packet capture . Ping done from a server 10.220.2.13 and it's being translated to 10.222.8.4, but the packet towards 10.1.204.108 is not being tunneled.

I have tried to do 2 configs. Both configs are outbound nat

  1. WAN as outgoing interface (packet capture WAN)
  2. IPSEC as outgoing interface (packet capture IPSEC)

PACKET CAPTURE WAN
VLAN10
em0 09:29:01.766196 00:26:0a:27:d6:00 > 00:50:56:a8:de:54, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 127, id 32194, offset 0, flags [none], proto ICMP (1), length 60)
10.220.2.13 > 10.1.204.108: ICMP echo request, id 1, seq 43537, length 40
10.220.2.13 > 10.1.204.108: ICMP echo request, id 1, seq 43538, length 40
10.220.2.13 > 10.1.204.108: ICMP echo request, id 1, seq 43539, length 40
10.220.2.13 > 10.1.204.108: ICMP echo request, id 1, seq 43540, length 40
VLAN10
em0 09:29:06.475942 00:26:0a:27:d6:00 > 00:50:56:a8:de:54, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 127, id 32195, offset 0, flags [none], proto ICMP (1), length 60)
VLAN10
em0 09:29:11.465461 00:26:0a:27:d6:00 > 00:50:56:a8:de:54, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 127, id 32196, offset 0, flags [none], proto ICMP (1), length 60)
VLAN10
em0 09:29:16.475901 00:26:0a:27:d6:00 > 00:50:56:a8:de:54, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 127, id 32197, offset 0, flags [none], proto ICMP (1), length 60)
WAN
em1 09:29:01.766572 00:50:56:a8:47:5c > 00:26:0a:27:d6:00, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 126, id 32194, offset 0, flags [none], proto ICMP (1), length 60)
WAN
em1
10.222.8.4 > 10.1.204.108: ICMP echo request, id 8203, seq 43537, length 40
10.222.8.4 > 10.1.204.108: ICMP echo request, id 8203, seq 43538, length 40
10.222.8.4 > 10.1.204.108: ICMP echo request, id 8203, seq 43539, length 40
10.222.8.4 > 10.1.204.108: ICMP echo request, id 8203, seq 43540, length 40
WAN
em1 09:29:06.476327 00:50:56:a8:47:5c > 00:26:0a:27:d6:00, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 126, id 32195, offset 0, flags [none], proto ICMP (1), length 60)
WAN
em1 09:29:11.465774 00:50:56:a8:47:5c > 00:26:0a:27:d6:00, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 126, id 32196, offset 0, flags [none], proto ICMP (1), length 60)
WAN
em1 09:29:16.476087 00:50:56:a8:47:5c > 00:26:0a:27:d6:00, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 126, id 32197, offset 0, flags [none], proto ICMP (1), length 60)


PACKET CAPTURE IPSEC
VLAN10
em0 09:36:51.695292 00:26:0a:27:d6:00 > 00:50:56:a8:de:54, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 127, id 32198, offset 0, flags [none], proto ICMP (1), length 60)
10.220.2.13 > 10.1.204.108: ICMP echo request, id 1, seq 43547, length 40
10.220.2.13 > 10.1.204.108: ICMP echo request, id 1, seq 43548, length 40
10.220.2.13 > 10.1.204.108: ICMP echo request, id 1, seq 43549, length 40
10.220.2.13 > 10.1.204.108: ICMP echo request, id 1, seq 43550, length 40
VLAN10
em0 09:36:56.466359 00:26:0a:27:d6:00 > 00:50:56:a8:de:54, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 127, id 32199, offset 0, flags [none], proto ICMP (1), length 60)
VLAN10
em0 09:37:01.483428 00:26:0a:27:d6:00 > 00:50:56:a8:de:54, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 127, id 32200, offset 0, flags [none], proto ICMP (1), length 60)
VLAN10
em0 09:37:06.466768 00:26:0a:27:d6:00 > 00:50:56:a8:de:54, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 127, id 32201, offset 0, flags [none], proto ICMP (1), length 60)
WAN
em1 09:36:51.695470 00:50:56:a8:47:5c > 00:26:0a:27:d6:00, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 126, id 32198, offset 0, flags [none], proto ICMP (1), length 60)
WAN
em1
10.220.2.13 > 10.1.204.108: ICMP echo request, id 1, seq 43547, length 40
10.220.2.13 > 10.1.204.108: ICMP echo request, id 1, seq 43548, length 40
10.220.2.13 > 10.1.204.108: ICMP echo request, id 1, seq 43549, length 40
10.220.2.13 > 10.1.204.108: ICMP echo request, id 1, seq 43550, length 40
WAN
em1 09:36:56.466440 00:50:56:a8:47:5c > 00:26:0a:27:d6:00, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 126, id 32199, offset 0, flags [none], proto ICMP (1), length 60)
WAN
em1 09:37:01.483573 00:50:56:a8:47:5c > 00:26:0a:27:d6:00, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 126, id 32200, offset 0, flags [none], proto ICMP (1), length 60)
WAN
em1 09:37:06.466915 00:50:56:a8:47:5c > 00:26:0a:27:d6:00, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 126, id 32201, offset 0, flags [none], proto ICMP (1), length 60)

in below image, test is 10.220.2.13 - Salling group is 10.1.204.108 - VLAN32_NAT_INTERFACE is 10.222.8.4

NAT rule -IPSEC
NAT rule -WAN

@AdSchellevis
Copy link
Member

@Jesper-fr can you please use our forum for discussions (as Michael requested)? .

@Jesper-fr
Copy link

Sorry....i'm new to GitHub. Were do I find forum ?

@AdSchellevis
Copy link
Member

no problem, things happen, our forum can be found here : https://forum.opnsense.org/

@Jesper-fr
Copy link

Jesper-fr commented Aug 16, 2019 via email

@AdSchellevis
Copy link
Member

please just add enough detail there as @mimugmail requested. (free) community support is provided by people helping out in their own time. If you have an actual reproducible bug or a feature request, you can always open a new ticket of your own.

Spamming threads in existing issues won't help solving the original ticket. (this case should probably already be closed, since the specific work-arounds where fixed a month ago. @fraenki close now?)

@AdSchellevis
Copy link
Member

This issue has been automatically timed-out (after 180 days of inactivity).

For more information about the policies for this repository,
please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.

If someone wants to step up and work on this issue,
just let us know, so we can reopen the issue and assign an owner to it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Contributor missing / timeout
Development

No branches or pull requests

10 participants