NAT before IPsec is not functional

[fraenki]

UPDATE

A limitation of the PF implementation on FreeBSD makes it impossible to apply NAT rules before sending packets through an IPsec tunnel. (I think OpenBSD fixed this limitation in their PF implementation a while ago. Unfortunately, the PF implementation on FreeBSD no longer follows OpenBSD's developments, so it's unlikely that we see this being ported over to FreeBSD.)

There are two steps required to fix this issue:

1. Patch strongSwan
2. Revive some old code

@ermal provided a nice explanation how these two things solve this issue. (Note that Ermal did mention "other ways of doing it", so there's likely a different/better solution possible.)

The drawbacks of this fix are obvious: The strongSwan patch will never be included in any upstream release, thus it may break with any new (major) release of strongSwan. And while this solution works very well, it's just a workaround for a limitation in PF/FreeBSD.

But, to be honest, the inability to use "NAT before IPsec" is a showstopper for me. That's why I'm currently using a custom build of strongSwan alongside a small patch to vpn.inc.

ORIGINAL REPORT

(for reference only)

If you configure IPsec NAT, the IPsec phase 2 tunnels may become unstable. In my case all phase 2 tunnels remain disconnected:

Oct 19 12:09:28 opnsense charon: 15[IKE] <con1-000|1> IKE_SA con1-000[1] established between X[X]...Y[Y]
Oct 19 12:09:28 opnsense charon: 15[IKE] IKE_SA con1-000[1] established between X[X]...Y[Y]
Oct 19 12:09:28 opnsense charon: 15[IKE] <con1-000|1> scheduling reauthentication in 28101s
Oct 19 12:09:28 opnsense charon: 15[IKE] <con1-000|1> maximum IKE_SA lifetime 28641s
Oct 19 12:09:32 opnsense charon: 13[IKE] <con1-000|1> sending retransmit 1 of request message ID 951205933, seq 4
Oct 19 12:09:39 opnsense charon: 12[IKE] <con1-000|1> sending retransmit 2 of request message ID 951205933, seq 4
Oct 19 12:09:52 opnsense charon: 06[IKE] <con1-000|1> sending retransmit 3 of request message ID 951205933, seq 4
Oct 19 12:10:16 opnsense charon: 08[IKE] <con1-000|1> sending retransmit 4 of request message ID 951205933, seq 4

The only solution in this case is to disable all IPsec NAT entries, stop ipsec and restart it. Afterwards all phase 2 tunnels will come up immediately. Once all phase 2 tunnels are established, it is possible to enable the IPsec NAT entries again (but this is dangerous because a reconnect of the tunnel is very unlikely to succeed).

We do not have this issue with IPsec NAT disabled.

While debugging #369 with @AdSchellevis we've been wondering why entries to ipsec.conf are added for IPsec NAT. Is this actually required for IPsec NAT to work? It seems that this confuses the ipsec daemon.

To be more precise: I do not use the IPsec NAT for masquerading my local networks, but instead to do actual NAT (allow my other local networks to access a remote IPsec network, even if the remote side does know nothing about these local networks). It's exactly what is described here (see "Remote End Notes").

[AdSchellevis]

@fraenki just checking, but if I'm not mistaken it functions normal if the nat entry is an actual tunnel, but if you want to nat some parts of your network, you can create entries over here which will never be active and may confuse strongswan sometimes, but will do the nat part, because it has nothing to do with ipsec.

The problem over here seems to be, that we can't configure these "do not NAT all my traffic tunnels" in the nat section of the firewall.

Am I right?

[fraenki]

@AdSchellevis I'm not sure what you mean with "...if the nat entry is an actual tunnel...".

On the other hand, yes, the problem seems to be that we're not able to create similar NAT rules elsewhere. Currently it's only possible on the IPsec phase 2 configuration page.
I've already tried to replace this "IPsec phase 2 NAT" entry with a regular "outbound NAT rule", but this didn't work. I'm not sure what magic was missing to make it work. Maybe it's related to #438?

[fraenki]

Just "confirmed" this issue with a second tunnel, FWIW. Disabling the IPsec phase 2 NAT entries immediately solved the connection issue.

[AdSchellevis]

That's good, and yes it's related to the other issue. You can only create NAT rules for tunnels you actually have at the moment, and there doesn't seem to be a logic reason for it.
When the rules are disabled, your situation should work (because it seems to keep the rules, #439), but eventually you need to be able to create outbound nat rules for this.

[fraenki]

OK, let me summarize what I've tried so far.

I have the following IPsec phase 2 tunnel configured:

• Local network: 192.168.1.0/24
• Remote network: 10.22.13.0/24

But I want to be able to reach the remote network from my second local network, 172.16.2.0/24, too. That's why I've added the following IPsec phase 2 NAT configuration (documentation/description):

• Local Network: 172.16.2.0/24
• NAT/BINAT Network: 192.168.1.0/24
• Remote Network: 10.22.13.0/24

This should allow us to access the remote network 10.22.13.0/24 from our second local network 172.16.2.0/24, because it gets NAT to 192.168.1.0/24. You see that a pf NAT rule is added:

binat on enc0 inet from 172.16.2.0/24 to 10.22.13.0/24 -> 192.168.1.0/24

And it works. But the ipsec daemons gets confused, as described earlier.

That's why I've tried the following:

• disabled the IPsec phase 2 NAT configuration
• the pf BINAT rule was still there
• added a regular outbound NAT rule (Firewall->NAT->Outbound)

Now the pf NAT table looks like this:

nat on enc0 inet from 172.16.2.0/24 to 10.22.13.0/24 -> 192.168.1.0/24 port 1024:65535
binat on enc0 inet from 172.16.2.0/24 to 10.22.13.0/24 -> 192.168.1.0/24

Unfortunately, the connection from 172.16.2.0/24 to 10.22.13.0/24 stopped working immediately after disabling the IPsec phase 2 NAT configuration. I guess the regular outbound NAT rule and the BINAT rule aren't enough to get it working. (FWIW, actually BINAT isn't really what I want, but it is OK for demonstration purposes.)

Is there any way to configure this IPsec NAT without interferring with the ipsec daemon?

[fraenki]

@AdSchellevis I've seen some (rather old) posts stating that PF can't do NAT for IPsec (at least not on FreeBSD). Is this statement still true? On the other hand some people advise to set net.inet.ipsec.filtertunnel=1 in this regard... do you have any comments on this to share? :)

[fraenki]

Setting the sysctl net.inet.ipsec.filtertunnel to 1 did not help to solve the NAT issue, but it would allow us to filter the traffic inside the tunnel (as the name suggests).

[fichtner]

@fraenki what's the status here?

[fraenki]

@fichtner Currently it cannot be solved so easily. I think I should open a new RFE issue with only details on the technical limitations/issues.

[fichtner]

We can update the first post if needed, just let me know how to transform the ticket (title+description). would be sad to lose the extra info in the comments.

[fraenki]

@fichtner I've updated my original submission with the description of the underlying problem.

[fichtner]

@fraenki thank you :)

[fraenki]

FWIW, the quick fix ("patch strongSwan, revive some old code") is still working on 16.1.3.

[fraenki]

more discussion on this topic: https://forum.opnsense.org/index.php?topic=3696.0

[fraenki]

So it seems that the described fix no longer works on 17.1.x: even with the patches, the traffic does enter the enc0 interface without any NAT rule being applied. :(
Any idea which change might have caused this breakage?

We had to rollback from 17.1 to 16.7, now "NAT before IPsec" is working again.
We won't be able to upgrade until this is fixed.

[fichtner]

enc(4) was reworked for 11.0, which is probably why this happens. pfSense has patches that may be related, but I don't want to review things that are not in FreeBSD. IPsec is complicated as is :(

[ermal]

Guys not sure why i get notifications from here but can i be removed?

[fichtner]

@ermal there's an unsubscribe button at the top right next to the OP. github mentions are working more sturdily now than they used to ;)

[fichtner]

@fraenki I tried forcing NATon IPsec device with local address via traffic from a different network, then added a gateway (code edits) to force the packet down the tunnel device, which seems to go through and the other end answers but the return path "hangs" when handing the payload back to the original network. this may be workable still. need to look into it more :)

[fraenki]

@fichtner Thanks for taking the time to search for a solution. It's very unfortunate that this old issue now turns into a show stopper for us. :( I was OK maintaining our custom package, but non-working "NAT before IPsec" would kill many services on our side.

[ermal]

@fichtner thanks for the tip i always miss that button.
While here i might give you insight into it.
With FreeBSD 10 the best way is to use the patches i made since the other way of using nat+rdr is very tricky.
With FreeBSD 11-stable you have to be smart and use if_ipsec with routing and send needed traffic to that interface, nat there as usual with any other interface.
The only complication is that you have to manager routes with it.

Have fun and hope it helps.

[AdSchellevis]

@mimugmail 9351e45 and 814d18a add the first version to add spd entries manually. it still needs a cleanup to record changes, but the rough idea is there.

[mimugmail]

@AdSchellevis Works great, thanks! 👍

[AdSchellevis]

@mimugmail nice team effort this is!
So, cleanup is in too 1fe8341

In case @fraenki wants to test too, you need to pull in the following changes (in this order):
9351e45
814d18a
1fe8341
76839db

Last item on my list is to extend the one-on-one nat and then we should close this issue in my opinion.

[mimugmail]

I applied all patches and will test when I'm in the office.

I think @fraenki first has to update from 16.1 to 17.1 (or .7) for testing it :)

[fichtner]

I'll add this to 17.7.1 if you all agree, as it doesn't seem to interfere with previous settings.

[mimugmail]

One-to-one NAT works also with different sized networks. Will there be further changes to the GUI? Then I'll document the steps and @jschellevis can import it to the docs.

As far as I can see this should also allow IPSEC with overlapping networks on both sides, will test this with a second lab ...

[AdSchellevis]

@mimugmail I think we're done, I don't have anything else planned there.

[AdSchellevis]

@fichtner 17.7.1 sound good to me, thanks!

[mimugmail]

@fichtner Would you mind to name @bu7cher (Andrey) in the Changelog as he did most of the research (it's ok for him)?

[fichtner]

I'll mention in the backport, for the changelog the credits only name code contributions itself.

[fraenki]

I can confirm that this NAT-before-IPsec implementation is indeed working for me. Great job, @AdSchellevis and @mimugmail!

There's one limitation that should be mentioned in the help text (and documentation):

root@opnsense:~ # /sbin/setkey -f /tmp/setkeyAsL4tW
libipsec: invalid IP address while parsing "host.example.com"
line 1: hostname nor servname provided, or not known at [ out ipsec esp/tunnel/host.example.com-1.2.3.4/require]
parse failed, line 1.

This means that it's required to use an IP address for "My identifier". Anything else, for example "Distinguished name", will not work. (Well, maybe it might work to get the IP address from the selected interface for those cases... but I'm not sure.)

[AdSchellevis]

@fraenki we can hide the option when not valid (! my ip or ip) or add a validation, what do you think?

[fraenki]

@AdSchellevis I've opened issue #1773, because I've found another limitation and a possible fix.

[fichtner]

Forum thread https://forum.opnsense.org/index.php?topic=7282.0 asks for official docs

[mimugmail]

I'll take this one ... long overdue :)