Kernel crash in Linux 3.12 #90

Closed
ydahhrk opened this Issue Apr 16, 2014 · 3 comments

Comments

Projects
None yet
1 participant
@ydahhrk
Member

ydahhrk commented Apr 16, 2014

Long story short:

We cannot use the kernel's icmpv6_send() function in kernel 3.12 because it causes a kernel crash (at least when called from prerouting). As far as I can tell, this is a bug in the kernel, and the only viable workaround is to not generate ICMPv6 errors when Jool is compiled in 3.12.

The RFCs say that these errors SHOULD be sent. "SHOULD" is defined as

SHOULD (...) means that there may exist valid reasons in particular
circumstances to ignore a particular item, but the full implications must
be understood and carefully weighed before choosing a different course.

It looks like we have a good reason to not do it. However, in order to minimize the damage, we should encourage users to not use Jool in kernel 3.12.

I need to clarify: After the commit I'm about to upload, Jool will no longer crash when it tries to send ICMPv6 errors in Linux 3.12; it will simply not send them. Using Jool in kernel 3.12 is not ideal, but it's also not the end of the world.


Long story long (copied from my mail to Philar):

(When you read the following text, keep in mind that "setting the packet's dst_entry" and "routing the packet" are the same thing.)

In kernel 3.12, they're dereferencing the packet's dst_entry without checking whether it's NULL first.
It seems that they realized the problem with this and added a NULL check in kernel 3.13. I guess we'll never know why they never fixed it in kernel 3.12.

It can be seen fairly clearly when one compares the different versions of the _decode_session6() function side-to-side:

http://lxr.free-electrons.com/source/net/ipv6/xfrm6_policy.c?v=3.11#L129
http://lxr.free-electrons.com/source/net/ipv6/xfrm6_policy.c?v=3.12#L129
http://lxr.free-electrons.com/source/net/ipv6/xfrm6_policy.c?v=3.13#L129

The way I see it, this is a bug in kernel 3.12 and not in Jool itself, and fixing it is either impossible or will require inordinate amounts of effort because of the following reasons:

  • Setting the packet's dst_entry doesn't seem to be a possibility because the function from the kernel that handles that (ip6_route_input()) is not available for loadable modules such as Jool (http://oss.sgi.com/archives/netdev/2001-06/msg00317.html). However, the kernel doesn't have a route towards the NAT64 prefix anyway, so even if we could use ip6_route_input() we'd still probably be unable to set the dst_entry.
  • Rolling out our own icmp6_send() function is also very troublesome because the kernel also doesn't export the ICMP error rate limiting functions :(.
  • We could move Jool to Netfilter's FORWARD hook (where the kernel has already routed the packet), but then we would have to ask users for additional configuration (because the kernel will need to know the NAT64 IPv6 prefix while routing). And we can't predict what other problems we'd have.

This is very annoying, because NAT64 isn't compatible with IPSec in the first place, so the the XFRM code shouldn't be getting in Jool's way >:(.

Apparently, our least troublesome solution is to simply avoid the calls to icmp6_send() when the module has been compiled in kernel 3.12. This will solve the kernel panic, but the ICMP error messages will still not be sent. Because the RFCs say that these errors SHOULD be sent, we need to recommend users to not use Jool in Linux 3.12.

@ydahhrk ydahhrk added this to the 3.1.4 milestone Apr 16, 2014

@ydahhrk ydahhrk self-assigned this Apr 16, 2014

ydahhrk added a commit that referenced this issue Apr 16, 2014

Workaround for issue #90.
Jool will no longer craft ICMPv6 errors in Linux 3.12. This is bad, but the only known alternative is a kernel crash.

@ydahhrk ydahhrk changed the title from Kernel crash in kernel 3.12 to Kernel crash in Linux 3.12 Apr 16, 2014

@ydahhrk

This comment has been minimized.

Show comment
Hide comment
@ydahhrk

ydahhrk Apr 16, 2014

Member

The fix is in the master branch now; closing.

Member

ydahhrk commented Apr 16, 2014

The fix is in the master branch now; closing.

@ydahhrk

This comment has been minimized.

Show comment
Hide comment
@ydahhrk

ydahhrk Apr 25, 2014

Member

Actually, it seems they have every intention of fixing this in 3.12, but for some reason it hasn't been merged.

https://lkml.org/lkml/2013/11/23/133

Member

ydahhrk commented Apr 25, 2014

Actually, it seems they have every intention of fixing this in 3.12, but for some reason it hasn't been merged.

https://lkml.org/lkml/2013/11/23/133

@ydahhrk

This comment has been minimized.

Show comment
Hide comment
@ydahhrk

ydahhrk May 22, 2014

Member

I forgot to mention:

This only affects ICMPv6 errors generated by Jool.

If a IPv4 node generates a ICMPv4 error and Jool translates it, the resulting ICMPv6 message is forwarded troublelessly.

Why? Translated errors aren't rate-limited, so they don't need to use icmpv6_send().

Member

ydahhrk commented May 22, 2014

I forgot to mention:

This only affects ICMPv6 errors generated by Jool.

If a IPv4 node generates a ICMPv4 error and Jool translates it, the resulting ICMPv6 message is forwarded troublelessly.

Why? Translated errors aren't rate-limited, so they don't need to use icmpv6_send().

ydahhrk added a commit that referenced this issue Dec 11, 2014

Moving Jool from Prerouting to Local In, Local Out and Forwarding.
This is necessary so NAT64 happens after iptables does filtering.
It's also needed so Jool catches local traffic, which is needed by local CLATs.
As an added bonus, it invalidates issue #90. Woot!

Progress so far, summary:
- Issue #33: Done.
- Issue #41: Done.
- Issue #107: Done.
- Issue #111: dhfelix is done, but haven't even started to review.
- Issue #116: EAM done, moved from prerouting done, dummy interface done. Missing (off the top of my head):
	- Adapting the global packet processing pipeline for stateless mode.
	- Configuration options.
	- Review RFC 6145 and updaters.
- Issue #120: Done.
- Issue #121: Not done.

Everything needs testing. There are known bugs with fragmentation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment