RFE: add NETFILTER_PKT tests

[rgbriggs]

Test for simplified normalized NETFILTER_PKT audit message.
Check for receipt of each nfmarked packet and for correct number of fields.

See: linux-audit/audit-kernel#11
Signed-off-by: Richard Guy Briggs rgb@redhat.com

[pcmoore]

Added one comment about checking all the fields, otherwise this looks good.

[rgbriggs]

@pcmoore @The-Mule Can we make sure this ipv6 icmp trigger command fixup gets added too?

[pcmoore]

I haven't really reviewed these last changes in depth, but yes, we should add the ICMPv6 test. I'll probably just squash all three commits into one during the merge.

[pcmoore]

When I ran this test just now I noticed that it is a bit noisy and doesn't clean up after itself:

# uname -r
4.11.0-0.rc7.git0.1.1.secnext.fc27.x86_64
# rpm -q audit
audit-2.7.5-1.fc27.x86_64
# ./test
1..13
# Running under perl version 5.024001 for linux
# Current time local: Tue Apr 18 17:35:16 2017
# Current time GMT:   Tue Apr 18 21:35:16 2017
# Using Test.pm version 1.28_01
ok 1
ok 2
ok 3
ok 4
ok 5
ok 6
ok 7
ok 8
ok 9
ok 10
ok 11
ok 12
ok 13
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
ip6tables: No chain/target/match by that name.
ip6tables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
ip6tables: No chain/target/match by that name.
ip6tables: No chain/target/match by that name.

My system's iptables configuration before running the tests:

# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
# ip6tables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

... and after the tests:

# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
AUDIT      icmp --  anywhere             anywhere             icmp echo-request AUDIT accept

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
# ip6tables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
AUDIT      tcp      anywhere             anywhere             tcp dpt:42424 AUDIT accept
AUDIT      ipv6-icmp    anywhere             anywhere             ipv6-icmp echo-request AUDIT accept

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination  

[rgbriggs]

I need more information about why it is failing.

Can you also list the mangle tables?
iptables -t mangle -L
ip6tables -t mangle -L
I expect to see another 4 unremoved rules there since there are 7 complaints but only 3 listed above. There are 3 rules for each combination of ipv4/ipv6 and filter/mangle for a total of 12. The test uses default chains, so using a missing chain is not possible.

On my rawhide VM (recent audit-next 4.8 kernel and audit-2.7.3-1.fc26.x86_64) and RHEL7 VM (3.10.0-643.el7.test.x86_64 and audit-2.6.5-3.el7.x86_64) both are clean with no ip*tables evidence left behind. As suggested by Ondrej, I run the test with:

cd git/audit-testsuite/; make clean; HARNESS_VERBOSE=1 HARNESS_OPTIONS="c" SUBDIRS="netfilter_pkt" make -e test
Since all the verifications succeed it must be ip*tables rule deletions that are failing, since if any rules failed to insert, those verifications should have failed. They use the same commands to insert and to delete, so why are only some of the deletions failing rather than all or none?

Can you issue lsmod? (Are xt_AUDIT and xt_mark loading? They must be or the verifications should have failed.)

[pcmoore]

After spending some time on this today it appears to be a kernel issue with the netfilter ICMP type match. @rgbriggs it sounds like you are running an older Rawhide kernel, can you verify this with a newer kernel on your system?

Test using the latest pcmoore/kernel-secnext kernel and a rule that specifies an ICMP type:

NOTE: similar results were obtained with the vanilla 4.11.0-0.rc7.git3.1.fc27.x86_64 Rawhide kernel.

# uname -r
4.11.0-0.rc7.git0.1.1.secnext.fc27.x86_64
# rpm -qa iptables
iptables-1.6.1-2.fc26.x86_64
# iptables -t mangle -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
# iptables -t mangle -A INPUT -i lo -p icmp --icmp-type 8 -j MARK --set-mark 0xdeadbeef
# iptables -t mangle -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-A INPUT -i lo -p icmp -m icmp --icmp-type 8 -j MARK --set-xmark 0xdeadbeef/0xffffffff
# iptables -t mangle -D INPUT -i lo -p icmp --icmp-type 8 -j MARK --set-mark 0xdeadbeef
iptables: No chain/target/match by that name.

Test using the pcmoore/kernel-secnext kernel and a rule that DOES NOT specify an ICMP type:

NOTE: similar results were obtained with the vanilla 4.11.0-0.rc7.git3.1.fc27.x86_64 Rawhide kernel.

# uname -r
4.11.0-0.rc7.git0.1.1.secnext.fc27.x86_64
# rpm -qa iptables
iptables-1.6.1-2.fc26.x86_64
# iptables -t mangle -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
# iptables -t mangle -A INPUT -i lo -p icmp -j MARK --set-mark 0xdeadbeef
# iptables -t mangle -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-A INPUT -i lo -p icmp -j MARK --set-xmark 0xdeadbeef/0xffffffff
# iptables -t mangle -D INPUT -i lo -p icmp -j MARK --set-mark 0xdeadbeef
# iptables -t mangle -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT

Test using the latest F25 kernel:

# uname -r
4.10.10-200.fc25.x86_64
# rpm -q iptables
iptables-1.6.1-2.fc26.x86_64
# iptables -t mangle -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
# iptables -t mangle -A INPUT -i lo -p icmp --icmp-type 8 -j MARK --set-mark 0xdeadbeef
# iptables -t mangle -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-A INPUT -i lo -p icmp -m icmp --icmp-type 8 -j MARK --set-xmark 0xdeadbeef/0xffffffff
# iptables -t mangle -D INPUT -i lo -p icmp --icmp-type 8 -j MARK --set-mark 0xdeadbeef
# iptables -t mangle -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT

[rgbriggs]

On 2017-04-24 14:48, Paul Moore wrote: After spending some time on this today it appears to be a kernel issue with the netfilter ICMP type match. @rgbriggs it sounds like you are running an older Rawhide kernel, can you verify this with a newer kernel on your system?

:-P I was running your 4.8 audit-next tree.

#### Test using the latest pcmoore/kernel-secnext kernel and a rule that specifies an ICMP type: # uname -r 4.11.0-0.rc7.git0.1.1.secnext.fc27.x86_64

# uname -r 4.11.0-rc3-ghak__-unswing-cap-fields+

# rpm -qa iptables iptables-1.6.1-2.fc26.x86_64

# rpm -qa iptables iptables-1.6.0-3.fc26.x86_64

# iptables -t mangle -S -P INPUT ACCEPT

Mine is much more polluted, but the target is there.

# iptables -t mangle -A INPUT -i lo -p icmp --icmp-type 8 -j MARK --set-mark 0xdeadbeef # iptables -t mangle -S -P INPUT ACCEPT -A INPUT -i lo -p icmp -m icmp --icmp-type 8 -j MARK --set-xmark 0xdeadbeef/0xffffffff

I get the same.

# iptables -t mangle -D INPUT -i lo -p icmp --icmp-type 8 -j MARK --set-mark 0xdeadbeef iptables: No chain/target/match by that name.

I confirm the same on 4.11, but not on 4.8.

#### Test using the pcmoore/kernel-secnext kernel and a rule that DOES NOT specify an ICMP type: # iptables -t mangle -S -P INPUT ACCEPT # iptables -t mangle -A INPUT -i lo -p icmp -j MARK --set-mark 0xdeadbeef # iptables -t mangle -S -P INPUT ACCEPT -A INPUT -i lo -p icmp -j MARK --set-xmark 0xdeadbeef/0xffffffff # iptables -t mangle -D INPUT -i lo -p icmp -j MARK --set-mark 0xdeadbeef # iptables -t mangle -S -P INPUT ACCEPT #### Test using the latest F25 kernel: # uname -r 4.10.10-200.fc25.x86_64 # iptables -t mangle -S -P INPUT ACCEPT # iptables -t mangle -A INPUT -i lo -p icmp --icmp-type 8 -j MARK --set-mark 0xdeadbeef # iptables -t mangle -S -P INPUT ACCEPT -A INPUT -i lo -p icmp -m icmp --icmp-type 8 -j MARK --set-xmark 0xdeadbeef/0xffffffff # iptables -t mangle -D INPUT -i lo -p icmp --icmp-type 8 -j MARK --set-mark 0xdeadbeef # iptables -t mangle -S -P INPUT ACCEPT

Ok, so this looks like a kernel bug or maybe an API change (which Linus will declare a bug) in 4.11, since the iptables package hasn't changed. - RGB

…

-- Richard Guy Briggs <rgb@redhat.com> Sr. S/W Engineer, Kernel Security, Base Operating Systems Remote, Ottawa, Red Hat Canada IRC: rgb, SunRaycer Voice: +1.647.777.2635, Internal: (81) 32635

[pcmoore]

:-P I was running your 4.8 audit-next tree.

Then you must be using an old audit/next branch; the audit/next branch has been based against 4.11-rc3 for several weeks now. Regardless, you saw I was testing with a 4.11-rcX based kernel, I'm surprised you didn't try reproducing that (I first reported problems about a week ago).

For reference, the kernel-secnext kernels have the audit/next, audit/stable-<latest>, selinux/next, and selinux/stable-<latest> branches merged on top of the latest Rawhide kernels. This is what I use for my weekly testing, and what your patches/PRs will likely need to pass to get merged.

• https://copr.fedorainfracloud.org/coprs/pcmoore/kernel-secnext
• http://www.paul-moore.com/docs/flock-kernel_testing_bleeding_edge-pmoore-072016-r2.pdf

I confirm the same on 4.11, but not on 4.8.

Good, that is what I expected. My F25 tests were on a 4.10 kernel so I would expect 4.8 to work (not a 100% given, but likely).

Ok, so this looks like a kernel bug or maybe an API change (which Linus will declare a bug) in 4.11, since the iptables package hasn't changed.

Any sane kernel developer is going to declare this a bug, that shouldn't be an issue. Regardless, it does impact this test and I see two paths forward for this test:

1. Debug the kernel issue and submit a fix upstream.
2. Rewrite the test to using something other than ICMP type matching in the netfilter rules.

... if you want to take option 2, let me know so I can report this bug to netdev.

[rgbriggs]

On 2017-04-25 05:05, Paul Moore wrote: > :-P I was running your 4.8 audit-next tree. Then you must be using an old audit/next branch; the audit/next branch has been based against 4.11-rc3 for several weeks now. Regardless, you saw I was testing with a 4.11-rcX based kernel, I'm surprised you didn't try reproducing that (I first reported problems about a week ago).

That patch was upstreamed before you jumped to v4.11-rcX. I simply re-tested on that. Given the symptoms, I honestly had trouble believing that it was a very new/recent kernel bug that affected only one option of an iptables command, and with the initial report, it didn't appear that there was enough information to track it down yet without more detailed output.

For reference, the kernel-secnext kernels have the audit/next, audit/stable-\<latest\>, selinux/next, and selinux/stable-\<latest\> branches merged on top of the latest Rawhide kernels. This is what I use for my weekly testing, and what your patches/PRs will likely need to pass to get merged. * https://copr.fedorainfracloud.org/coprs/pcmoore/kernel-secnext * http://www.paul-moore.com/docs/flock-kernel_testing_bleeding_edge-pmoore-072016-r2.pdf > I confirm the same on 4.11, but not on 4.8. Good, that is what I expected. My F25 tests were on a 4.10 kernel so I would expect 4.8 to work (not a 100% given, but likely). > Ok, so this looks like a kernel bug or maybe an API change (which > Linus will declare a bug) in 4.11, since the iptables package hasn't > changed. Any sane kernel developer is going to declare this a bug, that shouldn't be an issue. Regardless, it does impact this test and I see two paths forward for this test: > 1. Debug the kernel issue and submit a fix upstream. 2. Rewrite the test to using something other than ICMP type matching in the netfilter rules. ... if you want to take option 2, let me know so I can report this bug to netdev.

If this test had been merged when the upstream code that depended on it passing had been merged, then we would have gotten to this stage and either fixed the bug and submitted a patch upstream or reported it upstream and not needed to work around it. I'd still say that ICMP test is valid and needed since I wanted to test both control and data variants of ipv4 and ipv6 packets. I'll have a look to see if I can track down this bug. If I end up spending too much time on it, I'll punt upstream.

#43 (comment)

- RGB

[pcmoore]

That patch was upstreamed before you jumped to v4.11-rcX. I simply
re-tested on that. Given the symptoms, I honestly had trouble believing
that it was a very new/recent kernel bug that affected only one option
of an iptables command, and with the initial report, it didn't appear
that there was enough information to track it down yet without more
detailed output.

The point remains that I reported problems using a different kernel than you had tested on, which is okay given when you submitted this PR, but instead of trying a more recent kernel you simply kicked it back. It seems like trying to reproduce this on the reported kernel (or at least any v4.11 kernel) would be a very reasonable thing to do, especially since all it would have taken would be a "dnf update".

Instead I spent several hours debugging the test only to find that this is easily reproduced. That makes me grumpy. Reporting my findings here and seeing you respond back the way you have makes me even more grumpy. As a FYI, making maintainers grumpy is not a good way to get your code accepted.

If this test had been merged when the upstream code that depended on it
passing had been merged, then we would have gotten to this stage and
either fixed the bug and submitted a patch upstream or reported it
upstream and not needed to work around it.

The review loop for this PR has been going on during the v4.11 cycle, the kernel code is still in audit/next. If you would prefer I can pull your NETFILTER_PKT patches from audit/next?

Perhaps in the future I need to be stricter about the dependencies between the kernel patches and the tests so that the kernel patches only go in once the tests have been given the "ok".

I'd still say that ICMP test is valid and needed since I wanted to test
both control and data variants of ipv4 and ipv6 packets.

I probably wasn't clear in my previous comments. Yes, I agree, the ICMP tests are valid, however, I'm not going to merge a test with known failures, even if those failures are likely due to unrelated issues/bugs. That is why I suggested reworking the test as one path forward; it isn't my preferred choice, but it is a choice.

I'll have a look to see if I can track down this bug. If I end up
spending too much time on it, I'll punt upstream.

Okay, I won't bother reporting this to netdev then, you can share the investigation in this thread/PR with them.

[pcmoore]

Upstream fix: https://marc.info/?t=149436126600003&r=1&w=2

[pcmoore]

The upstream fix is now in Linus' tree so merged via 82e6551