Seeing "No such file or directory" for CIS benchmark 1.1.9 using K8s 1.2.1 #867
Assignees
Labels
Comments
|
I'm not sure on what is the difference if you say that running the command manually returns nothing but running it with kube-bench returns file not exist (maybe some output redirections? something like 2> /dev/null) anyway, I might have a solution for you, I think that if you change it to be: This should fix this issue without hurting the test integrity, since we either checking if the is permissions then is should be 644 or more restrictive but if it doesn't have permissions in the output at all it still should be fine and pass because it's not less restrictive then 644 it simply doesn't exist. |
|
Thanks @yoavrotems will have a play, and report back ... as you say, if it works, I'll open a PR 👍 |
|
Hey :) anything new? @davidhay1969 |
|
Apologies @yoavrotems got side-tracked, and forgot to test - on it now, watch this space ...... |
|
Alas, I suspect I'm doing it wrong, but still seeing the same warning for - id: 1.1.9
text: "Ensure that the Container Network Interface file permissions are set to 644 or more restrictive (Manual)"
audit: |
ps -ef | grep $kubeletbin | grep -- --cni-conf-dir | sed 's%.*cni-conf-dir[= ]\([^ ]*\).*%\1%' | xargs -I{} find {} -mindepth 1 | xargs stat -c permissions=%a
find /var/lib/cni/networks -type f | xargs --no-run-if-empty stat -c permissions=%a
use_multiple_values: true
tests:
bin_op: or
test_items:
- flag: "permissions"
compare:
op: bitmask
value: "644"
- flag: "permissions"
set: false
remediation: |
Run the below command (based on the file location on your system) on the master node.
For example,
chmod 644 <path/to/cni/files>
scored: falseI turned on verbose logging: - and can see: - in the pod logs .... Looking further now .... |
|
Again, running the commands manually against the Control Plane Node works as expected: -
|
|
Wondering whether the code "assumes" that a single Bash command will be included in the audit: |
ps -ef | grep $kubeletbin | grep -- --cni-conf-dir | sed 's%.*cni-conf-dir[= ]\([^ ]*\).*%\1%' | xargs -I{} find {} -mindepth 1 | xargs stat -c permissions=%a
find /var/lib/cni/networks -type f | xargs --no-run-if-empty stat -c permissions=%aLooking at the func runAudit(audit string) (output string, err error) {
var out bytes.Buffer
audit = strings.TrimSpace(audit)
if len(audit) == 0 {
return output, err
}
cmd := exec.Command("/bin/sh")
cmd.Stdin = strings.NewReader(audit)
cmd.Stdout = &out
cmd.Stderr = &out
err = cmd.Run()
output = out.String()
if err != nil {
err = fmt.Errorf("failed to run: %q, output: %q, error: %s", audit, output, err)
} else {
glog.V(3).Infof("Command: %q", audit)
glog.V(3).Infof("Output:\n %q", output)
}
return output, err
}😖 😕 |
|
There isn't suppose to be problem with multiple line audit we use it in other places and it works. |
|
Ahhhh, now that looks much better. Having changed from: - - id: 1.1.9
text: "Ensure that the Container Network Interface file permissions are set to 644 or more restrictive (Manual)"
audit: |
ps -ef | grep $kubeletbin | grep -- --cni-conf-dir | sed 's%.*cni-conf-dir[= ]\([^ ]*\).*%\1%' | xargs -I{} find {} -mindepth 1 | xargs stat -c permissions=%a
find /var/lib/cni/networks -type f | xargs --no-run-if-empty stat -c permissions=%a
use_multiple_values: true
tests:
test_items:
- flag: "permissions"
compare:
op: bitmask
value: "644"
remediation: |
Run the below command (based on the file location on your system) on the master node.
For example,
chmod 644 <path/to/cni/files>
scored: falseto: - - id: 1.1.9
text: "Ensure that the Container Network Interface file permissions are set to 644 or more restrictive (Manual)"
audit: |
ps -ef | grep $kubeletbin | grep -- --cni-conf-dir | sed 's%.*cni-conf-dir[= ]\([^ ]*\).*%\1%' | xargs -I{} find {} -mindepth 1 | xargs stat -c permissions=%a
find /var/lib/cni/networks -type f 2> /dev/null | xargs --no-run-if-empty stat -c permissions=%a
use_multiple_values: true
tests:
test_items:
- flag: "permissions"
compare:
op: bitmask
value: "644"
remediation: |
Run the below command (based on the file location on your system) on the master node.
For example,
chmod 644 <path/to/cni/files>
scored: falseI now see: - running and I no longer see a warning for Will do the same for Thanks @yoavrotems you're a Jedi! |
|
Yep, |
|
So, TL;DR;, your change "merely" adds in some output redirection for the diff --git a/cfg/cis-1.6/master.yaml b/cfg/cis-1.6/master.yaml
index 726df72..0b21cf6 100644
--- a/cfg/cis-1.6/master.yaml
+++ b/cfg/cis-1.6/master.yaml
@@ -122,7 +122,7 @@ groups:
text: "Ensure that the Container Network Interface file permissions are set to 644 or more restrictive (Manual)"
audit: |
ps -ef | grep $kubeletbin | grep -- --cni-conf-dir | sed 's%.*cni-conf-dir[= ]\([^ ]*\).*%\1%' | xargs -I{} find {} -mindepth 1 | xargs stat -c permissions=%a
- find /var/lib/cni/networks -type f | xargs --no-run-if-empty stat -c permissions=%a
+ find /var/lib/cni/networks -type f 2> /dev/null | xargs --no-run-if-empty stat -c permissions=%a
use_multiple_values: true
tests:
test_items:
@@ -140,7 +140,7 @@ groups:
text: "Ensure that the Container Network Interface file ownership is set to root:root (Manual)"
audit: |
ps -ef | grep $kubeletbin | grep -- --cni-conf-dir | sed 's%.*cni-conf-dir[= ]\([^ ]*\).*%\1%' | xargs -I{} find {} -mindepth 1 | xargs stat -c %U:%G
- find /var/lib/cni/networks -type f | xargs --no-run-if-empty stat -c %U:%G
+ find /var/lib/cni/networks -type f 2> /dev/null | xargs --no-run-if-empty stat -c %U:%G
use_multiple_values: true
tests:
test_items:which is awesome. Thanks again, @yoavrotems - do you want to go ahead and raise a PR ? Otherwise, more than happy to do that, giving you ALL THE CREDIT, and ask you to review it ? |
|
No credit needed but yes it will be awesome of you to PR it :) ! @davidhay1969 |
|
👍 will do it when I get home later this PM. Cheers 🥂 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Overview
I'm seeing: -
as part of the output from
kube-benchfor CIS test1.1.9( and also for1.1.10).This is being run against the K8s Master ( Control Plane ) node.
More specifically, this is the entire output from that particular run: -
Looking at master.yaml for the CIS
1.6tests, I can see the commands being run: -and it appears to be the second
findcommand which is borking 😢Looking at my K8s Master node, whilst I DO have the
/var/lib/cni/networkssubdirectory, it actually contains no content.However, the command, if run manually on the node itself, does work: -
find /var/lib/cni/networks -type f | xargs --no-run-if-empty stat -c permissions=%aalbeit returning nothing because, of course, the subdirectory is empty.
Therefore, I'm at a loss to know why
kube-benchis instead returning "No such file or directory" rather than nothing ( which, by my reading of the YAML, would be OK )What am I missing ?
How did you run kube-bench?
I'm running
kube-benchvia the job-master.yaml slightly modified to add logging: -rather than the default of: -
( note that I'm also skipping some tests )
I'm also using my own build of the
kube-benchimage, from the Dockerfile, as I needed to build it for my target IBM Zs390xplatform.What happened?
What did you expect to happen:
Both tests should pass i.e. not throw a WARN, as the preceding commands both run happily, when run manually: -
## Set kubeletbin variable
export kubeletbin="kubelet"## Check permissions
ps -ef | grep $kubeletbin | grep -- --cni-conf-dir | sed 's%.*cni-conf-dir[= ]\([^ ]*\).*%\1%' | xargs -I{} find {} -mindepth 1 | xargs stat -c permissions=%a## Check ownership
ps -ef | grep $kubeletbin | grep -- --cni-conf-dir | sed 's%.*cni-conf-dir[= ]\([^ ]*\).*%\1%' | xargs -I{} find {} -mindepth 1 | xargs stat -c %U:%GEnvironment
kube-benchimage built from Dockerfilekube-bench-masterjob run from slightly amended job-master.yamlkubectl versionRunning processes
ps -eaf | grep kubeConfiguration files
See above
Anything else you would like to add:
Nope, but I suspect it's pilot error on my part 😭
The text was updated successfully, but these errors were encountered: