feat: (recommend) Implement recommend functionality for Docker Client#461
Merged
Aryan-sharma11 merged 2 commits intokubearmor:mainfrom Jul 22, 2025
Merged
Conversation
Contributor
Author
|
@daemon1024 please review |
Member
|
Can you include screenshots of how it's working? |
Contributor
Author
|
Case 1: systemd mode
Case 2: k8s mode There is an error |
Member
rootxrishabh
left a comment
rootxrishabh
left a comment
There was a problem hiding this comment.
Initial review - Changes are working as expected.
Member
|
Output directory is Is this intended? |
Contributor
Author
shouldn't this be ill check what is the issue with output directory name |
Contributor
Author
|
@rootxrishabh policy directory is fixed. it was due to |
Contributor
Author
|
@rootxrishabh also added commit to trim new line character in final report generation. it fixes the weird number of blank lines after reports table |
rootxrishabh
requested changes
Jan 2, 2025
Member
rootxrishabh
left a comment
rootxrishabh
left a comment
There was a problem hiding this comment.
karmor crashes when using --k8s and -i together. @tesla59 PTAL
Member
|
karmor exists with an error. With accuknox/kubearmor-client - (tesla/non-k8s/karmor-recommend) > docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
45eb1cf51480 nginx "/docker-entrypoint.…" 20 minutes ago Up 20 minutes 80/tcp modest_pascal
accuknox/kubearmor-client - (tesla/non-k8s/karmor-recommend) > ./karmor recommend -k=false
ERRO[0000] no Object found to secure, hence nothing to recommend! namespace= |
Contributor
Author
|
hey @rootxrishabh When running When running I think the issue could be due to different envs, although unlikely. link to debug branch |
Member
There was a problem hiding this comment.
Tested karmor recommend --k8s=false -i nginx seems to work now. I am not able to reproduce the previous error.
accuknox/kubearmor-client - (tesla/non-k8s/karmor-recommend) > ./karmor recommend --k8s=false -i nginx
INFO[0000] Found outdated version of policy-templates Current Version=v0.2.3
INFO[0000] Downloading latest version [v0.2.6]
INFO[0001] policy-templates updated Updated Version=v0.2.6
INFO[0001] pulling image image=nginx
latest: Pulling from library/nginx
7ce705000c39: Pull complete
b3e9225c8fca: Pull complete
2b39a3d0829e: Pull complete
6d24e34787c7: Pull complete
066d623ff8e6: Pull complete
49486a4a61a6: Pull complete
34d83bb3522a: Pull complete
Digest: sha256:0a399eb16751829e1af26fea27b20c3ec28d7ab1fb72182879dcae1cca21206a
Status: Downloaded newer image for nginx:latest
INFO[0063] dumped image to tar tar=/var/folders/xp/hyg6_b6j7hgc23b7bd7d35l00000gn/T/karmor2062359421/pakTcFqD.tar
Distribution debian
created policy out/nginx-latest/write-under-dev-dir.yaml ...
created policy out/nginx-latest/k8s-client-tool-exec.yaml ...
created policy out/nginx-latest/file-system-mounts.yaml ...
created policy out/nginx-latest/crypto-miners.yaml ...
created policy out/nginx-latest/cis-commandline-warning-banner.yaml ...
created policy out/nginx-latest/file-integrity-monitoring.yaml ...
created policy out/nginx-latest/cronjob-cfg.yaml ...
created policy out/nginx-latest/pkg-mngr-exec.yaml ...
created policy out/nginx-latest/remote-file-copy.yaml ...
created policy out/nginx-latest/write-etc-dir.yaml ...
created policy out/nginx-latest/impair-defense.yaml ...
created policy out/nginx-latest/user-grp-mod.yaml ...
created policy out/nginx-latest/maint-tools-access.yaml ...
created policy out/nginx-latest/write-in-shm-dir.yaml ...
created policy out/nginx-latest/access-ctrl-permission-mod.yaml ...
created policy out/nginx-latest/remote-services.yaml ...
created policy out/nginx-latest/trusted-cert-mod.yaml ...
created policy out/nginx-latest/system-owner-discovery.yaml ...
created policy out/nginx-latest/system-network-env-mod.yaml ...
created policy out/nginx-latest/network-service-scanning.yaml ...
output report in out/report.txt ...
Container | nginx:latest
OS | linux
Arch | arm64
Distro | debian
Output Directory | out/nginx-latest
policy-template version | v0.2.3
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| POLICY | SHORT DESC | SEVERITY | ACTION | TAGS |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| write-under-dev-dir.yaml | Audit device directory for | 5 | Audit | NIST NIST_800-53_AU-2 |
| | enhanced security | | | NIST_800-53_SI-4 MITRE |
| | | | | MITRE_T1036_masquerading |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| k8s-client-tool-exec.yaml | Prevent execution of container | 5 | Block | MITRE_T1609_container_administration_command |
| | administration tools within a | | | MITRE_TA0002_execution |
| | container | | | MITRE_T1610_deploy_container |
| | | | | MITRE NIST_800-53 NIST_800-53_AU-2 |
| | | | | NIST_800-53_SI-4 NIST |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| file-system-mounts.yaml | Ensure successful file system | 5 | Audit | CIS CIS_Linux |
| | mounts are collected | | | CIS_4_Logging_and_Aduditing |
| | | | | CIS_4.1.1_Data_Retention |
| | | | | CIS_4.1.14_file_system_mount |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| crypto-miners.yaml | Cryptojacking, Crypto mining, | 10 | Block | cryptominer |
| | Malware protection | | | MITRE_T1496_resource_hijacking |
| | | | | MITRE |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| cis-commandline-warning-banner.yaml | Command Line Warning Banners | 5 | Block | CIS CIS_Linux CIS_1.7_Warning_Banners |
| | | | | CIS_1.7.1_Command_Line_Warning_Banners |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| file-integrity-monitoring.yaml | File Integrity | 1 | Block | NIST NIST_800-53_AU-2 |
| | Monitoring/Protection | | | NIST_800-53_SI-4 MITRE |
| | | | | MITRE_T1036_masquerading |
| | | | | MITRE_T1565_data_manipulation |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| cronjob-cfg.yaml | Audit access to cronjob files | 5 | Audit | NIST SI-4 |
| | as a part of system monitoring | | | NIST_800-53_SI-4 |
| | for better integrity | | | CIS CIS_Linux |
| | | | | CIS_5.1_Configure_Cron |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| pkg-mngr-exec.yaml | Prohibit package manager | 5 | Block | NIST |
| | process execution in | | | NIST_800-53_CM-7(4) |
| | containers to maintain system | | | SI-4 process |
| | integrity and limit authorized | | | NIST_800-53_SI-4 |
| | software versions and sources. | | | |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| remote-file-copy.yaml | Prevent data exfiltration | 5 | Block | MITRE |
| | attempts using utility tooling | | | MITRE_TA0008_lateral_movement |
| | | | | MITRE_TA0010_exfiltration |
| | | | | MITRE_TA0006_credential_access |
| | | | | MITRE_T1552_unsecured_credentials |
| | | | | NIST_800-53_SI-4(18) NIST |
| | | | | NIST_800-53 NIST_800-53_SC-4 |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| write-etc-dir.yaml | Prevent concealment of | 5 | Block | NIST_800-53_SI-7 NIST |
| | adversarial processes | | | NIST_800-53_SI-4 NIST_800-53 |
| | | | | MITRE_T1562.001_disable_or_modify_tools |
| | | | | MITRE_T1036.005_match_legitimate_name_or_location |
| | | | | MITRE_TA0003_persistence |
| | | | | MITRE MITRE_T1036_masquerading |
| | | | | MITRE_TA0005_defense_evasion |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| impair-defense.yaml | Audit defense control points | 6 | Audit | MITRE FGT1562 FIGHT 5G |
| | to detect defense impairments | | | MITRE_T1562_Impair _Defenses |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| user-grp-mod.yaml | Audit access to useradd and | 1 | Block | MySQL |
| | groupadd command! | | | CIS |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| maint-tools-access.yaml | Restrict or limit maintenance | 1 | Audit | PCI_DSS MITRE |
| | tool usage | | | MITRE_T1553_Subvert_Trust_Controls |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| write-in-shm-dir.yaml | Restrict adversaries from | 5 | Block | MITRE_TA0002_Execution |
| | writing malicious code under | | | MITRE |
| | the shm folder | | | |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| access-ctrl-permission-mod.yaml | Ensure discretionary | 5 | Block | CIS CIS_Linux CIS_4_Logging_and_Aduditing |
| | access control permission | | | CIS_4.1.1_Data_Retention |
| | modification events are | | | CIS_4.1.11_system_access_control_permission |
| | collected | | | |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| remote-services.yaml | Audit remote access services | 3 | Audit | MITRE FIGHT FGT1021 5G |
| | | | | MITRE_T1021_Remote_Services |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| trusted-cert-mod.yaml | Prevent certificate bundle | 1 | Block | MITRE |
| | tampering | | | MITRE_T1552_unsecured_credentials |
| | | | | FGT1555 FIGHT |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| system-owner-discovery.yaml | Limit adversaries from | 3 | Block | MITRE |
| | gathering system information | | | MITRE_T1082_system_information_discovery |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| system-network-env-mod.yaml | Ensure events that modify the | 5 | Block | CIS CIS_Linux |
| | system's network environment | | | CIS_4_Logging_and_Aduditing |
| | are collected | | | CIS_4.1.1_Data_Retention |
| | | | | CIS_4.1.7_system_network_environment |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| network-service-scanning.yaml | Audit execution of network | 5 | Audit | MITRE FGT1046 FIGHT 5G |
| | service scanning tools | | | MITRE_T1046_Network_Service_Discovery |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
rootxrishabh
requested changes
Jan 15, 2025
Member
rootxrishabh
left a comment
rootxrishabh
left a comment
There was a problem hiding this comment.
But the docker client is throwing an error for being newer than the daemon. Please think of a way to handle this as most users could face this issue. I see we are using github.com/docker/docker v25.0.5+incompatible. @daemon1024 any ideas?
accuknox/kubearmor-client - (tesla/non-k8s/karmor-recommend) > ./karmor recommend --k8s=false
Error: Error response from daemon: client version 1.44 is too new. Maximum supported API version is 1.43
Contributor
Author
|
hey @rootxrishabh can u share the output of
I think that begs the question how many Daemon version do we intend to support and which version should we use |
Contributor
Author
|
although the v27 docker client was already present in the |
Member
rootxrishabh
left a comment
rootxrishabh
left a comment
There was a problem hiding this comment.
My docker version is 24.0.2. So it should be supported?
Member
|
@tesla59, can we use github.com/docker/docker/clientv28? Let's check if it supports at least 3 versions below it. |
tesla59
force-pushed
the
tesla/non-k8s/karmor-recommend
branch
from
608c5f5 to
cd24886
Compare
Contributor
Author
|
hey @rootxrishabh , it seems to be working now with the older version using API Version Negotiation. |
daemon1024
moved this from P2 - PR Ready for review
to P1- PR Ready for review
in Release v1.6
tesla59
force-pushed
the
tesla/non-k8s/karmor-recommend
branch
2 times, most recently
from
ec9fb01 to
bb09e09
Compare
soorya-knox
moved this from P1- PR Ready for review
to P0 - PR Ready for review
in Release v1.6
Contributor
|
tested locally working as expected. |
rksharma95
previously approved these changes
Jul 15, 2025
Contributor
|
@tesla59 please rebase the main and squash the commits. |
cmd: rename k8s client to k8sclient
Signed-off-by: tesla59 <nishant@heim.id>
cmd: wrap k8sClient in interface to use in recommend package
Signed-off-by: tesla59 <nishant@heim.id>
k8s: fix ListDeployment method
Signed-off-by: tesla59 <nishant@heim.id>
recommend: refactor k8s policy generation
use a common interface to support other clients as well
also create common Object{} to support different k8s obejcts such as Deployment, Daemonset etc
Signed-off-by: tesla59 <nishant@heim.id>
recommend: use image name as Object's name while generating policy
Signed-off-by: tesla59 <nishant@heim.id>
recommend: remove DeploymentName Field from Object{}
Signed-off-by: tesla59 <nishant@heim.id>
recommend: use ListOptions to select deployment by labels
Signed-off-by: tesla59 <nishant@heim.id>
k8s: generate recommended policies for CronJob, DaemonSet and Jobs
Signed-off-by: tesla59 <nishant@heim.id>
k8s: generate recommend policy for statefulset and unowned replicaset
Signed-off-by: tesla59 <nishant@heim.id>
k8s: remove K8sClientWrapper abstraction
Signed-off-by: tesla59 <nishant@heim.id>
recommend: initialize dockerClient and generate policies for containers as well
Signed-off-by: tesla59 <nishant@heim.id>
recommend: add new flag k8s to specify which client to use
this removes dependency of recommend command on kubearmor
Signed-off-by: tesla59 <nishant@heim.id>
recommend: fallback to docker client if k8s client is not present
Signed-off-by: tesla59 <nishant@heim.id>
recommend: only log the client if images is not specified
Signed-off-by: tesla59 <nishant@heim.id>
recommend: generate policyDir based on image namespace set to null
Signed-off-by: tesla59 <nishant@heim.id>
recommend: trim \n in final report generation
Signed-off-by: tesla59 <nishant@heim.id>
recommend: handle err when listing objects
Signed-off-by: tesla59 <nishant@heim.id>
docker: Use API version negotiation to avoid version mismatch errors
Signed-off-by: tesla59 <nishant@heim.id>
tesla59
force-pushed
the
tesla/non-k8s/karmor-recommend
branch
from
bb09e09 to
ee40b91
Compare
Contributor
Author
|
@rksharma95 rebased to main and squashed the commits |
rootxrishabh
previously approved these changes
Jul 15, 2025
rksharma95
reviewed
Jul 16, 2025
Signed-off-by: rksharma95 <ramakant@accuknox.com>
rootxrishabh
approved these changes
Jul 22, 2025
Aryan-sharma11
approved these changes
Jul 22, 2025
github-project-automation
bot
moved this from P2 - PR Ready for review
to P0 - PR Ready for review
in Release v1.5
github-project-automation
bot
moved this from P0 - PR Ready for review
to Done
in Release v1.6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
part of kubearmor/KubeArmor#1815