Creating application security checklist #46326

AnshumanTripathi · 2024-05-12T02:17:53Z

Creating an application security checklist.

Issue

Deploy Preview

https://deploy-preview-46326--kubernetes-io-main-staging.netlify.app/docs/concepts/security/application-security-checklist/

netlify · 2024-05-12T02:20:44Z

✅ Pull request preview available for checking

Built without sensitive environment variables

Name	Link
🔨 Latest commit	`4f75e48`
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-io-main-staging/deploys/66ef3ddc6eb9530008ce648b
😎 Deploy Preview	https://deploy-preview-46326--kubernetes-io-main-staging.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

content/en/docs/concepts/security/application-security-checklist.md

sftim · 2024-06-12T09:25:01Z

/remove-sig release
/sig security

content/en/docs/concepts/security/application-security-checklist.md

mtardy · 2024-06-14T08:51:22Z

This looks good :)! I think it's a very valuable checklist for app developers.

NissesSenap

First of all, great job, I think this document is a great addition.

But here are my thoughts.
First, my question is, for whom is this document written for? Is it the 85-90% of app developers that run an http API or an event based app in a non-critical environment that just wants a good base layer? Or are we trying to point to all security features that exist from an app developer point of view?

Personally, I think we should aim this document mainly towards the 85-90% of the users.
Especially since there is a tendency to want to be able to complete everything on a checklist, but if it becomes too painful, you end up doing nothing.
See keep it relatively simple and provide some kind of leveling system.

I would add a header called ## base security. In there I would add the most basic stuff.

Then I would add another header, ## advanced security, or something like that.

I would keep everything you have written, but I would move these down to advanced security

rbac, if you are using rbac in your http api, you are most likely doing something wrong. I would emphasize that really hard in the document.
runtime class, custom runetime classes isn't something that the majority of Kubenetes environment will use.
Seccomp profiles, It's more or less impossible to setup unless you have some kind of tooling + since version 1.25 the default profile is enabled by default.
SElinux, most developers don't know what it is and just like secomp profile you more or less need tooling to create something decent.
AppArmor, even fewer people know what it is, and you need custom tooling to create something more advanced than default.

content/en/docs/concepts/security/application-security-checklist.md

AnshumanTripathi · 2024-06-24T06:39:28Z

First of all, great job, I think this document is a great addition.

But here are my thoughts. First, my question is, for whom is this document written for? Is it the 85-90% of app developers that run an http API or an event based app in a non-critical environment that just wants a good base layer? Or are we trying to point to all security features that exist from an app developer point of view?

Personally, I think we should aim this document mainly towards the 85-90% of the users. Especially since there is a tendency to want to be able to complete everything on a checklist, but if it becomes too painful, you end up doing nothing. See keep it relatively simple and provide some kind of leveling system.

I would add a header called ## base security. In there I would add the most basic stuff.

Then I would add another header, ## advanced security, or something like that.

I would keep everything you have written, but I would move these down to advanced security

rbac, if you are using rbac in your http api, you are most likely doing something wrong. I would emphasize that really hard in the document.

runtime class, custom runetime classes isn't something that the majority of Kubenetes environment will use.

Seccomp profiles, It's more or less impossible to setup unless you have some kind of tooling + since version 1.25 the default profile is enabled by default.

SElinux, most developers don't know what it is and just like secomp profile you more or less need tooling to create something decent.

AppArmor, even fewer people know what it is, and you need custom tooling to create something more advanced than default.

Thanks for the detailed response. I like this format and I will try to update the document accordingly.
Regarding the target audience I did add this point

This checklist assumes a developer is a Kubernetes cluster user that interacts with namespaced scope objects.
https://github.com/kubernetes/website/pull/46326/files#diff-5285725b7ae5cc6b415f12dbb2acf91aea311bf5dd5dc71851b29c216afcb9adR20

I did realize that identifying the target audience is key here and then keeping the list generic so that it can be used by everyone. Because app developers can consist of tools that can be running cluster wide like log and metric collectors or policy agents. This is where the line between admins and developers start to blur. I'll try to add more details about the target audience for this list.

AnshumanTripathi · 2024-07-22T07:37:11Z

@NissesSenap I have updated the checklist structure based on your recommendation.

Can you expand on

rbac, if you are using rbac in your http api, you are most likely doing something wrong. I would emphasize that really hard in the document.

I'm not sure what you mean here.

NissesSenap · 2024-07-23T08:39:02Z

@NissesSenap I have updated the checklist structure based on your recommendation.

Can you expand on

rbac, if you are using rbac in your http api, you are most likely doing something wrong. I would emphasize that really hard in the document.

I'm not sure what you mean here.

That is great @AnshumanTripathi , I will try to take a deeper look at the update as soon as possible. My point is that the majority of all apps ruining in Kubernetes don't need access to its API. So I would emphasize that custom rbac rules for apps just aren't needed, in the majority of applications. Still good to talk about it, just like you have, but point to that there are limited use cases for it.

AnshumanTripathi · 2024-07-23T16:56:08Z

@NissesSenap I have updated the checklist structure based on your recommendation.
Can you expand on

rbac, if you are using rbac in your http api, you are most likely doing something wrong. I would emphasize that really hard in the document.

I'm not sure what you mean here.

That is great @AnshumanTripathi , I will try to take a deeper look at the update as soon as possible. My point is that the majority of all apps ruining in Kubernetes don't need access to its API. So I would emphasize that custom rbac rules for apps just aren't needed, in the majority of applications. Still good to talk about it, just like you have, but point to that there are limited use cases for it.

Makes sense. I am not covering applications using Kubernetes client as a part of this checklist. I think having a doc about secure practices for applications that use the Kubernetes API (and operator) can be very helpful. I will create an issue for this :)

AnshumanTripathi · 2024-07-24T19:01:36Z

@NissesSenap created kubernetes/sig-security#121 for hardening guide for using Kubernetes API
Cross posted issue on k website - #47405

Signed-off-by: Anshuman Tripathi <anshuman.tripathi305@gmail.com> Update based on feedback Signed-off-by: Anshuman Tripathi <anshuman.tripathi305@gmail.com> Update based on feedback Signed-off-by: Anshuman Tripathi <anshuman.tripathi305@gmail.com> Update checklist reading guide Signed-off-by: Anshuman Tripathi <anshuman.tripathi305@gmail.com> Update checklist structure based on feedback Signed-off-by: Anshuman Tripathi <anshuman.tripathi305@gmail.com>

k8s-ci-robot · 2024-09-21T21:43:02Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign sftim for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

content/en/docs/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

tengqm · 2024-09-22T02:19:39Z

I'd suggest we place this under content/en/docs/reference/security/, a new directory for consolidating security related contents. This page is not suitable for the concepts section.

AnshumanTripathi · 2024-09-22T02:36:59Z

I'd suggest we place this under content/en/docs/reference/security/, a new directory for consolidating security related contents. This page is not suitable for the concepts section.

Actually we can move this checklist and the existing checklist to content/en/docs/reference/issues-security maybe? I dont think I should do that as a part of this change though.

tengqm · 2024-09-22T02:42:51Z

I'd suggest we place this under content/en/docs/reference/security/, a new directory for consolidating security related contents. This page is not suitable for the concepts section.

Actually we can move this checklist and the existing checklist to content/en/docs/reference/issues-security maybe? I dont think I should do that as a part of this change though.

The issues-security directory is positioned to be a folder for disclosing the vulnerabilities of kubernetes release. What we are trying to compile is a collection of guidelines, best practices for users.

I don't think we want to kick this in and then follow up with another PR to move it.
We may want to get the page into a suitable folder in the first place.
The migration of other relevant pages could be left as follow-ups, for sure.

AnshumanTripathi · 2024-09-22T02:44:35Z

I'd suggest we place this under content/en/docs/reference/security/, a new directory for consolidating security related contents. This page is not suitable for the concepts section.

Actually we can move this checklist and the existing checklist to content/en/docs/reference/issues-security maybe? I dont think I should do that as a part of this change though.

The issues-security directory is positioned to be a folder for disclosing the vulnerabilities of kubernetes release. What we are trying to compile is a collection of guidelines, best practices for users.

I don't think we want to kick this in and then follow up with another PR to move it. We may want to get the page into a suitable folder in the first place. The migration of other relevant pages could be left as follow-ups, for sure.

Makes sense. I'll try to make that change

sftim

Thanks! We could really use this new page.

Here's some advice about how to get this ready to merge.

sftim · 2024-09-23T11:24:57Z

content/en/docs/concepts/security/application-security-checklist.md

+
+For sensitive workloads consider using Kernel emulation tools like [gVisor](https://gvisor.dev/docs/) or [kata-containers](https://katacontainers.io/).
+
+In high trust environments, consider using [confidential virutal machines](/blog/2023/07/06/confidential-kubernetes/) to improve cluster security even further.


Suggested change

In high trust environments, consider using [confidential virutal machines](/blog/2023/07/06/confidential-kubernetes/) to improve cluster security even further.

In high trust environments, consider using [confidential virtual machines](/blog/2023/07/06/confidential-kubernetes/) to improve cluster security even further.

Watch out for linking to non-evergreen blog articles (anything that gets marked as out of date once at least a year old).

sftim · 2024-09-23T11:25:35Z

content/en/docs/concepts/security/application-security-checklist.md

+
+Some containers may require a different isolation level from what is provided by the default runtime of the cluster. `runtimeClassname` can be used in a podspec to define a differnt runtime class.
+
+For sensitive workloads consider using Kernel emulation tools like [gVisor](https://gvisor.dev/docs/) or [kata-containers](https://katacontainers.io/).


Kata containers is not a kernel emulator:

Suggested change

For sensitive workloads consider using Kernel emulation tools like [gVisor](https://gvisor.dev/docs/) or [kata-containers](https://katacontainers.io/).

For sensitive workloads consider using kernel emulation tools like [gVisor](https://gvisor.dev/docs/), or virtualized isolation using a mechanism such as [kata-containers](https://katacontainers.io/).

sftim · 2024-09-23T11:26:04Z

content/en/docs/concepts/security/application-security-checklist.md

This page needs a third party links disclaimer; see https://kubernetes.io/docs/contribute/style/hugo-shortcodes/#third-party-content-marker

API verbs like apply and create are written as lower case bold. We use upper case non bold when documenting verbs at the underlying HTTP level (eg OPTIONS, PUT, GET).

(nit) When a sentence ends with a period, ie ., don't include the period inside the anchor text of a hyperlink.

sftim · 2024-09-23T11:27:08Z

content/en/docs/concepts/security/application-security-checklist.md

+The following checklist provide a base security hardening recommendations that would apply to most applications deploying to Kubernetes.
+
+### Application design
+- [ ] Following the right [design principles for the application.](https://kubernetes.io/blog/2018/03/principles-of-container-app-design/)


I don't recommend linking to that article. Find a better source, such as https://www.cncf.io/wp-content/uploads/2022/06/CNCF_cloud-native-security-whitepaper-May2022-v2.pdf

sftim · 2024-09-23T11:27:36Z

content/en/docs/concepts/security/application-security-checklist.md

+  - [ ] CPU limit might be set on sensitive workloads.
+
+### Service account
+- [ ] Avoid using `default` service account. Create service accounts for workloads.


Suggested change

- [ ] Avoid using `default` service account. Create service accounts for workloads.

- [ ] Avoid using the `default` ServiceAccount. Create ServiceAccounts for each workload or microservice.

sftim · 2024-09-23T11:35:15Z

content/en/docs/concepts/security/application-security-checklist.md

+### Container Security Context
+- [ ] Disable privilege escalations using `allowPrivilegeEscalation: false`.
+- [ ] Configure the root filesystem to be read-only with `readOnlyRootFilesystem: true`.
+- [ ] Avoid running privileged containers with `privileged: false`.


Suggested change

- [ ] Avoid running privileged containers with `privileged: false`.

- [ ] Avoid running privileged containers (set `privileged: false`).

sftim · 2024-09-23T11:35:44Z

content/en/docs/concepts/security/application-security-checklist.md

+- [ ] Avoid using `default` service account. Create service accounts for workloads.
+- [ ] `automountServiceAccountToken` should be set to `false` unless the pod specifically requires access to the Kubernetes API to operate.
+
+### Pod Security Context


Suggested change

### Pod Security Context

### Pod-level `securityContext` recommendations {#security-context-pod}

sftim · 2024-09-23T11:36:01Z

content/en/docs/concepts/security/application-security-checklist.md

+- [ ] Optionally add a supplimentary group with `fsGroup` to access persistent volumes.
+- [ ] Appropriate Pod Security Standard policies are enforced to all pods in the application namespace.
+
+### Container Security Context


Suggested change

### Container Security Context

### Container-level `securityContext` recommendations {#security-context-container}

sftim · 2024-09-23T11:37:46Z

content/en/docs/concepts/security/application-security-checklist.md

+- [ ] Set `runAsNonRoot: true`.
+- [ ] Configure User permissions and access controls using `runAsUser` and `runAsGroup`.
+- [ ] Optionally add a supplimentary group with `fsGroup` to access persistent volumes.
+- [ ] Appropriate Pod Security Standard policies are enforced to all pods in the application namespace.


Maybe this:

Suggested change

- [ ] Appropriate Pod Security Standard policies are enforced to all pods in the application namespace.

- [ ] The application deploys into a namespace that enforces an appropriate _Pod security standard_.

If you cannot control this enforcement for the cluster(s) where the application is deployed,

take this into account either through documentation or additional defense in depth.

sftim · 2024-09-23T11:39:33Z

content/en/docs/concepts/security/application-security-checklist.md

+- [ ] Avoid creating RBAC permissions to create or update roles which can lead to [privilege escalation](/docs/reference/access-authn-authz/rbac/#privilege-escalation-prevention-and-bootstrapping).
+- [ ] Review bindings for the `system:unauthenticated` group and remove them where possible, as this gives access to anyone who can contact the API server at a network level.
+
+The `create`, `update` and `delete` should be used judicously. The `patch` permission can [allow users to update labels on the namespace or deployments](/docs/concepts/security/rbac-good-practices/#namespace-modification) which can increase the attack surface.


Try this:

Suggested change

The `create`, `update` and `delete` should be used judicously. The `patch` permission can [allow users to update labels on the namespace or deployments](/docs/concepts/security/rbac-good-practices/#namespace-modification) which can increase the attack surface.

The **create**, **update** and **delete** verbs should be permitted judiciously. The **patch** verb if allowed on a Namespace can

[allow users to update labels on the namespace or deployments](/docs/concepts/security/rbac-good-practices/#namespace-modification) which can increase the attack surface.

For sensitive workloads, consider providing a recommended ValidatingAdmissionPolicy that further restricts the permitted write actions.

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. language/en Issues or PRs related to English language labels May 12, 2024

k8s-ci-robot requested review from kbhawkey and salaxander May 12, 2024 02:17

k8s-ci-robot added sig/docs Categorizes an issue or PR as relevant to SIG Docs. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 12, 2024

AnshumanTripathi force-pushed the anshuman/app_sec_checklist branch 2 times, most recently from 252c1b0 to 1dae29a Compare May 14, 2024 06:11

raesene reviewed Jun 6, 2024

View reviewed changes

content/en/docs/concepts/security/application-security-checklist.md Outdated Show resolved Hide resolved

raesene reviewed Jun 6, 2024

View reviewed changes

content/en/docs/concepts/security/application-security-checklist.md Show resolved Hide resolved

raesene reviewed Jun 6, 2024

View reviewed changes

content/en/docs/concepts/security/application-security-checklist.md Outdated Show resolved Hide resolved

k8s-ci-robot added sig/security Categorizes an issue or PR as relevant to SIG Security. and removed sig/release Categorizes an issue or PR as relevant to SIG Release. labels Jun 12, 2024

mtardy reviewed Jun 14, 2024

View reviewed changes

AnshumanTripathi force-pushed the anshuman/app_sec_checklist branch from ad29660 to 78f4f72 Compare June 19, 2024 19:28

AnshumanTripathi marked this pull request as ready for review June 19, 2024 19:28

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 19, 2024

k8s-ci-robot requested review from dipesh-rawat and Princesso June 19, 2024 19:28

AnshumanTripathi mentioned this pull request Jun 19, 2024

Security Checklist for Applications from a developer perespective kubernetes/sig-security#111

Open

NissesSenap reviewed Jun 24, 2024

View reviewed changes

content/en/docs/concepts/security/application-security-checklist.md Show resolved Hide resolved

AnshumanTripathi force-pushed the anshuman/app_sec_checklist branch from 78f4f72 to a93d643 Compare July 22, 2024 07:35

AnshumanTripathi force-pushed the anshuman/app_sec_checklist branch from a93d643 to c1c6f4a Compare July 22, 2024 07:41

AnshumanTripathi mentioned this pull request Aug 8, 2024

Security recommendation/hardening guide for applications that use Kubernetes client #47405

Open

AnshumanTripathi mentioned this pull request Sep 15, 2024

REQUEST: New membership for AnshumanTripathi kubernetes/org#5160

Closed

11 tasks

AnshumanTripathi force-pushed the anshuman/app_sec_checklist branch from c1c6f4a to 4f75e48 Compare September 21, 2024 21:42

drackpack mentioned this pull request Sep 22, 2024

Add a desc #48035

Closed

sftim reviewed Sep 23, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Creating application security checklist #46326

Creating application security checklist #46326

AnshumanTripathi commented May 12, 2024 •

edited

Loading

netlify bot commented May 12, 2024 •

edited

Loading

sftim commented Jun 12, 2024

mtardy commented Jun 14, 2024

NissesSenap left a comment

AnshumanTripathi commented Jun 24, 2024 •

edited

Loading

AnshumanTripathi commented Jul 22, 2024

NissesSenap commented Jul 23, 2024

AnshumanTripathi commented Jul 23, 2024

AnshumanTripathi commented Jul 24, 2024 •

edited

Loading

k8s-ci-robot commented Sep 21, 2024

tengqm commented Sep 22, 2024

AnshumanTripathi commented Sep 22, 2024

tengqm commented Sep 22, 2024

AnshumanTripathi commented Sep 22, 2024

sftim left a comment

sftim Sep 23, 2024

sftim Sep 23, 2024

sftim Sep 23, 2024

sftim Sep 23, 2024

sftim Sep 23, 2024

sftim Sep 23, 2024

sftim Sep 23, 2024

sftim Sep 23, 2024

sftim Sep 23, 2024

sftim Sep 23, 2024


		For sensitive workloads consider using Kernel emulation tools like [gVisor](https://gvisor.dev/docs/) or [kata-containers](https://katacontainers.io/).

		In high trust environments, consider using [confidential virutal machines](/blog/2023/07/06/confidential-kubernetes/) to improve cluster security even further.


		Some containers may require a different isolation level from what is provided by the default runtime of the cluster. `runtimeClassname` can be used in a podspec to define a differnt runtime class.

		For sensitive workloads consider using Kernel emulation tools like [gVisor](https://gvisor.dev/docs/) or [kata-containers](https://katacontainers.io/).

	- [ ] Avoid using `default` service account. Create service accounts for workloads.
	- [ ] Avoid using the `default` ServiceAccount. Create ServiceAccounts for each workload or microservice.

	- [ ] Avoid running privileged containers with `privileged: false`.
	- [ ] Avoid running privileged containers (set `privileged: false`).

	### Pod Security Context
	### Pod-level `securityContext` recommendations {#security-context-pod}

	### Container Security Context
	### Container-level `securityContext` recommendations {#security-context-container}

-- [ ] Appropriate Pod Security Standard policies are enforced to all pods in the application namespace.
+- [ ] The application deploys into a namespace that enforces an appropriate _Pod security standard_.
+      If you cannot control this enforcement for the cluster(s) where the application is deployed,
+      take this into account either through documentation or additional defense in depth.

Creating application security checklist #46326

Are you sure you want to change the base?

Creating application security checklist #46326

Conversation

AnshumanTripathi commented May 12, 2024 • edited Loading

Issue

Deploy Preview

netlify bot commented May 12, 2024 • edited Loading

✅ Pull request preview available for checking

sftim commented Jun 12, 2024

mtardy commented Jun 14, 2024

NissesSenap left a comment

Choose a reason for hiding this comment

AnshumanTripathi commented Jun 24, 2024 • edited Loading

AnshumanTripathi commented Jul 22, 2024

NissesSenap commented Jul 23, 2024

AnshumanTripathi commented Jul 23, 2024

AnshumanTripathi commented Jul 24, 2024 • edited Loading

k8s-ci-robot commented Sep 21, 2024

tengqm commented Sep 22, 2024

AnshumanTripathi commented Sep 22, 2024

tengqm commented Sep 22, 2024

AnshumanTripathi commented Sep 22, 2024

sftim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AnshumanTripathi commented May 12, 2024 •

edited

Loading

netlify bot commented May 12, 2024 •

edited

Loading

AnshumanTripathi commented Jun 24, 2024 •

edited

Loading

AnshumanTripathi commented Jul 24, 2024 •

edited

Loading