CIS Hardening Benchmark for Bottlerocket

[gregdek]

To quote our friends at Microsoft: "CIS benchmarks are internationally recognized as security standards for defending IT systems and data against cyberattacks. Used by thousands of businesses, they offer prescriptive guidance for establishing a secure baseline configuration."

We intend to work with our friends at CIS to create a CIS Bottlerocket Community. Much of the work will likely happen at the CIS Workbench site, but we can use this issue to track our progress towards the goal of having a CIS Benchmark for Bottlerocket.

[mhoppe-cedar]

This is great new @gregdek ! I am new to bottlerocket and trying to figure out how to build it in a fips-compliant way. I have found procedures for working with AL2 (al2 hardening), but not familiar with working in systems without a package manager really. Would you be able to offer me any guidance?

[gregdek]

Part of the issue here is that Bottlerocket is essentially "hardened by default". It's not intended to be rebuilt in most cases.

Unlike most operating systems, which are potentially insecure and require specific configuration to secure them, Bottlerocket is already highly secure, and if you want to open up ssh or allow containers to run in a more highly privileged way, you have to make conscious decisions to do that.

Now the question we have to answer is, how do we approach various third party certifications that make it clear that the mechanisms we use to harden Bottlerocket match the third party's requirements? That's what we're working on now.

[mhoppe-cedar]

Thank you for the thoughtful reply. I completely get it and appreciate the work you and your team are doing in this space. We're sold on bottlerocket being the future :).

[vennemp]

When you say third-party certifications are you including research into configurations to ensure containers hosted in bottlerocket are FIPS 140-2 compliant? Obviously this will require the containers themself to use the proper crypto modules but some of the compliance depends on using a kernel in some sort of "FIPS mode"

[zmrow]

@vennemp Do you mind clarifying a bit? Are you asking if we will be validating the containers customers intend to run on Bottlerocket?

[gregdek]

I am no longer an employee of AWS, so this is just friendly advice.

First, re: CIS -- there is not yet a CIS standard for the "container optimized OS" (at least, there wasn't as of a few months ago), and the CIS benchmarks for full-blown OSes are wildly inaccurate for a container optimized OS. There is, however, a CIS standard for nodes that run under Kubernetes, and Bottlerocket should fit well under that standard. You should be able to run kube-bench to show your org that Bottlerocket meets the CIS standard for nodes that run under Kubernetes. That's not the same as "certification" but it might be good enough.

FIPS is a completely different set of standards and is not addressed by this issue. If you're interested in FIPS compliance you should probably open a separate issue so the team can track and prioritize.

[ManuelMueller1st]

I am no longer an employee of AWS, so this is just friendly advice.

First, re: CIS -- there is not yet a CIS standard for the "container optimized OS" (at least, there wasn't as of a few months ago), and the CIS benchmarks for full-blown OSes are wildly inaccurate for a container optimized OS. There is, however, a CIS standard for nodes that run under Kubernetes, and Bottlerocket should fit well under that standard. You should be able to run kube-bench to show your org that Bottlerocket meets the CIS standard for nodes that run under Kubernetes. That's not the same as "certification" but it might be good enough.

FIPS is a completely different set of standards and is not addressed by this issue. If you're interested in FIPS compliance you should probably open a separate issue so the team can track and prioritize.

Quote
Additionally, this benchmark is not applicable to nodes’ operating system level security configuration as this is outside the scope for the CIS Kubernetes Benchmark. You may want to refer to the CIS Amazon Linux 2 Benchmark for further reference on applicable security controls for Amazon Linux 2 images.

https://aws.amazon.com/de/blogs/containers/introducing-cis-amazon-eks-benchmark/

[vennemp]

Thanks for your comments. I opened a new issue as recommended. #1667

[misterek]

Looks like this one has been moved around a bit.

Is this no longer on the roadmap it seems?

[joebowbeer]

A level 2 recommendation for container-optimized OS, followed by links to Bottlerocket, was added to the CIS Benchmark for EKS v1.1.0, published at cisecurity.org on 4/13/2022.

3.3.1 Prefer using Container-Optimized OS when possible (Manual)

[voidlily]

Is there a benchmark for Bottlerocket in a similar vein to how there's a benchmark for Amazon Linux 2?

I'm asking because I saw a Google Container-Optimized OS benchmark 1.0.0 published in December 2021 and what from that benchmark would translate to necessary hardening work for Bottlerocket AMIs as well.

[voidlily]

https://github.com/bottlerocket-os/bottlerocket/blob/develop/SECURITY_FEATURES.md
https://github.com/bottlerocket-os/bottlerocket/blob/develop/SECURITY_GUIDANCE.md

I was looking at these, particularly SECURITY_GUIDANCE, and maybe this could work as a benchmark

[bcressey]

This issue has bounced between a few milestones, partly because it doesn't map cleanly to a particular Bottlerocket release. It's definitely still on the roadmap, though!

I've been working steadily on the CIS benchmark for Bottlerocket for the last month. There are a few supporting features coming in the 1.9.0 release (#2234, #2235, #2236) and I expect to have the draft published for community feedback in July.

[misterek]

This is fantastic. Nice work @bcressey !

[voidlily]

any updates on this benchmark?

[bcressey]

@voidlily the CIS benchmark is in the consensus review stage at the moment. It looks like you'd need to sign up at https://workbench.cisecurity.org/ to view the draft benchmark, but depending on feedback it should be public next week.

[misterek]

@bcressey it looks like it's been Accepted but not Published quite yet? That's exciting!

I was considering the best way to validate this. Any thoughts on if having something like a bootstrap container validate these settings would be a reasonable approach? I was considering maybe having a bootstrap container that published a json file with validation on every boot, or something like that.

[bcressey]

@misterek the bootstrap container approach sounds reasonable.

I've also thought about adding support for the benchmark to https://github.com/google/localtoast and integrating that tool into the host, with a corresponding actions API and apiclient ... subcommand to trigger a scan. I haven't looked very closely at it yet, since the integration is blocked by #2053 (to make it easier to include Go programs that rely on modules).

[misterek]

Oh, I hadn't seen that. Maybe I'll try to use that in a bootstrap container. Seems basically like what I was trying to do anyway.

[kyhau]

From AWS What's New

Center for Internet Security (CIS) Benchmark for Bottlerocket is now available
https://aws.amazon.com/about-aws/whats-new/2022/08/center-for-internet-security-bottlerocket-available/

[mhoppe-cedar]

thanks @kyhau!

[misterek]

Awesome work @bcressey .

[jpmcb]

I've also thought about adding support for the benchmark to https://github.com/google/localtoast and integrating that tool into the host, with a corresponding actions API and apiclient ... subcommand to trigger a scan.

Since #2053 has merged, I looked a bit at adding localtoast to the host: it has a buildtime dependency on protoc to generate it's gRPC protos and needs some work to get it's vendored dependencies in, but in theory, it should now be possible to bring that in if we want to begin exploring that!

[joebowbeer]

This stated goal was to create CIS Benchmark for Bottlerocket, and this has been done, right?

Does more work remain for this? Close this and open a new issue for localtoast?

[stmcginnis]

@bcressey can this issue be closed?

[stmcginnis]

The only open part of this discussion I think was being able to make it easy to run this benchmark on demand. That enhancement is being tracked in #2731, so closing this issue as completed.

Please subscribe to #2731 if you are interested in updates on that work.

[nike21oct]

hi all, i have a EKS cluster which is using bottlerocket AMI and nginx as a ingress controller and when i implemented these IP tables rules by bootstrap-container my application stop open from outside the cluster , i mean my ingress is not functioning my nginx ingress controller pod went to crashloopbackoff , ngix controller load balancer in aws in target group the Protocol : Port is TLS: 32443 and health check is using protocol http and port is 32002, so what should i need to do?
please help me here