[EKS] Encrypt secrets at rest in etcd

[ChrisCooney]

Tell us about your request
As part of K8s 1.13, encryption of k8s secret resources inside etcd is now fully supported. EKS should support this, as many organisations require that their sensitive information is stored in an encrypted format.

Which service(s) is this request for?
EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
Many organisations have a hard requirement that sensitive information is encrypted, both in transit and in storage. Currently, EKS does not support encryption of k8s secrets at rest, which creates a barrier to adoption.

Are you currently working around this issue?
There are third party controllers that we're using, but since k8s 1.13 is going to support this natively, this will likely become the dominant method for at rest encryption.

EDIT: Since EKS does support encryption in Etcd, this should clearly be documented somewhere in the Aws documentation.

[ejc3]

AWS encrypts etcd at rest:
https://blog.uprightvinyl.co.uk/2018/02/08/aws-builders-day-part-3-kubernetes-on-aws-with-amazon-eks/

Are you looking for keys you control?

[ChrisCooney]

Ah wow, nevermind then. I read an article recently that pointed out EKS was still plaintext at rest. I'll be sure to comment on the article. Thank you, I'll close this now.

[thisisananth]

I think we would be more comfortable if AWS states that in their documentation. It is not stated anywhere!!!

[maltekrupa]

I opened an issue about this exact topic a couple of days ago with AWS support. I'm still waiting for an answer.

I also asked to put this information somewhere in the documentation to make sure I don't have to forward customers to a third party blog when they ask me about encryption of EKS.

[thisisananth]

HI @temal- Did you hear back from AWS about this? Can you please update this with what they told? Thanks!

[maltekrupa]

@thisisananth Sadly not. As soon as I hear back, I'll write it down here!

@ChrisCooney Do you think it is worth reopening this issue? From my point of view it is still not clear if EKS supports encryption at rest for etcd.

[ChrisCooney]

I'm inclined to agree. The absence of detail in the documentation is enough. Perhaps the title needs to change to prevent it being benched.

[jacob-koren-zooz]

Any update on this issue? Was the documentation updated? I can't find anything about encryption of secrets in EKS

[maltekrupa]

Latest update:

I heard from the service team that they are working on the official documentation and will be published soon.

[pmnhatdn]

Hi @temal- have you got any update from AWS on the update of EKS secret encryption documentation?

Thanks

[maltekrupa]

@pmnhatdn Not yet, no.

[maltekrupa]

I got an answer:

Greetings Malte!!!
I appreciate for the patience while I get the response from the service team.
Good news is that I was able to get the answer to this particular query from service team. The answer is yes, the data stored by etcd is encrypted at rest.
EKS team planning to share the details through the FAQ in future, however no ETA can be provided.
Again, should you have any other questions or queries, please so not hesitate to reach out to us and we will be happier to assist.
Have a great day ahead!
Best regards,

🎉

My issue with AWS about this topic is closed. I guess we should let the github issue open till the documentation appears in the FAQ. Else this comment from me is as worthy as the blogpost posted in the beginning.

[fgreinacher]

Based on the information above is it safe to assume that using Kubernetes secrets for sensitive data in a production environment is an appropriate solution?

[jason-riddle]

Can this title be updated to “Add etcd encryption-at-rest documentation to EKS service docs/faq”? The title in its current form makes it seem like etcd isn’t encrypted at rest which is confusing when people open to read the comments and is not the case.

[tabern]

It seems like there are two asks here, a public statement about the encryption of data at rest within EKS etcd and the ability to enable envelope encryption of Kubernetes secrets using a customer managed key (CMK).

As of today, we now include a public statement about etcd encryption in the EKS documentation:

All of the data stored by the etcd nodes and associated Amazon EBS volumes is encrypted. Amazon EKS uses master encryption keys that generate volume encryption keys which are managed by the Amazon EKS service.

Additionlly, we plan to enable the AWS Encryption Provider on EKS and allow you to provide a AWS KMS CMK that will be used for envelope encryption of Kubernetes secrets on the cluster.

You can track this on #530

-nate

[ChrisCooney]

As the person who opened this issue, the docs was my only ask. Thanks for sorting ❤️❤️

…

On Fri, 11 Oct 2019, 07:08 Nate Taber, ***@***.***> wrote: Closed #263 <#263>. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#263?email_source=notifications&email_token=AAUCOSXQ6NCGUWYDE3WYUA3QOAJ4TA5CNFSM4HHMOE6KYY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOUE5X2SA#event-2705030472>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAUCOSWOU5KQEV4LCJO23I3QOAJ4TANCNFSM4HHMOE6A> .