- File encryption - applications should be able to encrypt files without having to store encryption keys.
- Data encryption - protecting sensitive information inside files. Applications should be able to decrypt encrypted information stored in files without having to store encryption keys. For example, database and sftp credentials.
- Integrity of the encrypted data should be protected.
- Encryption methodology should be designed to implement compliance requirements.
- Tenant level access to keys
- Separate keys for production and test.
- Follow least privilege principle when granting access to tenant keys to various applications in a solution/implementation.
- Maintain a trail of access logs for all encryption operations.
- Ability to scale encryption operations
- Highly available encryption service
Application server configuration files usually contain database credentials for application schemas. These credentials are stored in plaintext and can be read by anyone that has access to the application server configuration. Similarly, application configuration files have credentials to service endpoints, like SFTP and message queues, that are stored in plaintext.
Sensitive information should be stored in encrypted form, and only applications should have permissions to decrypt it. Also, encryption keys should not be accessible by users to prevent them from decrypting sensitive information.
All of hCentive's products work with PHI and PII. As a compliance requirement, these files should be encrypted when they are stored. The files may be accessed by different applications that make up an implementation. The files should be decrypted only when authorized applications accesses them.
Support personnel require access to the files encrypted files containing PHI and PII to debug issues. Currently, they download the files to their workstations to debug problems. Ideally, there should be an interface in the administrative console of the applications to review files. This will remove the need for support personnel to download files.
hCentive's product implementations are normally deployed by setting up it's customers as tenants in various multi-tenant applications. Tenant level keys will allow implementation of fine-grained access to encrypted data for tenants. It is also simpler to manage key policies per tenant compared to shared keys between tenants. Applications for a single tenant can also be set up with tenant specific keys.
Amazon Web Services (AWS) released their Key Management Service (KMS) recently. Most of hCentive's requirements can be implemented with it. Custom solutions can also be built using the AWS Identity and Access Management (IAM) and KMS APIs for requirements that are not met out of the box.
Each tenant has it's own master key. This master key is used to generate data keys for encryption. Authorization policies define which applications and users have permissions to perform actions.
- Account Master Key - managed by AWS; assigned per account
- Tenant Master Key - created for each tenant; rooted in the account master key; rotation policy defines lifetime.
- Tenant Data Keys - used to encrypt data; only stored in memory when required; encrypted data key stored along with encrypted data to enable decryption.
Envelope encryption is a basic construct that utilizes two or more cryptographic keys to secure data. The tenant data key is derived from the tenant master key and is used to encrypt data. The envelope contains the ciphertext, which is generated by encrypting data with the data key, and the data key that is encrypted with the master key. The application decrypting the data has access to the master key with which it decrypts the encrypted data key, and then uses that to decrypt data.
Authenticated encryption prevents tampering wth the encrypted data by restricted access to KMS via IAM policies. If a user is able to decrypt data then the data must be encrypted by an authorized KMS user.
The encryption context is a key-value map that is provided to KMS with encryption and decryption requests. The map used for encryption must match the map used for decryption. It is the additional authenticated data (AAD) that provides a context to the encrypted data.
The encryption context provides the following benefits
- Additional authenticated data (AAD)
- Auditing - is logged in CloudTrail logs
- Authorization context - can be used in the IAM key policy
Encryption context should not be hard-coded, and should derived from the data being encrypted. It should also be stored with the data where possible. It is immutable and, thus, requires careful consideration. It should not contain sensitive information as it is logged in CloudTrail logs. It is used to classify data by through the IAM authorization policy.
- KMS API calls logged in CloudTrail
- Correlate CloudTrail logs with application logs
- Track calls not made by applications
Each tenant is assigned it's own master key. Data keys are generated from the tenant master key for encrypting data. The data key itself is encrypted with the master key.
Each tenant master key can be assigned a human readable alias. It also has a key ID assigned to it that AWS generates for it. It is recommended to use the alias to look up a key. When a key is rotated, the same alias can be assigned to the new key without any impact to applications.
Tenant master keys are created per environment. The key alias nomenclature includes environment and tenant information.
Example for test environments - /net/hcinternal/dev/fhm
Example for production - /com/hcentive/prod/fhm
The encryption context is KMS's implementation of additional authenticated data (AAD). It is used to ensure that unencrypted data related to the ciphertext is not tampered with. The encryption context should have all the information to uniquely identify the location of the ciphertext. Ideally, the encryption context should not be hard-coded but derived from the data that is being encrypted. It is a map of key-value pairs.
At a minimum, the following keys are recommended in the encryption context -
tenant- tenant ID or code for a customerproduct- product deployed for the tenantstack- stack in the development lifecycle e.g. dev, qa, production, etc.app- application component e.g. tomcat, batch, jboss, etc.context- identifier for the application e.g. tomcat endpoint, batch job id, etc.
For example, the encryption context for a tomcat server, that needs to decrypt the database password in an application context file, will have the following encryption context -
{
"tenant":"fhm",
"product":"phix",
"stack":"dev",
"app":"tomcat",
"context":"fhm-dev1.hcinternal.net"
}Tenant master keys are protected by security mechanisms implemented using KMS policies and grants along with IAM (Identity and Access Management) policies.
The root account has complete access to all tenant keys in the account. A key administrator role is created per tenant to manage data keys for the tenant. Roles are created based on the level of access required by applications. For example, a role is created for chef/jenkins to encrypt database credentials that go in application server configuration files, and another for application servers to be able to decrypt the credentials.
The root account has access to everything
{
"Sid": "Enable IAM User Permissions",
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::111122223333:root"},
"Action": "kms:*",
"Resource": "*"
}Key administrators are allowed to manage tenant key
{
"Sid": "Allow access for Key Administrators",
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::111122223333:role/fhm-keyadmin"},
"Action": [
"kms:Create*",
"kms:Describe*",
"kms:Enable*",
"kms:List*",
"kms:Put*",
"kms:Update*",
"kms:Revoke*",
"kms:Disable*",
"kms:Get*",
"kms:Delete*",
"kms:CancelKeyDeletion",
"kms:ScheduleKeyDeletion"
],
"Resource": "arn:aws:kms:us-east-1:111122223333:key/3bf54771-g5g9-460g-ced4-d81b2669bxxx"
}Allow deployment servers to generate data keys from the tenant master key
{
"Sid": "Allow use of the key",
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::111122223333:role/fhm-deployment"},
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt",
"kms:GenerateDataKey*",
"kms:DescribeKey"
],
"Resource": "arn:aws:kms:us-east-1:111122223333:key/3bf54771-g5g9-460g-ced4-d81b2669bxxx"
},
{
"Sid": "Allow attachment of persistent resources",
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::111122223333:role/fhm-deployment"},
"Action": [
"kms:CreateGrant",
"kms:ListGrants",
"kms:RevokeGrant"
],
"Resource": "arn:aws:kms:us-east-1:111122223333:key/3bf54771-g5g9-460g-ced4-d81b2669bxxx",
"Condition": {"Bool": {"kms:GrantIsForAWSResource": true}}
} Allow application servers to only decrypt data
{
"Sid": "Allow use of the key",
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::111122223333:role/fhm-appserver"},
"Action": [
"kms:Decrypt"
],
"Resource": "arn:aws:kms:us-east-1:111122223333:key/3bf54771-g5g9-460g-ced4-d81b2669bxxx"
},
{
"Sid": "Allow attachment of persistent resources",
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::111122223333:role/fhm-appserver"},
"Action": [
"kms:CreateGrant",
"kms:ListGrants",
"kms:RevokeGrant"
],
"Resource": "arn:aws:kms:us-east-1:111122223333:key/3bf54771-g5g9-460g-ced4-d81b2669bxxx",
"Condition": {"Bool": {"kms:GrantIsForAWSResource": true}}
} Grants are defined based on constraints applied by encryption context - either to the exact context or a subset. They are typically created programmatically to delegate use of keys to AWS principals. Key policies are used for access control for relatively static assignments policy rules. Grants are used to enable more granular permissions for keys.
Grants can be created on demand for applications when the administrative (provisioning) application encrypts data for an application. For example, when a new application setup is initiated with Jenkins and Chef, the credentials for the new schema are encrypted by the Jenkins and Chef deployment process. The process also creates the relevant grants for the application being provisioned to allow it access to decrypt the credentials.
Jenkins and Chef provisioners encrypt credentials for applications. The application server provisioning recipe generates encrypted credentials, and encrypted data key, at the time of provisioning. It creates the necessary grants for the application server to decrypt credentials. It also updates configuration files with encrypted information, and copies the encrypted data key to the application server.
The KmsEncryptionProvider makes available methods for encrypt and decrypt operations. Application code that needs to encrypt or decrypt data can use this provider instead of implementing the design themselves.
The encrypt method will take plaintext as input, and outputs ciphertext and encrypted data key. The encrypted data key is required by the decrypt method to decrypt the ciphertext.
The provider expects AWS credentials to available in AWS recommended form. It also expects the encryption context, and the AWS key alias for encrypt operations. This information is stored in the ~/.aws directory of the process owner.
Tomcat connects to JDBC data sources using a standard resource factory. The BasicDataSourceFactory creates the Resource from the Context configuration. The KmsDataSourceFctory is a custom resource factory that extends the BasicDataSourceFactory class and overloads the getObjectInstance method to decrypt encrypted password specified in the web application context file using the KmsEncryptionProvider.
The context file specifies the encrypted password in the password attribute, and the data source factory in the factory attribute
<Resource name="jdbc/hcentive-ds-tomcat" auth="Container" type="javax.sql.DataSource"
maxActive="50" maxIdle="30" maxWait="10000"
username="tomcat"
password="CiCDC/7u0b8SMQLLPx4igqW8n3lbugXYo6Qq/daw09qsZxKRAQEBAgB4gwv+7tG/EjECyz8eIoKlvJ95W7oF2KOkKv3WsNParGcAAABoMGYGCSqGSIb3DQEHBqBZMFcCAQAwUgYJKoZIhvcNAQcBMB4GZlS9bE7/eg=jARBAxOPJwjO7URSxzMRecCARCAJYvNMVgWBuaNT/+V92b0aLZNrkA8rsN6iB7XH5iA/ZlS9bE7/eg="
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/hcentive-ds-tomcat"
factory="com.hcentive.util.datasourcefactory.KmsDataSourceFactory" />The KmsDataSourceFactory overrides the getObjectInstance method to decrypt the password using the KmsDataSourceFactory and sets it in an instance of the BasicDataSource class, and returns it to Tomcat
public Object getObjectInstance(Object obj, Name name, Context nameCtx, Hashtable<?, ?> environment) throws Exception {
Object o = super.getObjectInstance(obj, name, nameCtx, environment);
if (o != null) {
BasicDataSource ds = (BasicDataSource) o;
if (ds.getPassword() != null && ds.getPassword().length() > 0) {
final String pwd = KmsEncryptionProvider.decrypt(ds.getPassword());
final String decodedpwd = new String(Base64.getDecoder().decode(pwd.getBytes()));
ds.setPassword(decodedpwd);
}
return ds;
}
return null;
}TBD
TBD
Most applications at hCentive use Hibernate as the database persistence framework. Hibernate provides the UserType interface to implement user-defined types. The hcentive-kms-provider implements the EncryptedStringType, EncryptedDoubleAsStringType and EncryptedBigDecimalAsStringType classes to persist encrypted data. All these classes extend the AbstractEncryptedAsStringType abstract class. The class reads the encryption context from ThreadLocal. Fields in Hibernate domain objects can be annotated as types of these classes to encrypt their values.
The AbstractEncryptedAsStringType class overrides to nullSafeSet method to use the KMSEncryptor to encrypt the field value -
@Override
public void nullSafeSet(PreparedStatement st, Object value, int index,
SessionImplementor session) throws HibernateException, SQLException {
checkInitialization();
if (value == null) {
st.setNull(index, sqlType);
} else {
String stringValue = value == null ? null : value.toString();
st.setString(index, this.encryptor.encrypt(stringValue,
EncryptionContextProvider.getEncryptionContext()));
}
}It also overrides the nullSafeGet method to decrypt the field value -
@Override
public Object nullSafeGet(ResultSet rs, String[] names,
SessionImplementor session, Object owner)
throws HibernateException, SQLException {
checkInitialization();
final String message = rs.getString(names[0]);
return rs.wasNull() ? null : convertToObject(this.encryptor.decrypt(
message, EncryptionContextProvider.getEncryptionContext()));
}The checkInitialization method in AbstractEncryptedAsStringType ensures that the encryptor is set up.
As an example, when defining a domain object, use the following annotations to define custom encrypted types
@TypeDefs({
@TypeDef(name = "encryptedString", typeClass = EncryptedStringType.class),
@TypeDef(name = "encryptedDoubleAsString", typeClass = EncryptedDoubleAsStringType.class),
@TypeDef(name = "encryptedBigDecimalAsString", typeClass = EncryptedBigDecimalAsStringType.class)
})Annotate fields in a domain object definition to specify an encrypted type
@Type(type = "encryptedBigDecimalAsString")
@Column(name = "salary")
private BigDecimal salary;AWS S3 supports both server-side and client-side encryption using KMS. With server-side encryption the client requests S3 to the object before storing it in the bucket, and decrypts it when the client downloads the object. With client-side encryption, the client encrypts the data before uploading it to S3; decryption is performed by the client as well after download.
S3 buckets are tenant specific. When a bucket is created, a tenant master key is specified to encrypt it's objects. Also, a bucket policy is create to deny unencrypted object uploads. The [PUT object|http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT.html] request should include the x-amz-server-side-encryption set with a value of aws:kms.
{
"Version":"2012-10-17",
"Id":"PutObjPolicy",
"Statement":[{
"Sid":"DenyUnEncryptedObjectUploads",
"Effect":"Deny",
"Principal":"*",
"Action":"s3:PutObject",
"Resource":"arn:aws:s3:::fhm/*",
"Condition":{
"StringNotEquals":{
"s3:x-amz-server-side-encryption":"aws:kms"
}
}
}
]
}The encryption context of the object is set by setting the amz-server-side-encryption-context header with a value that will identify the object. S3 adds the pre-defined key aws:s3:arn to the encryption context with the value equal to the object's ARN. This pre-defined value guarantees that the encryption context is not identical between two different S3 objects.
Jenkins is the deployment executor for all deployments. It runs the builds and deployment jobs to set up application instances for all projects. It is also used to generate the encrypted password for application configurations.
The jenkins user on the instance has permissions to encrypt passwords for applications and application servers. It generates the encrypted password and sets it in the application configuration.
The jenkins instance can be launched with an IAM role that has permissions to encrypt passwords for all products for which it provisions servers.
Or, AWS credentials can be set up for the jenkins user on the instance to allow the user to encrypt passwords. It should the following environment variables set
AWS_CONFIG_HOME=/home/jenkins/.aws
AWS_CONFIG_FILE=/home/jenkins/.aws/config
AWS_CREDENTIAL_PROFILES_FILE=/home/jenkins/.aws/config
AWS_ACCESS_KEY_ID=xxxxxxx
AWS_SECRET_ACCESS_KEY=xxxxxxx
AWS_REGION=us-east-1
AWS_DEFAULT_REGION=us-east-1
AWS_KMS_KEY_SPEC=AES_256For both options, the user making the KMS API calls (either from the command-line or the SDK) should have permissions to use the keys.
The encryption key alias and context are derived from instance metadata and job details. For example, if the deployment is for a dev server in the PHIX product, KMS key alias is
alias/net/hcinternal/dev/phix
And, it's encryption context is set as
{
"product":"phix",
"tenant":"product",
"stack":"dev",
"context":"phix-1.dev.hcinternal.net"
}This is used in the kms command to generate the encrypted password. For example, if the database password for a PHIX dev server is password (bad idea), then the command to encrypt the password will be
$ aws kms encrypt --key-id $AWS_KMS_KEY_ALIAS --region us-east-1 --encryption-context $AWS_KMS_ENCRYPTION_CONTEXT --plaintext password --output json --query CiphertextBlob --profile phix-dev-kms-tomcat-admin | sed 's/"//g'The output will be something like this
CiCDC/7u0b8SMQLLPx4igqW8n3lbugXYo6Qq/daw09qsZxKPAQEBAgB4gwv+7tG/EjECyz8eIoKlvJ95W7oF2KOkKv3WsNParGcAAABmMGQGCSqGSIb3DQEHBqBXMFUCAQAwUAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAygboga3ffjKv7rhaACARCAI7lecYwCW0eQpENQcEs7ew4F0ySnA7hwBgREmJxC/sRlhNZd
This encrypted password is set in the application configuration.
The instance running the application server should be set up with AWS credentials. AWS instances can be launched with an IAM role to allow instances to make KMS API calls without having individual credentials for every instance. The KMS key policy should grant access to the role to decrypt passwords using the tenant key.
The other way to set up an instance to be able to make KMS API call is by using AWS API credentials. This is also the only way to allow non-AWS instances (like Armor) to make KMS API calls. These credentials are normally stored in environment variables or in the default credentials file in the .aws directory of the process owner's home folder. Details on setting up credentials are available in the providing AWS Credentials in the AWS SDK for Java page in AWS documentation.
KMS requires a [key alias]((http://docs.aws.amazon.com/kms/latest/developerguide/programming-aliases.html), and an encryption context in json format. For example -
{
"product":"phix",
"tenant":"product",
"stack":"dev",
"context":"phix-1.dev.hcinternal.net"
}For example, if using environment variables to specify AWS API credentials, the list of variables will look like this
AWS_CONFIG_HOME=/home/tomcat/.aws
AWS_CONFIG_FILE=/home/tomcat/.aws/config
AWS_CREDENTIAL_PROFILES_FILE=/home/tomcat/.aws/config
AWS_ACCESS_KEY_ID=xxxxxxx
AWS_SECRET_ACCESS_KEY=xxxxxxx
AWS_REGION=us-east-1
AWS_DEFAULT_REGION=us-east-1KMS specific environment variables can specified in the setenv.sh script that is run by Tomcat at startup to set environment variables.
AWS_KMS_KEY_ALIAS=alias/net/hcinternal/dev/phix
AWS_KMS_ENCRYPTION_CONTEXT='{"product":"phix","tenant":"product","stack":"dev","context":"phix-1.dev.hcinternal.net"}'
AWS_KMS_KEY_SPEC=AES_256NOTE: The AWS credentials have permissions to only decrypt the password. This prevents a user on the applications from re-encrypting a password. AWS instances that are launched with an IAM role will not require credentials to be set in the environment.
The Tomcat context file specifies the encrypted password, generated by jenkins, in the password attribute, and the data source factory in the factory attribute
<Resource name="jdbc/hcentive-ds-tomcat" auth="Container" type="javax.sql.DataSource"
maxActive="50" maxIdle="30" maxWait="10000"
username="tomcat"
password="CiCDC/7u0b8SMQLLPx4igqW8n3lbugXYo6Qq/daw09qsZxKPAQEBAgB4gwv+7tG/EjECyz8eIoKlvJ95W7oF2KOkKv3WsNParGcAAABmMGQGCSqGSIb3DQEHBqBXMFUCAQAwUAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAygboga3ffjKv7rhaACARCAI7lecYwCW0eQpENQcEs7ew4F0ySnA7hwBgREmJxC/sRlhNZd"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/hcentive-ds-tomcat"
factory="com.hcentive.util.datasourcefactory.KmsDataSourceFactory" />TBD
TBD
- Metadata service required for non-AWS infrastructure