1.0

.travis.yml

@@ -1,3 +1,4 @@
+sudo: false
 language: python
 python:
     - "2.6"
@@ -8,9 +9,14 @@ env:
     - TOX_ENV=py27
     - TOX_ENV=py34
     - TOX_ENV=flake8
-before_install:
-    - sudo apt-get update
-    - sudo apt-get install -y libyaml-dev python-dev libffi-dev python-pip gnupg
+addons:
+    apt:
+        packages:
+            - libyaml-dev
+            - python-dev
+            - libffi-dev
+            - python-pip
+            - gnupg
 install:
     - pip install tox
     - pip install -r requirements.txt

Makefile

@@ -79,6 +79,8 @@ functional-tests-once:
 		python sops/__init__.py -e -p "1022470DE3F0BC54BC6AB62DE05550BC07FB1A0A" /tmp/testdata.$$type > /tmp/testdataenc.$$type; \
 		echo "Testing $$type re-decryption" && \
 		python sops/__init__.py -d /tmp/testdataenc.$$type > /dev/null || exit 1; \
+		echo "Testing removing PGP key to $$type encrypted file" && \
+		python sops/__init__.py -r --rm-pgp 85D77543B3D624B63CEA9E6DBC17301B491B3F21 /tmp/testdataenc.$$type || exit 1; \
 	done
 
 pypi: tests functional-tests

README.rst

@@ -71,7 +71,7 @@ Your AWS credentials must be present in `~/.aws/credentials`. sops uses boto3.
 
 .. code::
 
-	$ cat ~/.aws/credentials 
+	$ cat ~/.aws/credentials
 	[default]
 	aws_access_key_id = AKI.....
 	aws_secret_access_key = mw......
@@ -156,15 +156,33 @@ steps, apart from the actual editing, are transparent to the user.
 Adding and removing keys
 ~~~~~~~~~~~~~~~~~~~~~~~~
 
-When creating a new files, `sops` uses the PGP and KMS defined in the command
+When creating new files, `sops` uses the PGP and KMS defined in the command
 line arguments `--kms` and `--pgp`, or from the environment variables
 `SOPS_KMS_ARN` and `SOPS_PGP_FP`. That information is stored in the file under
-the `sops` section. When editing a file, it is trivial to add or remove keys:
-invoke `sops` with the flag **-s** to display the master keys while editing, and
-add or remove kms or pgp keys under the sops section.
+the `sops` section, such that decrypting files does not require providing those
+parameters again.
 
-For example, to add a KMS master key to a file, we would add the following
-entry:
+Master PGP and KMS keys can be added and removed from a `sops` file in one of
+two ways: by using command line flag, or by editing the file directly.
+
+Command line flag `--add-kms`, `--add-pgp`, `--rm-kms` and `--rm-pgp` can be
+used to add and remove keys from a file. These flags use the comma separated
+syntax as the `--kms` and `--pgp` arguments when creating new files.
+
+.. code:: bash
+
+	# add a new pgp key to the file and rotate the data key
+	$ sops -r --add-pgp 85D77543B3D624B63CEA9E6DBC17301B491B3F21 example.yaml
+
+	# remove a pgp key from the file and rotate the data key
+	$ sops -r --rm-pgp 85D77543B3D624B63CEA9E6DBC17301B491B3F21 example.yaml
+
+Alternatively, invoking `sops` with the flag **-s** will display the master keys
+while editing. This method can be used to add or remove kms or pgp keys under the
+sops section.
+
+For example, to add a KMS master key to a file, add the following entry while
+editing:
 
 .. code:: yaml
 
@@ -184,6 +202,10 @@ When the file is saved, `sops` will update its metadata and encrypt the data key
 with the freshly added master keys. The removed entries are simply deleted from
 the file.
 
+When removing keys, it is recommended to rotate the data key using `-r`,
+otherwise owners of the removed key may have add access to the data key in the
+past.
+
 Assuming roles and using KMS in various AWS accounts
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -244,15 +266,13 @@ Key Rotation
 ~~~~~~~~~~~~
 
 It is recommended to renew the data key on a regular basis. `sops` supports key
-rotation via the `-r` flag. A simple approach is to decrypt and reencrypt all
-files in place with rotation enabled:
+rotation via the `-r` flag. Invoking it on an existing file causes sops to
+reencrypt the file with a new data key, which is then encrypted with the various
+KMS and PGP master keys defined in the file.
 
 .. code:: bash
 
-	for file in $(find . -type f -name "*.yaml"); do
-		sops -d -i $file
-		sops -e -i -r $file
-	done
+	sops -r example.yaml
 
 Examples
 --------
@@ -323,10 +343,10 @@ In-place encryption/decryption also works on binary files.
 	$ sha512sum /tmp/somerandom
 	9589bb20280e9d381f7a192000498c994e921b3cdb11d2ef5a986578dc2239a340b25ef30691bac72bdb14028270828dad7e8bd31e274af9828c40d216e60cbe /tmp/somerandom
 
-	$ sops -e -i /tmp/somerandom 
+	$ sops -e -i /tmp/somerandom
 	please wait while a data encryption key is being generated and stored securely
 
-	$ sops -d -i /tmp/somerandom 
+	$ sops -d -i /tmp/somerandom
 
 	$ sha512sum /tmp/somerandom
 	9589bb20280e9d381f7a192000498c994e921b3cdb11d2ef5a986578dc2239a340b25ef30691bac72bdb14028270828dad7e8bd31e274af9828c40d216e60cbe /tmp/somerandom
@@ -550,7 +570,7 @@ systems. Not unlike many other organizations that operate sufficiently complex
 automation, we found this to be a hard problem with a number of prerequisites:
 
 1. Secrets must be stored in YAML files for easy integration into hiera
-	    
+
 2. Secrets must be stored in GIT, and when a new CloudFormation stack is
    built, the current HEAD is pinned to the stack. (This allows secrets to
    be changed in GIT without impacting the current stack that may
@@ -611,7 +631,7 @@ The security of the data stored using sops is as strong as the weakest
 cryptographic mechanism. Values are encrypted using AES256_GCM which is the
 strongest symetric encryption algorithm known today. Data keys are encrypted
 in either KMS, which also uses AES256_GCM, or PGP which uses either RSA or
-ECDSA keys. 
+ECDSA keys.
 
 Going from the most likely to the least likely, the threats are as follows:
 

example.json

@@ -1,51 +1,51 @@
 {
-    "firstName": "ENC[AES256_GCM,data:Sf9dCw==,iv:OtsxqCFAvsDfiUIu+FmMT+9SZZ+hwFXxWAoA/fFt4n0=,tag:T+mdTNgbGHiNksabARl0SQ==,type:str]",
-    "lastName": "ENC[AES256_GCM,data:8CSE1Fc=,iv:ZwNczZao5fK44uYH+TU+RwXSC6OHjbBWCrQiO97Ws3I=,tag:MzyRtngiN63kfHNkBttpKw==,type:str]",
-    "age": "ENC[AES256_GCM,data:UHBRAg==,iv:79skTitX1Tq8EtiyAYeP6Ir8dAkiporJGSHqRThHh5g=,tag:HJC3yTJS9jqncxqmMbAuCA==,type:float]",
+    "firstName": "ENC[AES256_GCM,data:RwGNjw==,iv:nZqLELdF3m03J0udkBhhbT/ob2NxGv8LtuoInvilN4A=,tag:bln64G53ls2V+StjeoPDKg==,type:str]", 
+    "lastName": "ENC[AES256_GCM,data:lUfbfis=,iv:LOKFqXfnOqcGMZ1J50tjHBcUazhVlnAn89kD+vYFzwE=,tag:NGQReEHn+XV49SVaAsfpSA==,type:str]", 
+    "age": "ENC[AES256_GCM,data:If+rEA==,iv:2IoWeDsYgRyc83C08Hw1BMyDwVq2ybMDjLEYr6Yz9AA=,tag:IMU3jtJXQK+oyOxTmQpqFA==,type:float]", 
     "address": {
-        "city": "ENC[AES256_GCM,data:DKo/DSI8QjU=,iv:ZVT8sB8Lq7Q1l4kRmEpjq78BLXL6VSG5Wl+s0skKz9k=,tag:Dtv5h2tivM4E1T/JQmhiwQ==,type:str]",
-        "postalCode": "ENC[AES256_GCM,data:DxjkWjslhRKFeA==,iv:jZYRetIj1Brxj0Dhc6e06NOwQt4nR0wW6iVRN/n5SwI=,tag:ajqvdmJwFwsMxbLx03/x3Q==,type:str]",
-        "state": "ENC[AES256_GCM,data:haM=,iv:dZlMji6974EpdMsW+ZF6kGt4cUG2jJiz1mANZLZaMhU=,tag:9F2gABkPf1M7h6Sx9h34Fg==,type:str]",
-        "streetAddress": "ENC[AES256_GCM,data:KnPa8Gihd9+dHcXZZg==,iv:KA/JWp/fW0BaTvRlc0SHYZPtdVU6Jzryp8L5CHo1a4I=,tag:bb1hmGbEMRbQxi4NITR5iw==,type:str]"
-    },
+        "city": "ENC[AES256_GCM,data:3Y42Y91x/ZQ=,iv:IQ/L3sd0ZbM3n+CH3o9TBBYwzLM/r83sBD/Mk3pZ2/A=,tag:xbVTKiDl699bnIGvB4j8xA==,type:str]", 
+        "postalCode": "ENC[AES256_GCM,data:/QhEsQu7YWfguA==,iv:7lTIgG333LaEyBcoasevpLV3bHindHHPaoAU8gkTRz4=,tag:mwGWiTxdPIipcasTD0wUyA==,type:str]", 
+        "state": "ENC[AES256_GCM,data:5II=,iv:hX/U4tlW3plzX78l5H6gN3ej/OD5Zi9TQM7FA7+xayg=,tag:wOpKTLMNmBzogZc4QNwo4Q==,type:str]", 
+        "streetAddress": "ENC[AES256_GCM,data:yWSOIEzJEa5jxQL/0w==,iv:9iXFWOcHy5nc/7Qr+I38NAP4nyF1qiruK18bPFAQq4U=,tag:fAVNUdxLUObCplJl5mqROw==,type:str]"
+    }, 
     "phoneNumbers": [
         {
-            "number": "ENC[AES256_GCM,data:qgbbyAXoBbkDr1bA,iv:Y8Z2nBp2yV6ldfAU9Zjsb6gCBLQrNMEqvkwSSZ3Y2Z4=,tag:DmGMtmhmnuWYwhvxvihelQ==,type:str]",
-            "type": "ENC[AES256_GCM,data:29QEQA==,iv:x+GSbhrTvvNj46Kv1FE1bghPBBAm37sLJVMuclg1OnM=,tag:ZgB+OcAKYZ3vPzy5hKOWWQ==,type:str]"
-        },
+            "number": "ENC[AES256_GCM,data:qfn79FKKwgeD1oxO,iv:j0RdoEFzHFI1u1goU8AKRCO0dxBZbvEhvNgVQueVsBg=,tag:RzheQ5/nGjjnw8iCLnpwFA==,type:str]", 
+            "type": "ENC[AES256_GCM,data:M9V4UQ==,iv:K9fM2vFwttRoq+97oAcomxeAiR2u3peWlguk6JrHiUI=,tag:5dbFPjEPvVOnO8C5Y43a0A==,type:str]"
+        }, 
         {
-            "number": "ENC[AES256_GCM,data:z+CGrbAnrTwABu8b,iv:w5BfJFjJtIoXtTbkhhbRsGNP9cvhYiRIzhxay6WIjbs=,tag:eaSOt8CLk1w21uE7+2I0Tw==,type:str]",
-            "type": "ENC[AES256_GCM,data:e/dNOAmq,iv:ZFoDttfnZIeHnDfbIzT9t2UgLK/0Bf3oFJ1CmN+Ovco=,tag:B1BUM+UwZdCjOgIX52FDTw==,type:str]"
+            "number": "ENC[AES256_GCM,data:LZIOa1sP4Epa1y/3,iv:8xXFqs4hxC0JoPrXIxim00sqYxW5bAF+u9AItq10xtA=,tag:9jDYNB+p2MB/XX/zH468zQ==,type:str]", 
+            "type": "ENC[AES256_GCM,data:w8QcDV2m,iv:qfvYhtK1qvTTiX/AzGm6nTFHzne1DlVIkC8Xfkb8lDo=,tag:sdV1MDOEptfoyooTQHjC6g==,type:str]"
         }
-    ],
+    ], 
     "sops": {
-        "mac": "ENC[AES256_GCM,data:DA4H2c++XgR3Oy8LThU70sIGfd11jdy+7vSs0b2n1bEWd6XgCkCEJWDpy58CJPISJY/y8iOCyEh/oeOaD0/Y+qqoOxAeK/FQmXDpKRQrrSxPcn3nbAMbkaTvTaPwjrgtJXH2cN0lQwlPF+s9FifZXKJe0aGwj7jbxjfWXaEc6pk=,iv:iwbQBuoEaEWLhA1TYeuoxFoO1ITHGg2qeciD4aUA2pw=,tag:oz9bhv60yOiQ8IxurJw0Zw==,type:str]",
-        "version": 0.90000000000000002,
+        "mac": "ENC[AES256_GCM,data:zfsUtJMJ1ZSYbug+0seq/ZF8dS6SM4yBfQMZRqgFw+JXayG/9jkWSgjwF5pmGoLvmaWgSq7AlCwbFhiP8ZYJxkf3zsDzg0sj3/k4VMblx+su+QWVOeF0CqevmdyDi24cNlR6YQ9f3Pbzpd339rC8EXHueU31WXgiItpxhsAp5rQ=,iv:uG/tSi4GhH/U8DfSpnDcbD2CENTEGyiaVSg9VZW+tJg=,tag:nKvnL759yqgan7lmzDLE2Q==,type:str]", 
+        "version": 1.0, 
         "kms": [
             {
-                "created_at": 1444233149.7954021,
-                "enc": "CiC6yCOtzsnFhkfdIslYZ0bAf//gYLYCmIu87B3sy/5yYxKnAQEBAgB4usgjrc7JxYZH3SLJWGdGwH//4GC2ApiLvOwd7Mv+cmMAAAB+MHwGCSqGSIb3DQEHBqBvMG0CAQAwaAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAwO9pCAxCN0oznQ7x8CARCAOxrIQYZ7J8/aCCnLUf0zLqL96AwfyYS76+g51sLaQlNTMqNGfslT6cZmw24CdsNrvtz8QypP74+pM7Xd",
+                "created_at": "2015-11-25T14:34:39Z", 
+                "enc": "CiC6yCOtzsnFhkfdIslYZ0bAf//gYLYCmIu87B3sy/5yYxKnAQEBAgB4usgjrc7JxYZH3SLJWGdGwH//4GC2ApiLvOwd7Mv+cmMAAAB+MHwGCSqGSIb3DQEHBqBvMG0CAQAwaAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAzZr3EKR6/6z7Bg0MECARCAOyKbAecKpjU5xADsXg3XLgqs10pr7t9CjpSgYYw/oq3IkSMhED+jZ5RzpRByMSOcl7XOPVShTBP0UROI", 
                 "arn": "arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e"
-            },
+            }, 
             {
-                "created_at": 1444233151.305619,
-                "enc": "CiBdfsKZbRNf/Li8Tf2SjeSdP76DineB1sbPjV0TV+meTxKnAQEBAgB4XX7CmW0TX/y4vE39ko3knT++g4p3gdbGz41dE1fpnk8AAAB+MHwGCSqGSIb3DQEHBqBvMG0CAQAwaAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAxyg2xy9gTYriI3dBgCARCAO2NVWrAab3DY5GdcLzNxTm8wKkyn/8km/5mxGWZX5zerOgZjXsyFAUW9plckQjRAe1JeXbSjhZq5ev/k",
+                "created_at": "2015-11-25T14:34:39Z", 
+                "enc": "CiBdfsKZbRNf/Li8Tf2SjeSdP76DineB1sbPjV0TV+meTxKnAQEBAgB4XX7CmW0TX/y4vE39ko3knT++g4p3gdbGz41dE1fpnk8AAAB+MHwGCSqGSIb3DQEHBqBvMG0CAQAwaAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAxrelu7r4H0CTrjEl4CARCAOxg41AwzpQPlfAKPAg9EJCF2jl1j61m1hAW3GstrT90j9xTVHsG21xrJHcQOPK/+X/AEr5fjaMLPMEnq", 
                 "arn": "arn:aws:kms:ap-southeast-1:656532927350:key/9006a8aa-0fa6-4c14-930e-a2dfb916de1d"
             }
-        ],
+        ], 
         "pgp": [
             {
-                "fp": "1022470DE3F0BC54BC6AB62DE05550BC07FB1A0A",
-                "enc": "-----BEGIN PGP MESSAGE-----\nVersion: GnuPG v1\n\nhIwDEEVDpnzXnMABBACqAUiwqLNn7w7WyQ6J24oIxPC/9Hx5NmqWLperCA2W3lIM\ndvW8WHP10MsbPaj7CWCkillppKmxkuQqivRvgcxWeSMXbotlnCAfczwDCpf1o1/T\nJctzN4qSBlCpEF+2OyRTEVz22Zd7UfOZqzoJ4e7yA3WiLpe47X5YhpPie0HClNJe\nAZ1zPj9zvAuHLf1ZRthuLwpM40cyjfPmPI0jDknUpfvWV6GueXcozSuJEWJcVBKn\nDyR3mZfWpxPee0CVmuqTMT8OIM2p+5uNNrNWqy5eM8nhY6lRyMxN915xccN36g==\n=LxSJ\n-----END PGP MESSAGE-----\n",
-                "created_at": "2015-10-08T15:33:31Z"
-            },
+                "fp": "1022470DE3F0BC54BC6AB62DE05550BC07FB1A0A", 
+                "enc": "-----BEGIN PGP MESSAGE-----\nVersion: GnuPG v1\n\nhIwDEEVDpnzXnMABA/94OKD8wELXxP6gd+RdirSKvvBhtbyNMDHrkyju75OOqtgw\nnLLAmTq0sea2MZ5IdhhoudVLUbEHJqZDZyLD5KdKCGdS7psAEXjp7usQNNy0mp9G\nIXcjvM2RZArQYwkUYDHYzCbF1vI4W3zdsof8HV9DrJU3St1OzLNICnojYDrk4tJe\nAYKKHiIu3wq9NzOGaElsdpVj1yIAcVTXMU6fV76Z+fQmxQwtz3VDTzrNGcQzGMg5\nM2jkjHENFrYtNIJdobNGftnjcpzTruYxTMp0ysNuJPznqHH2jVO3XodRR0rLlw==\n=cBux\n-----END PGP MESSAGE-----\n", 
+                "created_at": "2015-11-25T14:34:39Z"
+            }, 
             {
-                "fp": "85D77543B3D624B63CEA9E6DBC17301B491B3F21",
-                "created_at": 1444233151.3096631,
-                "enc": "-----BEGIN PGP MESSAGE-----\nVersion: GnuPG v1\n\nhQIMA0t4uZHfl9qgARAAgQdMpnTNMCdbdFRpBsC9kxi334LbBrFUkp5lI+YzutZy\nSic85ea06FGL3O93tII9mwGAsESwKlN4nX0d31vuh/lYxMDakyd1IK/BkMG4Z1xG\n52MsACG/pyitMBXkIIyjmR0tVR+CixDsy5cUJxoWq+mfuE2ywziPY+KbEZ50hFXg\naAdKCdInXlLHdId+aXhThhXUGN1seQjtdyZjVXnp8c9hHS2YQdyp/SZf47NJ4A2y\nkO40kNS4oaHUUZIZLtzaFhWytZlpWEJJkIgH/vefL3jLW4SiIiqz24wr7MncsF+A\np8Pteulc5VrvA5CzQIq9qF3Zwn9HV2a0KWLZ/J29EYzSM8u9HLOYqsmNKt0TcVbX\n6eoG3JTJoRDrzO0DZvR3pMm4gQ0WXzHKzpu8g+JYnoQ19AMWJAPbTp5ej3MWHcXD\nXFjz4gsSYbwc4h/zVBOWsYoHlyTLUMwg2BA1YiL89xs8MIhIHOAmvM0mv+QuZQ7S\nCfc1mS04CZSmJvTcNkvE5n76n2iXs6nYNk8TYyQlhYebuQmJQKJuUYjKIHhuxZFa\n30WaSGnKHqIQn1pl7jqyqm8sVTzaKMyhbM0T+UQUJhXcWVr7r+CtRAt8XjVnJMvo\nviJwTWy1Ddo0Vu1licMFJXMnQbQlVh+CZS6FHqcbxfPaYfe7JldGmhwKg+F/NEHS\nXgEf78iLm3FNb4yeOkB/z2xjiZ3XvUAQjsUK5ofF1CJYcQ//YIFex1oO55Z0+qIt\njdDtqivLgf4SFRf0uhOxUrQNuFAvY361F1mvrGPcTubh/Ygq0aVzWzgC9gn7DTo=\n=uQw0\n-----END PGP MESSAGE-----\n"
+                "fp": "85D77543B3D624B63CEA9E6DBC17301B491B3F21", 
+                "created_at": "2015-11-25T14:34:39Z", 
+                "enc": "-----BEGIN PGP MESSAGE-----\nVersion: GnuPG v1\n\nhQIMA0t4uZHfl9qgAQ//d2ZWmkv6LbR4aIw8z4FFs4PyOMWqMS8jhFvmAcS7u0m3\ni1rsuUduD5+PM7EGJo8MibaQL1eaHIobeE6VhXDRkMWBt6vF8sk71H1+JZ6tdOyy\n7rxWFs7B2pn3WakbGlFqrF19gAqYKoUeK938eAika2SwZIFcRv5/nxv6+FZxwZJe\ngSieoGBlaDIHoLVRWCGHQSXAt+mItBJPSgP+jHwUGYvoi0IKUwIXqXQn+jWuFJC5\n3xLr8P3LaMNG9E/7492ZsVZ9F6pX3pNWD5cUd0ThaHrBaEDxWxH3OSfl2axbGTUb\n2+KnHZ274Omf1OzaXUqHPtE8aXcloESBr9jEZe7P8VWQ1PR8lDSSgjQtJ86NrVEx\nmAm4ENtTF/pb9l3IXacNpXZBrHm7JkA6kdDZFS8fF63FnAj23Efzks4IZmKLuRM7\ngFxv07xyKgPnEQKBbedM2XTk+PF3yI9JTTHNY57EIxzbc/V+16HYoC0tvwJ3KgpP\nBh3LcoroKwNaAMbCaLaeqmyIVmrpV0mLwneRlN3yKWdk7z6tP8ONb7T/mKtMjZIh\naStCZxs/Au+UUVt15vZb8w+qzBrfzKjad/+ilbGNu+/48oV13/J4Wq3xymdR5U0e\np2qiJp9X7/EB2ZSp7dKfbYDsIcUrOL8/94ygl1xm7GcwwJOnZ5pCeTz2sLVj5cPS\nXgFuzwS2S02vdHyE3NQ7Q//eX6sIPtb1+nrdIITkrtemUXwvgPL38+J+4xUtXTLC\nu3ZvfbehSYx3sKdRpuR789zK7+aOwbNqbZTFaMMj9eNQzhXl94cXMqCUrBl3Y+I=\n=LXE7\n-----END PGP MESSAGE-----\n"
             }
-        ],
-        "lastmodified": "2015-10-26T18:15:31Z",
+        ], 
+        "lastmodified": "2015-11-25T14:34:39Z", 
         "attention": "This section contains key material that should only be modified with extra care. See `sops -h`."
     }
 }