From 1aaa05b7f115e3141821b6dee99a6a0b511b8397 Mon Sep 17 00:00:00 2001 From: Greg Surbey Date: Tue, 16 Aug 2016 14:14:52 -0400 Subject: [PATCH 1/6] Adding to and improving the Purpose paragraph in the Readme document. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 9740e0d..7b5fbe3 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ --- #### Purpose: - This is a first pass at a service that validates container identity from Rancher and then provides access to Vault. Rancher nor this service actually manage secrets in Vault, that is still left to the user and Vault. What this service will do is create Vault Tokens with policies allowed by the token provided to the service. The token obtained from this service is then used to communicate directly with Vault. +The Secrets Bridge service is a standardized way of integrating Rancher and Vault such that Docker containers at startup are securely connected with their secrets within Vault. The Secrets Bridge service is composed of a server and agents. At container startup, the service first validates the container's identity with Rancher, and then provides the container with access to Vault. Neither Rancher nor the service actually manages any secrets within Vault; that is still left to the user and Vault. What this service will do is create Vault Tokens which are assigned a subset of policies allowed by the initial grantor-default token provided to the Secrets Bridge server at startup. The app token obtained through this service is then used by the container to communicate directly with Vault. This allows a user to define a custom process in their containers that can inject the secrets it reads from Vault into the app that ultimately uses them, using whatever custom input methods required by the user's app. #### How it works: From a6dc84524757fdb031820e72889f3abb78e97e31 Mon Sep 17 00:00:00 2001 From: Greg Surbey Date: Tue, 16 Aug 2016 14:37:31 -0400 Subject: [PATCH 2/6] Small grammar improvements in the Readme document. --- README.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 7b5fbe3..4173bab 100644 --- a/README.md +++ b/README.md @@ -3,9 +3,9 @@ ### Status: Experimental POC (Read: Do NOT use for production) #### To Dos: - * Create catalog entry + * Create catalog entry. * Make work with TLS production Vault setup (currently only works with a Dev Vault configuration). - * Add support for K8s and Swarm + * Add support for K8s and Swarm. * Cattle needs signature verification call. @@ -15,17 +15,17 @@ The Secrets Bridge service is a standardized way of integrating Rancher and Vaul #### How it works: -In Vault, a user will create a Role for this service, scoping to an environment is probably a good idea. This Role should be assigned all of the Vault policies you need it to create tokens for. Vault only lets you create tokens for a subset of your own assigned tokens. +In Vault, a user will create a Role for this service; scoping to an environment is probably a good idea. This Role should be assigned all of the Vault policies you need it to create tokens for. Vault only lets you create tokens for a subset of your own assigned tokens. -To accomplish this, you need to create some default policies for the `secrets-bridge` along with a Vault Cubbyhole. See Vaults documentation for more details on Cubbyholes, this service relies heavily on them. +To accomplish this, you need to create some default policies for the `secrets-bridge` along with a Vault Cubbyhole. See Vaults documentation for more details on Cubbyholes, as this service relies heavily on them. A service would then be deployed into an operators/tools/non-application environment. This item will likely be launchable from the catalog. Once the server side service is deployed, you would then deploy the agents into your application environment. These agents then listen for Docker events and send container start events to the service. -The service then verifies with Rancher (see notes/todos below) the containers Identity. If the identity can not be verified, then nothing else happens. If the container is verified, the service checks for a policy key set on the config service tokens config path. If a policy is found, it then generates a temporary token to create a Cubbyhole and a permanent token with the applied policy. The permanent key is placed into the Cubbyhole with the temporary key, which will have a short TTL and 1 more use to get the permanent key. +The service then verifies with Rancher (see notes/todos below) the container's Identity. If the identity can not be verified, then nothing else happens. If the container is verified, the service checks for a policy key set on the config service tokens config path. If a policy is found, the service then generates a temporary token to create a Cubbyhole and a permanent token with the applied policy. The permanent key is placed into the Cubbyhole with the temporary key, which will have a short TTL and 1 more use to get the permanent key. -The response to the agent contains, the Docker container ID (from Rancher), the Vault path of the Cubbyhole and the Temporary token. From this, the Agent does a Docker copy of the Cubbyhole information to: `/tmp/secrets.txt` on the target container. +The response to the agent contains: the Docker container ID (from Rancher), the Vault path of the Cubbyhole, and the Temporary token. From this, the Agent performs a Docker copy of the Cubbyhole information to: `/tmp/secrets.txt` inside the target container. A process inside the container can then read those credentials to get the permanent key and use Vault as it normally would. @@ -66,7 +66,7 @@ path "secret/*" { capabilities = ["deny"] } ``` -`Note: default is lower-cased within name grantor-default b/c the vault write command converts the name to lowercase. And the name within the auth/token/create/ must be consistent for path searches.` +`Note: default is lower-cased within name grantor-default b/c the vault write command converts the name to lowercase. And the name within the auth/token/create/ must be consistent for path searches.` This policy gives the `grantor-default` role the ability to grant tokens. @@ -117,7 +117,7 @@ export VAULT_ADDR=http://xxx.xxx.xxx.xxx:$VAULT_PORT export ROOT_TOKEN=62c08fb4-e635-6a2d-f315-002e374e2ff1 export RANCHER_ENVIRONMENT_API_URL=http://xxx.xxx.xxx.xxy:XXXX/v1/projects/YYY ``` -Set RANCHER_ENVIRONMENT_API_URL to the URL of API key for the Rancher Environment being used. For example, RANCHER_ENVIRONMENT_API_URL=http://192.168.101.128:8080/v1/projects/1a5 +Set RANCHER_ENVIRONMENT_API_URL to the URL of API key for the Rancher Environment being used. For example, RANCHER_ENVIRONMENT_API_URL=http://192.168.101.128:8080/v1/projects/1a5 ##### Step 4: Create grantor-default role @@ -134,12 +134,12 @@ vault write secrets/secrets-bridge/Default/Stack2/app2 policies=default,app2 ##### Step 6: Configure Vault for Secrets-Bridge startup -Start but creating a temporary token with a TTL of 15m and a max usage of 2 attempts (1st used to place permanent token within cubbyhole; 2nd usage is when secrets-bridge starts up and contacts Vault to get permanent token) +Start by creating a temporary token with a TTL of 15m and a max usage of 2 attempts (1st used to place permanent token within cubbyhole; 2nd usage is when secrets-bridge starts up and contacts Vault to get permanent token) ``` TEMP_TOKEN=$(curl -s -H "X-Vault-Token: $ROOT_TOKEN" ${VAULT_URL}/v1/auth/token/create -d '{"policies": ["default"], "ttl": "15m", "num_uses": 2}' | jq -r '.auth.client_token') ``` -Then create a permanent token for the grantor-default role. This token will be used by the secrets-bride to interact with Vault and create temp tokens for applications. +Then create a permanent token for the grantor-default role. This token will be used by the secrets-bridge to interact with Vault and create temp tokens for applications. ``` PERM_TOKEN=$(curl -s -X POST -H "X-Vault-Token: $ROOT_TOKEN" ${VAULT_URL}/v1/auth/token/create/grantor-default -d '{"policies": ["default", "grantor-default", "app1", "app2"], "ttl": "72h", "meta": {"configPath": "secret/secrets-bridge/Default"}}' | jq -r '.auth.client_token') From d74a2f4f42621ac8ec12ee98df3057fe65e420fa Mon Sep 17 00:00:00 2001 From: Greg Surbey Date: Tue, 16 Aug 2016 14:52:49 -0400 Subject: [PATCH 3/6] Swapping the order of operations for the creation of PERM_TOKEN and TEMP_TOKEN in the Readme document. --- README.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 4173bab..779fcae 100644 --- a/README.md +++ b/README.md @@ -134,18 +134,19 @@ vault write secrets/secrets-bridge/Default/Stack2/app2 policies=default,app2 ##### Step 6: Configure Vault for Secrets-Bridge startup -Start by creating a temporary token with a TTL of 15m and a max usage of 2 attempts (1st used to place permanent token within cubbyhole; 2nd usage is when secrets-bridge starts up and contacts Vault to get permanent token) +Start by creating a permanent token for the grantor-default role. This token will be used by the secrets-bridge to interact with Vault and create temp tokens for applications. ``` -TEMP_TOKEN=$(curl -s -H "X-Vault-Token: $ROOT_TOKEN" ${VAULT_URL}/v1/auth/token/create -d '{"policies": ["default"], "ttl": "15m", "num_uses": 2}' | jq -r '.auth.client_token') +PERM_TOKEN=$(curl -s -X POST -H "X-Vault-Token: $ROOT_TOKEN" ${VAULT_URL}/v1/auth/token/create/grantor-default -d '{"policies": ["default", "grantor-default", "app1", "app2"], "ttl": "72h", "meta": {"configPath": "secret/secrets-bridge/Default"}}' | jq -r '.auth.client_token') ``` -Then create a permanent token for the grantor-default role. This token will be used by the secrets-bridge to interact with Vault and create temp tokens for applications. + +Then create a temporary token with a TTL of 15m and a max usage of 2 attempts (1st used to place permanent token within cubbyhole; 2nd usage is when secrets-bridge starts up and contacts Vault to get permanent token) ``` -PERM_TOKEN=$(curl -s -X POST -H "X-Vault-Token: $ROOT_TOKEN" ${VAULT_URL}/v1/auth/token/create/grantor-default -d '{"policies": ["default", "grantor-default", "app1", "app2"], "ttl": "72h", "meta": {"configPath": "secret/secrets-bridge/Default"}}' | jq -r '.auth.client_token') +TEMP_TOKEN=$(curl -s -H "X-Vault-Token: $ROOT_TOKEN" ${VAULT_URL}/v1/auth/token/create -d '{"policies": ["default"], "ttl": "15m", "num_uses": 2}' | jq -r '.auth.client_token') ``` -Finally, place the permanent token within a cubbyhole using the temporary token. The secrets bridge will use the temporary token (2nd and final usage) to retrieve the permanent token from the cubbyhole when it starts up. +Finally, place the permanent token within a cubbyhole using the temporary token. The secrets bridge will use the temporary token (2nd and final usage) to retrieve the permanent token from the cubbyhole when it starts up. ``` curl -X POST -H "X-Vault-Token: ${TEMP_TOKEN}" ${VAULT_URL}/v1/cubbyhole/Default -d "{\"permKey\": \"${PERM_TOKEN}\"}" From 0d123e799e47f1ca1935af38d215cdaf6ae89935 Mon Sep 17 00:00:00 2001 From: Greg Surbey Date: Tue, 16 Aug 2016 14:56:49 -0400 Subject: [PATCH 4/6] Fixing typo in vault write command in the Readme document. --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 779fcae..8bbdde7 100644 --- a/README.md +++ b/README.md @@ -128,8 +128,8 @@ curl -s -X POST -H "X-Vault-Token: ${VAULT_TOKEN}" -d '{"allowed_policies": "def ##### Step 5: Assign policies to applications ``` -vault write secrets/secrets-bridge/Default/Stack1/app1 policies=default,app1 -vault write secrets/secrets-bridge/Default/Stack2/app2 policies=default,app2 +vault write secret/secrets-bridge/Default/Stack1/app1 policies=default,app1 +vault write secret/secrets-bridge/Default/Stack2/app2 policies=default,app2 ``` ##### Step 6: Configure Vault for Secrets-Bridge startup From beaf86e842ef37079813d3b0bb7d0126227d6a08 Mon Sep 17 00:00:00 2001 From: Greg Surbey Date: Tue, 23 Aug 2016 16:49:13 -0400 Subject: [PATCH 5/6] Added a hint about needing container labels set, and also how to get debug output in the Readme document. --- README.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 8bbdde7..d63e356 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ To accomplish this, you need to create some default policies for the `secrets-br A service would then be deployed into an operators/tools/non-application environment. This item will likely be launchable from the catalog. -Once the server side service is deployed, you would then deploy the agents into your application environment. These agents then listen for Docker events and send container start events to the service. +Once the server side service is deployed, you would then deploy the agents into your application environment. These agents then listen for Docker container start events. If your container has the secrets.bridge.enabled and nameKey labels correctly set then the agent will send that container's start event to the service. The service then verifies with Rancher (see notes/todos below) the container's Identity. If the identity can not be verified, then nothing else happens. If the container is verified, the service checks for a policy key set on the config service tokens config path. If a policy is found, the service then generates a temporary token to create a Cubbyhole and a permanent token with the applied policy. The permanent key is placed into the Cubbyhole with the temporary key, which will have a short TTL and 1 more use to get the permanent key. @@ -164,3 +164,11 @@ secrets-bridge server --vault-url $VAULT_ADDR --rancher-url $RANCHER_ENVIRONMENT ``` secrets-bridge agent --bridge-url http://[IP Of Secrets Bridge Server]:8181 ``` + +#### Debugging: + +To enable debug output the '-d' flag has to be placed before any of the other command line arguments. For example: + +``` +secrets-bridge -d agent --bridge-url http://[IP Of Secrets Bridge Server]:8181 +``` \ No newline at end of file From d99bda9fc0a1546c93c541c27168f5c1ee929e95 Mon Sep 17 00:00:00 2001 From: Greg Surbey Date: Tue, 6 Sep 2016 12:39:16 -0400 Subject: [PATCH 6/6] Removed the K8 and TLS todos, since it should work with both now. --- README.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/README.md b/README.md index d63e356..5d93e6c 100644 --- a/README.md +++ b/README.md @@ -4,11 +4,8 @@ #### To Dos: * Create catalog entry. - * Make work with TLS production Vault setup (currently only works with a Dev Vault configuration). - * Add support for K8s and Swarm. * Cattle needs signature verification call. - --- #### Purpose: The Secrets Bridge service is a standardized way of integrating Rancher and Vault such that Docker containers at startup are securely connected with their secrets within Vault. The Secrets Bridge service is composed of a server and agents. At container startup, the service first validates the container's identity with Rancher, and then provides the container with access to Vault. Neither Rancher nor the service actually manages any secrets within Vault; that is still left to the user and Vault. What this service will do is create Vault Tokens which are assigned a subset of policies allowed by the initial grantor-default token provided to the Secrets Bridge server at startup. The app token obtained through this service is then used by the container to communicate directly with Vault. This allows a user to define a custom process in their containers that can inject the secrets it reads from Vault into the app that ultimately uses them, using whatever custom input methods required by the user's app.