Skip to content

Manage application configuration stored in cloud services.

License

Notifications You must be signed in to change notification settings

dabblebox/stash

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Stash CLI and Go Integration Library

Stash provides basic CLI commands and a Go integration library to sync, edit, get, and purge configuration files in cloud services instead of working with individual key/value pairs, unique commands, and encryption details specific to each cloud service. Does using Stash improve security?

sync

Stashed files are replaced locally by a stash.yml file that can be shared and checked into source control safely. stash.yml remembers the cloud service and file encryption details making editing, restoration, or consumption a single command. There are multiple AWS methods that can be considered when consuming configuration.

BuildNDeploy GoDoc

AWS Supported Services

When stashing configuration files, a cloud service, a.k.a. stash, must be chosen. When a service supports key/value pairs, .env and .json configuration files can be parsed, stashed individually, and restored as a single file.

Service File Types Encryption Granting Access
AWS Secrets Manager .env, .json, .js, .ts, .yml, .xml, .sql, .cert, id_rsa KMS Secrets
AWS Parameter Store .env KMS Parameters
AWS S3 Storage * KMS Files

Get Started

  1. Install CLI
$ curl -s https://raw.githubusercontent.com/dabblebox/stash/master/install.io | sh
  1. Manage Configuration
$ export AWS_REGION=us-east-1
$ export AWS_PROFILE=user-profile-devops

# sync configs
$ stash sync config/dev/.env

# edit configs
$ stash edit -t dev

# get configs
$ stash get -t dev

# generate Terraform
$ stash get -t dev -o terraform

For authentication, see Specifying Credentials in the AWS SDK for Go documentation.

Commands

$ stash sync

sync

Upload and sync new or modified configuration files to a cloud service.

Command:

stash sync [<file_path>|<regex>...] [flags]

Examples:

# file paths
$ stash sync config/dev/.env config/qa/.env

# regular expressions (escape \backslashes or 'quote' expressions)
$ stash sync .*\\.env$ .*\\.json$
Flag Short Example Default Description
--file -f slickapp.yml stash.yml catalog path with file name
--context -c slickapp parent folder prefix for cloud service keys
--service -s secrets-manager, parameter-store, s3 cloud service
--tags -t config,dev,app file path and name file reference tags
$ stash edit

sync

Download and open configuration inside an editor. When the file is closed after modifications, sync with the cloud service.

Optionally set preferred editor. (default: vim)

$ export EDITOR="code"

Command:

stash edit [<file_path>...] [flags]

Examples:

# browse
$ stash edit

# file paths
$ stash edit config/dev/.env config/qa/.env

# file tags
$ stash edit -t config,dev
Flag Short Example Description
--file -f stash.yml catalog path with file name
--service -s secrets-manager, parameter-store, s3 cloud service
--tags -t config,dev,app file reference tags
$ stash get

download

Download configuration files and apply optional transformations to the config. By default, the files are sent to stdout allowing the config to be piped anywhere including a new file location. The files can be restored to their original folder locations using the flag, -o file.

IMPORTANT: When restoring configuration for a service, make sure configuration is not printed anywhere or sent to logs via stdout.

Command:

stash get [<file_path>...] [flags]

Examples:

# by file paths
$ stash get config/dev/.env config/qa/.env 

# by file tags
$ stash get -t config,dev

# by cloud service
$ stash get -s s3

# restore original files
$ stash get -o file

# create new files
$ stash get >> .env

# apply data transformation
$ stash get -o json >> .env

# export environment variables
$ eval $( stash get -t dev -o terminal-export )
Flag Short Example Description
--file -f stash.yml catalog path with file name
--service -s secrets-manager, parameter-store, s3 cloud service
--tags -t config,dev,app file reference tags
--output -o terminal-export configuration output

Configuration Outputs

Using the --output or -o flag, the stashed configuration can be downloaded, transformed or replaced, and sent to stdout. Each service supports specific outputs and file types as charted below. When the output flag is omitted, the original file data is sent to stdout.

Secrets Manager Parameter Store S3 Storage
file * * * file system original file
terraform * * * file system terraform scripts
ecs-task-inject-json * * .env stdout AWS ECS task definition secrets / envfile (JSON) (key/arn)
ecs-task-inject-env * * .env stdout AWS ECS task definition secrets / envfile (ENV) (key/arn)
ecs-task-env .env .env .env stdout AWS ECS task definition environment (JSON) (key/value)
json .env .env .env stdout JSON object
terminal-export-literal .env .env .env stdout prepend "export " to each key/value pair (single quotes)
terminal-export .env .env .env stdout prepend "export " to each key/value pair (double quotes)
$ stash purge

purge

Purge permanently deletes files stashed in a cloud service.

Command:

stash purge [<file_path>...] [flags]

Examples:

# by file names
$ stash purge config/dev/.env config/qa/.env 

# by file tags
$ stash purge -t config,dev

# by cloud service
$ stash purge -s s3
Flag Short Example Description
--file -f stash.yml catalog path with file name
--service -s secrets-manager, parameter-store, s3 cloud service
--tags -t config,dev,app file reference tags
--warn -s false skips warning prompts
$ stash clean

clean

Clean deletes tracked local files to avoid secrets remaining on developer machines.

Command:

stash clean [<file_path>...] [flags]

Examples:

# by file names
$ stash clean config/dev/.env config/qa/.env 

# by file tags
$ stash clean -t config,dev

# by cloud service
$ stash clean -s s3
Flag Short Example Description
--file -f stash.yml catalog path with file name
--service -s secrets-manager, parameter-store, s3 cloud service
--tags -t config,dev,app file reference tags
$ stash list

list

List displays tracked files, tags, and cloud service keys.

Command:

stash list [<file_path>...] [flags]

Examples:

# all
$ stash list

# by file names
$ stash list config/dev/.env config/qa/.env 

# by file tags
$ stash list -t config,dev

# by cloud service
$ stash list -s s3
Flag Short Example Description
--file -f stash.yml catalog path with file name
--service -s secrets-manager, parameter-store, s3 cloud service
--tags -t config,dev,app file reference tags
$ stash tag

tag

Tags identify or group stashed files in the stash.yml catalog allowing actions to be performed against file groups.

Command:

stash tag [<file_path>...] [flags]

Examples:

# overwrite tags
$ stash tag config/dev/.env config/qa/.env -t app,non-prod

# add tag
$ stash tag -a non-prod

# delete tag
$ stash tag -d config

# add tag by tags
$ stash tag -t non-prod -a config

# add tag by service
$ stash tag -s s3 -a config
Flag Short Example Description
--file -f stash.yml catalog path with file name
--service -s secrets-manager, parameter-store, s3 cloud service
--tags -t config,dev,app file reference tags
--add -a app,non-prod add tags
--delete -d app,non-prod delete tags
$ stash inject

After updating Secrets Manager or Parameter Store through $ stash sync or manually in the AWS Console, the stashed values can be injected into a configuration file locally or in a running container.

Add tokens to any configuration file using ${SECRET_NAME::SECRET_KEY} for Secrets Manager or ${PARAM_PATH::PARAM_NAME} for Parameter Store.

Example config.json

{
  "db_user": "${app/dev/db::user}",
  "db_password": "${app/dev/db::password}",
}

Inject secrets from the specified stash, -s secrets-manager, into the configuration file, config.json. The inject command does not require a stash.yml catalog file.

Command:

stash inject [<file_path>...] [flags]

Examples:

$ stash inject config.json -s secrets-manager
Flag Short Example Description
--service -s secrets-manager, parameter-store, s3 cloud service
--output -o terminal-export file output format

Environment Variables

Override Catalog Fields

Any field in stash.yml can be overridden when exported before using $ stash get. (use "_" to access children)

S3 Bucket Example

files:
  sample_dev__env:
    path: sample/dev/.env
    type: env
    stash: s3
    opt:
      s3_bucket: configs
$ export STASH_FILES_SAMPLE_DEV__ENV_OPT_S3_BUCKET=new-configs
Template Catalog Files

Additional template files can be added to the catalog to allow environment variable expansion when getting configuration for an application. This is useful when the stash.yml file cannot be aware of all application environments during the artifact/image build step.

$ export ENV=dev
$ export VERSION=v1.0.0
files:
  template:
    path: sample/${VERSION}/${ENV}/.env
    type: env
    stash: s3
    tags:
    - config
$ stash get -t config
Set Defaults / Override Prompts

When syncing files, setting environment variables will override prompts.

Variable default Description
STASH_CATALOG stash.yml name of the catalog file
STASH_CONTEXT working directory prefix for cloud keys
STASH_KMS_KEY_ID Default Account Key KMS Key ID or Default Account Key
STASH_S3_BUCKET S3 bucket name
STASH_SERVICE prompt user cloud service
STASH_WARN true confirm purge

Application Integration

Get Config
package main

import (
	"log"

	"github.com/dabblebox/stash"
	"github.com/dabblebox/stash/component/output"
)

config, err := stash.GetMap(stash.GetOptions{})
if err != nil {
  log.Fatal(err)
}

for k, v := range config {
  log.Printf("%s=%s\n", k, v)
}
Inject Config
package main

import (
	"log"

	"github.com/dabblebox/stash"
	"github.com/dabblebox/stash/component/output"
)

files, err := stash.Inject(stash.InjectOptions{
  Files: []string{"config.json"}
})
if err != nil {
  log.Fatal(err)
}

for _, f := range files {
  log.Printf("%s\n", string(f.Data))
}