datastore.rst

Datastore

The goal of the datastore service is to allow users to store common parameters and their values within for reuse in the definition of sensors, actions, and rules. The datastore service store the data as a key value pair and they can be get/set using the CLI or the python client. From the sensor and action plugins, since they are implemented in python, the key value pairs are accessed from the python client. For rule definitions in YAML/JSON, the key value pairs are referenced with a specific string substitution syntax and the references are resolved on rule evaluation.

Storing and Retrieving Key Value Pairs via CLI

Set the value of a key value pair:

st2 key set os_keystone_endpoint http://localhost:5000/v2.0
st2 key set aws_cfn_endpoint https://cloudformation.us-west-1.amazonaws.com

Load a list of key value pairs from a JSON file. The following is the JSON example using the same keys from the create examples above:

[
    {
        "name": "os_keystone_endpoint",
        "value": "http://localhost:5000/v2.0"
    },
    {
        "name": "aws_cfn_endpoint",
        "value": "https://cloudformation.us-west-1.amazonaws.com"
    }
]

st2 key load mydata.json

The load command also allows you to directly load the output of "key list -j" command. This is useful if you want to migrate datastore items from a different cluster or if you want to version control the datastore items and load the from version controlled files:

st2 key list -j > mydata.json
st2 key load mydata.json

Get individual key value pair or list all:

st2 key list
st2 key get os_keystone_endpoint
st2 key get os_keystone_endpoint -j

Update an existing key value pair:

st2 key set os_keystone_endpoint http://localhost:5000/v3

Delete an existing key value pair:

st2 key delete os_keystone_endpoint

Scoping Datastore Items

By default, all items stored in key value store are stored in st2kv.system scope. This basically means every user has access to these variables. You'd typically use the jinja expression {{st2kv.system.key_name}} to refer to these variables in actions or workflows. Note that until v2.1, the scope used to be called system and therefore you'd have used jinja expression {{st2kv.system.key_name}}. Starting version 1.5 of , you can now scope variables to a specific user. With authentication enabled, you can now control who can read or write into those variables. For example, to set variable date_cmd for the currently authenticated user, use:

st2 key set date_cmd "date -u" --scope=user

The name of the user is figured out from the X-Auth-Token or St2-Api-Key header passed with the API call. Basically, from the API call authentication credentials, we figure out the user and assign this variable to that particular user. To get the key back, use:

st2 key get date_cmd --scope=user

Remember, if you want a variable date_cmd as a system variable, you can use:

st2 key set date_cmd "date +%s" --scope=system

or simply:

st2 key set date_cmd "date +%s"

This variable won't clash with the user variables under same name. Also, you can refer to user variables in actions or workflows. The jinja syntax to do so is {{st2kv.user.date_cmd}}. Until v2.1, this expression used to be {{user.date_cmd}} but is now deprecated.

Note that the notion of st2kv.user is available only when actions or workflows are run manually. The notion of st2kv.user is non-existent when automations are run (actions kicked off via rules). So the use of user scoped variables is limited to manual execution of actions or workflows.

Storing and Retrieving via Python Client

Create new key value pairs. The API endpoint is set either via the Client init (base_url) or from environment variable (ST2_BASE_URL). The default ports for the API servers are assumed:

>>> from st2client.client import Client
>>> from st2client.models import KeyValuePair
>>> client = Client(base_url='http://localhost')
>>> client.keys.update(KeyValuePair(name='os_keystone_endpoint', value='http://localhost:5000/v2.0'))

Get individual key value pair or list all:

>>> keys = client.keys.get_all()
>>> os_keystone_endpoint = client.keys.get_by_name(name='os_keystone_endpoint')
>>> os_keystone_endpoint.value
u'http://localhost:5000/v2.0'

Update an existing key value pair:

>>> os_keystone_endpoint = client.keys.get_by_name(name='os_keystone_endpoint')
>>> os_keystone_endpoint.value = 'http://localhost:5000/v3'
>>> client.keys.update(os_keystone_endpoint)

Delete an existing key value pair:

>>> os_keystone_endpoint = client.keys.get_by_name(name='os_keystone_endpoint')
>>> client.keys.delete(os_keystone_endpoint)

Referencing Key Value Pair in Rule Definition

Key value pairs are referenced via specific string substitution syntax in rules. In general, variable for substitution is enclosed with double brackets (i.e. {{var1}}). To refer to a key value pair, prefix the variable name with "st2kv.system" (i.e. {{st2kv.system.os_keystone_endpoint}}). An example rule is provided below. Please refer to the documentation section for Rules on rule related syntax.

{
    "name": "daily_clean_up_rule",
    "trigger": {
        "name": "st2.timer.daily"
    },
    "enabled": true,
    "action": {
        "name": "daily_clean_up_action",
        "parameters": {
            "os_keystone_endpoint": "{{st2kv.system.os_keystone_endpoint}}"
        }
    }
}

Securing Secrets (admin only)

Note

This guide and the corresponding implementation is alpha quality. We are working on the feature and feedback is welcome. Until the feature matures and deployment issues identified and fixed, no guarantee is made w.r.t security of the credentials stored in key value store.

Key value store now allows users to store encrypted values (secrets). Symmetric encryption is used to encrypt secrets. The administrator is responsible for generating symmetric key used for encryption / decryption. It goes without saying that the operator and administrator (or anyone else who has access to the key) can decrypt the encrypted values if they want to.

To generate a symmetric crypto key (AES256 used for both encryption and decryption) as an admin, please run:

sudo mkdir -p /etc/st2/keys/
sudo st2-generate-symmetric-crypto-key --key-path /etc/st2/keys/datastore_key.json

It is recommended that the key is placed in a private location such as /etc/st2/keys/ and permissions are appropriately modified so that only API process owner (usually st2) can read and admin can read/write to that file.

To make sure only st2 and root can access the file on the box, run:

sudo usermod -a -G st2 st2                              # Add user ``st2`` to ``st2`` group
sudo mkdir -p /etc/st2/keys/
sudo chown -R st2:st2 /etc/st2/keys/                    # Give user and group ``st2`` ownership for key
sudo chmod o-r /etc/st2/keys/                           # Revoke read access for others
sudo chmod o-r /etc/st2/keys/datastore_key.json         # Revoke read access for others

Once the key is generated, needs to be made aware of the key. To do this, edit st2 configuration file (usually /etc/st2/st2.conf) and add the following lines:

[keyvalue]
encryption_key_path = /etc/st2/keys/datastore_key.json

Once the config file changes are made, restart :

sudo st2ctl restart

Validate you are able to set an encrypted key value in datastore:

st2 key set test_key test_value --encrypt

You shouldn't see any errors. If you see errors like "MESSAGE: Crypto key not found", you haven't setup the keys correctly.

Now as an admin, you are all set with configuring server side.

Storing Secrets

Please note that if an admin has not setup an encryption key, you will not be allowed to save secrets in the key value store. Contact your admin to setup encryption keys as per the section above.

To save a secret in key value store:

st2 key set api_token SECRET_TOKEN --encrypt

By default, getting a key tagged as secret (via --encrypt) will always return encrypted values only. To get plain text, please run with command --decrypt flag:

st2 key get api_token --decrypt

Note

Keep in mind that --decrypt flag can either be used by an administrator (administrator is able to decrypt every value) and by the user who set that value in case of the user-scoped datastore item (i.e. if --scope=user flag was passed when originally setting the value).

If you are using system scoped variables (st2kv.system) to store secrets, you can decrypt them and use as parameter values in rules or actions. This is supported via jinja filter decrypt_kv (read more about jinja filters<applying-filters-with-jinja>). For example, to pass a decrypted password as a parameter, simply do

aws_key: "{{st2kv.system.aws_key | decrypt_kv}}"

Decrypting user scoped variables is currently unsupported.

Security notes

We wish to discuss security details and be transparent about the implementation and limitations of the security practices to attract more eyes to it and therefore build better quality into security implementations. For the key value store, we have settled on AES256 symmetric encryption for simplicity. We use python library keyczar for doing this.

We have made a trade off that admin is allowed to decrypt the secrets in key value store. This made our implementation simpler. We are looking into how to let users pass their own keys for encryption every time they want to consume a secret from key value store. This requires more UX thought and also moves the responsibility of storing keys to the users. Your ideas are welcome here.

Please note that the global encryption key still disables users with direct access to databases to still see only encrypted secret in database. Still the onus is on admin to restrict access to database via network daemons only and not allow physical access to the box (or run databases on different boxes as st2). Note that several layers of security needs to be in place that is beyond the scope of this document. While we can help people with deployment questions on StackStorm Slack community, please follow your own best security practices guide.