Merge pull request #174 from akutz/feature/config-levels

Config Refactor - Dynamic Config Keys

.build/test.sh

@@ -3,6 +3,12 @@
 echo "mode: set" > acc.out
 FAIL=0
 
+go test -cover ./rexray/cli || FAIL=1
+
+if [ "$FAIL" -ne 0 ]; then
+    exit 1
+fi
+
 COVER_PKG="github.com/emccode/rexray","github.com/emccode/rexray/core"
 go test -coverpkg=$COVER_PKG  -coverprofile=profile.out ./test || FAIL=1
 if [ -f profile.out ]; then
@@ -24,6 +30,7 @@ for DIR in $(find . -type d \
              -not -path './.git*' \
              -not -path '*/_*' \
              -not -path './vendor/*' \
+             -not -path './rexray/*' \
              -not -path './core' \
              -not -path '.'); do
 

.docs/user-guide/config.md

@@ -45,12 +45,14 @@ changed.
 ## Configuration Methods
 There are three ways to configure `REX-Ray`:
 
-* Configuration files
-* Environment variables
 * Command line options
+* Environment variables
+* Configuration files
 
 The order of the items above is also the order of precedence when considering
-options set in multiple locations that may override one another.
+options set in multiple locations that may override one another. Values set
+via CLI flags have the highest order of precedence, followed by values set by
+environment variables, followed, finally, by values set in configuration files.
 
 ## Configuration Files
 There are two `REX-Ray` configuration files - global and user:
@@ -69,10 +71,117 @@ options as described in the following section:
 
 ```yaml
 logLevel: warn
-osDrivers: linux
-volumeDrivers: docker
+osDrivers:
+- linux
+volumeDrivers:
+- docker
+```
+
+## Configuration Properties
+The section [Configuration Methods](#configuration-methods) mentions there are
+three ways to configure REX-Ray: config files, environment variables, and the
+command line. However, this section will illuminate the relationship between the
+names of the configuration file properties, environment variables, and CLI
+flags.
+
+These are the global configuration properties with their default values as
+represented in a a YAML configuration file:
+
+```yaml
+logLevel: warn
+osDrivers:
+- linux
+storageDrivers:
+volumeDrivers:
+- docker
+```
+
+The property `logLevel` is a string and the properties `osDrivers`,
+`storageDrivers`, and `volumeDrivers` are all arrays of strings. These values
+can also be set via environment variables or the command line, but to do so
+requires knowing the names of the environment variables or CLI flags to use.
+Luckily those are very easy to figure out just by knowing the property names.
+
+### Top-Level Properties
+For any configuration file property there is really only one exception to
+consider when deriving the name of the property's equivalent environment
+variable or CLI flag: is the property a top-level property? The top-level
+properties are:
+
+  * `logLevel`
+  * `osDrivers`
+  * `storageDrivers`
+  * `volumeDrivers`
+
+For any top-level property the name of its equivalent environment variable is
+the name of the property, in all caps, prefixed with `REXRAY_`. Additionally,
+the name of the property's CLI flag is equivalent to the property name, as is.
+The following table illustrates the transformations:
+
+Property Name | Environment Variable | CLI Flag
+--------------|----------------------|-------------
+`logLevel`    | `REXRAY_LOGLEVEL`    | `--logLevel`
+`osDrivers`   | `REXRAY_OSDRIVERS`   | `--osDrivers`
+`storageDrivers`    | `REXRAY_STORAGEDRIVERS`   | `--storageDrivers`
+`volumeDrivers`     | `REXRAY_VOLUMEDRIVERS`    | `--volumeDrivers`
+
+### All Other Properties
+The other properties that might appear in the `REX-Ray` configuration file all
+fall under some type of heading. For example, a possible configuration of the
+Amazon Web Services (AWS) Elastic Compute Cloud (EC2) storage driver might look
+like this:
+
+```yaml
+aws:
+    accessKey: MyAccessKey
+    secretKey: MySecretKey
+    region:    USNW
+```
+
+For any property that's nested, the rule for environment variables is as
+follows:
+
+  * Each nested level becomes a part of the environment variable name followed
+    by an underscore `_` except for the terminating part.
+  * The entire environment variable name is uppercase.
+
+Nested properties follow these rules for CLI flags:
+
+  * The root level's first character is lower-cased with the rest of the root
+    level's text left unaltered.
+  * The remaining levels' first characters are all upper-cased with the the
+    remaining text of that level left unaltered.
+  * All levels are then concatenated together.
+
+The following table illustrates the transformations:
+
+Property Name | Environment Variable | CLI Flag
+--------------|----------------------|-------------
+`aws.accessKey`    | `AWS_ACCESSKEY`    | `--awsAccessKey`
+`aws.secretKey`   | `AWS_SECRETKEY`   | `--awsSecretKey`
+`aws.region`    | `AWS_REGION`   | `--awsRegion`
+
+### String Arrays
+Please note that properties that are represented as arrays in a configuration
+file, such as the `osDrivers`, `storageDrivers`, and `volumeDrivers` above, are
+not arrays, but multi-valued strings where a space acts as a delimiter. This is
+because the Viper project
+[does not bind](https://github.com/spf13/viper/issues/112) Go StringSlices
+(string arrays) correctly to [PFlags](https://github.com/spf13/pflag).
+
+For example, this is how one would specify the storage drivers `ec2` and
+`xtremio` in a configuration file:
+
+```yaml
+storageDrivers:
+    - ec2
+    - xtremio
 ```
 
+However, to specify the same values in an environment variable,
+`REXRAY_STORAGEDRIVERS="ec2 xtremio"`, and as a CLI flag,
+`--storageDrivers="ec2 xtremio"`.
+
 ## Logging
 The `REX-Ray` log level determines the level of verbosity emitted by the
 internal logger. The default level is `warn`, but there are three other levels

.docs/user-guide/ec2.md

@@ -10,14 +10,19 @@ driver manager and is used to connect and manage storage on EC2 instances. The
 EC2 driver is made possible by the
 [goamz project](https://github.com/mitchellh/goamz).
 
-## Configuration Options
-The following are the configuration options for the `ec2` storage driver.
-
- EnvVar | YAML | CLI  
---------|------|------
-`AWS_ACCESS_KEY` | `awsAccessKey` | `--awsAccessKey`
-`AWS_SECRET_KEY` | `awsSecretKey` | `--awsSecretKey`
-`AWS_REGION` | `awsRegion` | `--awsRegion`
+## Configuration
+The following is an example configuration of the AWS EC2 driver.
+
+```yaml
+aws:
+    accessKey: MyAccessKey
+    secretKey: MySecretKey
+    region:    USNW
+```
+
+For information on the equivalent environment variable and CLI flag names
+please see the section on how non top-level configuration properties are
+[transformed](./config/#all-other-properties).
 
 ## Activating the Driver
 To activate the EC2 driver please follow the instructions for

.docs/user-guide/openstack.md

@@ -9,21 +9,26 @@ The OpenStack driver registers a storage driver named `openstack` with the
 `REX-Ray` driver manager and is used to connect and manage storage on OpenStack
 instances.
 
-## Configuration Options
-The following are the configuration options for the `openstack` storage driver.
-
- EnvVar | YAML | CLI
---------|------|------
-`OS_AUTH_URL` | `openstackAuthUrl` | `--openstackAuthUrl`
-`OS_USERID` | `openstackUserId` | `--openstackUserId`
-`OS_USERNAME` | `openstackUserName` | `--openstackUserName`
-`OS_PASSWORD` | `openstackPassword` | `--openstackPassword`
-`OS_TENANT_ID` | `openstackTenantId` | `--openstackTenantId`
-`OS_TENANT_NAME` | `openstackTenantName` | `--openstackTenantName`
-`OS_DOMAIN_ID` | `openstackDomainId` | `--openstackDomainId`
-`OS_DOMAIN_NAME` | `openstackDomainName` | `--openstackDomainName`
-`OS_REGION_NAME` | `openstackRegionName` | `--openstackRegionName`
-`OS_AVAILABILITY_ZONE_NAME` | `openstackAvailabilityZoneName` | `--openstackAvailabilityZoneName`
+## Configuration
+The following is an example configuration of the OpenStack driver.
+
+```yaml
+openstack:
+    authURL:              https://domain.com/openstack
+    userID:               0
+    userName:             admin
+    password:             mypassword
+    tenantID:             0
+    tenantName:           customer
+    domainID:             0
+    domainName:           corp
+    regionName:           USNW
+    availabilityZoneName: Gold
+```
+
+For information on the equivalent environment variable and CLI flag names
+please see the section on how non top-level configuration properties are
+[transformed](./config/#all-other-properties).
 
 ## Activating the Driver
 To activate the OpenStack driver please follow the instructions for

.docs/user-guide/rackspace.md

@@ -9,19 +9,20 @@ The Rackspace driver registers a storage driver named `rackspace` with the
 `REX-Ray` driver manager and is used to connect and manage storage on Rackspace
 instances.
 
-## Configuration Options
-The following are the configuration options for the `rackspace` storage driver.
+## Configuration
+The following is an example configuration of the Rackspace driver.
 
- EnvVar | YAML | CLI  
---------|------|------
-`OS_AUTH_URL` | `rackspaceAuthUrl` | `--rackspaceAuthUrl`
-`OS_USERID` | `rackspaceUserId` | `--rackspaceUserId`
-`OS_USERNAME` | `rackspaceUserName` | `--rackspaceUserName`
-`OS_PASSWORD` | `rackspacePassword` | `--rackspacePassword`
-`OS_TENANT_ID` | `rackspaceTenantId` | `--rackspaceTenantId`
-`OS_TENANT_NAME` | `rackspaceTenantName` | `--rackspaceTenantName`
-`OS_DOMAIN_ID` | `rackspaceDomainId` | `--rackspaceDomainId`
-`OS_DOMAIN_NAME` | `rackspaceDomainName` | `--rackspaceDomainName` 
+```yaml
+rackspace:
+    authURL:    https://domain.com/rackspace
+    userID:     0
+    userName:   admin
+    password:   mypassword
+    tenantID:   0
+    tenantName: customer
+    domainID:   0
+    domainName: corp
+```
 
 ## Activating the Driver
 To activate the Rackspace driver please follow the instructions for

.docs/user-guide/scaleio.md

@@ -8,22 +8,27 @@ Scale-out with simplified storage management
 The ScaleIO registers a storage driver named `scaleio` with the `REX-Ray`
 driver manager and is used to connect and manage ScaleIO storage.
 
-## Configuration Options
-The following are the configuration options for the `scaleio` storage driver.
-
- EnvVar | YAML | CLI  
---------|------|------
-`GOSCALEIO_ENDPOINT` | `scaleIoEndpoint` | `--scaleIoEndpoint`
-`GOSCALEIO_INSECURE` | `scaleIoInsecure` | `--scaleIoInsecure`
-`GOSCALEIO_USECERTS` | `scaleIoUseCerts` | `--scaleIoUseCerts`
-`GOSCALEIO_USERNAME` | `scaleIoUserName` | `--scaleIoUserName`
-`GOSCALEIO_PASSWORD` | `scaleIoPassword` | `--scaleIoPassword`
-`GOSCALEIO_SYSTEMID` | `scaleIoSystemId` | `--scaleIoSystemId`
-`GOSCALEIO_SYSTEMNAME` | `scaleIoSystemName` | `--scaleIoSystemName`
-`GOSCALEIO_PROTECTIONDOMAINID` | `scaleIoProtectionDomainId` | `--scaleIoProtectionDomainId`
-`GOSCALEIO_PROTECTIONDOMAIN` | `scaleIoProtectionDomainName` | `--scaleIoProtectionDomainName`
-`GOSCALEIO_STORAGEPOOLID` | `scaleIoStoragePoolId` | `--scaleIoStoragePoolId`
-`GOSCALEIO_STORAGEPOOL` | `scaleIoStoragePoolName` | `--scaleIoStoragePoolName`
+## Configuration
+The following is an example configuration of the ScaleIO driver.
+
+```yaml
+scaleio:
+    endpoint:             https://domain.com/scalio
+    insecure:             false
+    useCerts:             true
+    userName:             admin
+    password:             mypassword
+    systemID:             0
+    systemName:           sysv
+    protectionDomainID:   0
+    protectionDomainName: corp
+    storagePoolID:        0
+    storagePoolName:      gold
+```
+
+For information on the equivalent environment variable and CLI flag names
+please see the section on how non top-level configuration properties are
+[transformed](./config/#all-other-properties).
 
 ## Activating the Driver
 To activate the ScaleIO driver please follow the instructions for

.docs/user-guide/xtremio.md

@@ -8,18 +8,23 @@ Not just a flash in the pan
 The XtremIO registers a storage driver named `xtremio` with the `REX-Ray`
 driver manager and is used to connect and manage XtremIO storage.
 
-## Configuration Options
-The following are the configuration options for the `xtremio` storage driver.
-
- EnvVar | YAML | CLI  
---------|------|------
-`GOXTREMIO_ENDPOINT` | `xtremIoEndpoint` | `--xtremIoEndpoint`
-`GOXTREMIO_USERNAME` | `xtremIoUserName` | `--xtremIoUserName`
-`GOXTREMIO_PASSWORD` | `xtremIoPassword` | `--xtremIoPassword`
-`GOXTREMIO_INSECURE` | `xtremIoInsecure` | `--xtremIoInsecure`
-`GOXTREMIO_DM` | `xtremIoDeviceMapper` | `--xtremIoDeviceMapper`
-`GOXTREMIO_MULTIPATH` | `xtremIoMultipath` | `--xtremIoMultipath`
-`GOXTREMIO_REMOTEMANAGEMENT` | `xtremIoRemoteManagement` | `--xtremIoRemoteManagement`
+## Configuration
+The following is an example configuration of the XtremIO driver.
+
+```yaml
+xtremio:
+    endpoint:         https://domain.com/xtremio
+    userName:         admin
+    password:         mypassword
+    insecure:         false
+    deviceMapper:     false
+    multipath:        true
+    remoteManagement: false
+```
+
+For information on the equivalent environment variable and CLI flag names
+please see the section on how non top-level configuration properties are
+[transformed](./config/#all-other-properties).
 
 ## Activating the Driver
 To activate the XtremIO driver please follow the instructions for