debugging details for storage driver initialization

[cduchesne]

When running the rexray cli with "-l debug", I don't get the sense that there is any useful information to help me debug an issue with a storage driver. I have been specifically using ScaleIO in my testing and frequently run into a challenge here or there, mostly due to my fault of course, where the issue should easily be noticeable if some more details were presented to the console via "-l debug".

Example scenarios where more debug details would help significantly. In all three cases I provide (there are many more), the response looks the same. This most likely applies to other drivers besides ScaleIO.

Case 1: invalid login credentials
REX-Ray response:

# rexray volume get -l debug
DEBU[0000] updated log level                             logLevel=debug
INFO[0000] [linux]                                      
INFO[0000] [docker]                                     
INFO[0000] [ScaleIO]                                    
DEBU[0000] core get drivers                              osDrivers=[linux] storageDrivers=[ScaleIO] volumeDrivers=[docker]
INFO[0000] docker volume driver initialized              availabilityZone= iops= moduleName= provider=docker size= volumeRootPath=/data volumeType=
INFO[0000] os driver initialized                         moduleName= provider=linux
DEBU[0000] error initializing driver                     driverName=ScaleIO error.password="******" error.provider=ScaleIO error.useCerts=false error.apiVersion=2.0 error.moduleName= error.endpoint="http://scaleio-gw-proxy:9090/api" error.userName=admin error.insecure=true
DEBU[0000] checking volume path cache setting            pathCache=false
Oops, an error occured!

  no storage drivers initialized

ScaleIO gateway response (via proxy): HTTP ERROR 401 Unauthorized with JSON message:

{
    "message": "Unauthorized",
    "httpStatusCode": 401,
    "errorCode": 0
}

Result: REX-Ray doesn't tell me that it simply received an auth failure response from api.

Case 2: Invalid Configuration on ScaleIO Gateway
REX-Ray response:

# rexray volume get -l debug
DEBU[0000] updated log level                             logLevel=debug
INFO[0000] [linux]                                      
INFO[0000] [docker]                                     
INFO[0000] [ScaleIO]                                    
DEBU[0000] core get drivers                              osDrivers=[linux] storageDrivers=[ScaleIO] volumeDrivers=[docker]
DEBU[0001] error initializing driver                     driverName=ScaleIO error.apiVersion=2.0 error.userName=admin error.useCerts=false error.moduleName= error.endpoint="http://scaleio-gw-proxy:9090/api" error.password="******" error.provider=ScaleIO error.insecure=true
INFO[0001] docker volume driver initialized              availabilityZone= iops= moduleName= provider=docker size= volumeRootPath=/data volumeType=
INFO[0001] os driver initialized                         moduleName= provider=linux
DEBU[0001] checking volume path cache setting            pathCache=false
Oops, an error occured!

  no storage drivers initialized

ScaleIO gateway response (via proxy): HTTP 500 Internal Server Error with JSON message:

{
    "message": "System id 2ba755b05c0e16a6 is different from gateway SystemId (mdmAddresses points to a cluster and system id belongs to a different cluster)).",
    "httpStatusCode": 500,
    "errorCode": 0
} 

Result: With the message provided by the scaleio-gateway, I at least know to check the configuration there and do some comparisons between my config in rexray, the config on my scaleio gateway, and my scaleio cluster configuration. Without the debug details, it is much more challenging.

Case 3: Invalid REX-Ray config item ("systemName: scaleio")
REX-Ray response:

# rexray volume get -l debug
DEBU[0000] updated log level                             logLevel=debug
INFO[0000] [linux]                                      
INFO[0000] [docker]                                     
INFO[0000] [ScaleIO]                                    
DEBU[0000] core get drivers                              osDrivers=[linux] storageDrivers=[ScaleIO] volumeDrivers=[docker]
INFO[0000] os driver initialized                         moduleName= provider=linux
DEBU[0000] error initializing driver                     driverName=ScaleIO error.userName=admin error.provider=ScaleIO error.moduleName= error.endpoint="http://scaleio-gw-proxy:9090/api" error.useCerts=false error.insecure=true error.password="******" error.apiVersion=2.0
INFO[0000] docker volume driver initialized              availabilityZone= iops= moduleName= provider=docker size= volumeRootPath=/data volumeType=
DEBU[0000] checking volume path cache setting            pathCache=false
Oops, an error occured!

  no storage drivers initialized

ScaleIO gateway response (via proxy) - heavily trimmed for readability:

[{
    "swid": "",
    "daysInstalled": 3,
    "systemVersionName": "EMC ScaleIO Version: R2_0.5014.0",
    "installId": "2037751116a9651e",
    "name": "scaleio_default,
    "id": "2ba755b05c0e16a6"
}]

Result: REX-Ray doesn't tell me that it connected/authenticated to the api just fine and then failed to find a ScaleIO system named "scaleio"

[clintkitson]

Thanks @cduchesne. We are addressing this in 0.4.0 which will rectify the problem for all drivers during the initialization phase.

[clintkitson]

This is fixed in 0.4.0. See the following example.

RRO[0000] default module(s) failed to initialize       
INFO[0000] service sent registered modules start signals 
ERRO[0000] service initialized failed                   
INFO[0000] error authenticatin                           endpoint=https://192.168.50.12/apxi inner=Not Found insecure=true password=****** provider=scaleIO useCerts=false userName=admin