Xport enhancements

[allmightyspiff]

makes the debugging transport a bit more useful.

• Adds call logging that doesn't reset every time you look at it, along with a log of the ENTIRE request
• Adds print_reproduceable function for xml and rest calls to make it much easier to copy/paste just the api call that is not working properly.
• Removes a bunch of the junk debugging output that is cluttering everything up.

[coveralls]

Coverage increased (+0.07%) to 87.074% when pulling e520056 on allmightyspiff:xport-enhancements into f8f743a on softlayer:master.

[allmightyspiff]

$ ./slcli -v vs detail 26750507 > out.txt
Calling: SoftLayer_Virtual_Guest::getObject(id=26750507)
Calling: SoftLayer_Virtual_Guest::getReverseDomainRecords(id=26750507)
execution_time    1.456582s
                            Virtual_Guest::getObject (1.067999s)
                  SoftLayer_Virtual_Guest::getReverseDomainRecords (0.318586s)
version           softlayer-python/v5.4.3
python_version    3.6.2 (v3.6.2:5fd33b5, Jul  8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)]
library_location  C:\Users\allmi\Source\softlayer-python\SoftLayer

$ ./slcli -vv vs detail 26750507 > out.txt
Calling: SoftLayer_Virtual_Guest::getObject(id=26750507)
Calling: SoftLayer_Virtual_Guest::getReverseDomainRecords(id=26750507)
execution_time    1.354056s
                            Virtual_Guest::getObject (0.928660s)
                  SoftLayer_Virtual_Guest::getReverseDomainRecords (0.349873s)
version           softlayer-python/v5.4.3
python_version    3.6.2 (v3.6.2:5fd33b5, Jul  8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)]
library_location  C:\Users\allmi\Source\softlayer-python\SoftLayer
id      26750507                                                                                                                                                                                                                                                                                                                                      
mask    mask[id,globalIdentifier,fullyQualifiedDomainName,hostname,domain,createDate,modifyDate,provisionDate,notes,dedicatedAccountHostOnlyFlag,privateNetworkOnlyFlag,primaryBackendIpAddress,primaryIpAddress,networkComponents[id, status, speed, maxSpeed, name,                                                                                 
                                             macAddress, primaryIpAddress, port,                                                                                                                                                                                                                                                                      
                                             primarySubnet,                                                                                                                                                                                                                                                                                           
                                             securityGroupBindings[                                                                                                                                                                                                                                                                                   
                                                securityGroup[id, name]]],lastKnownPowerState.name,powerState,status,maxCpu,maxMemory,datacenter,activeTransaction[id, transactionStatus[friendlyName,name]],lastOperatingSystemReload.id,blockDevices,blockDeviceTemplateGroup[id, name, globalIdentifier],postInstallScriptUri,operatingSystem[passwords[username,password],
                                           softwareLicense.softwareDescription[                                                                                                                                                                                                                                                                       
                                               manufacturer,name,version,                                                                                                                                                                                                                                                                             
                                               referenceCode]],softwareComponents[                                                                                                                                                                                                                                                                    
                            passwords[username,password,notes],                                                                                                                                                                                                                                                                                       
                            softwareLicense[softwareDescription[                                                                                                                                                                                                                                                                                      
                                                manufacturer,name,version,                                                                                                                                                                                                                                                                            
                                                referenceCode]]],hourlyBillingFlag,userData,billingItem[id,nextInvoiceTotalRecurringAmount,                                                                                                                                                                                                           
                                       children[categoryCode,nextInvoiceTotalRecurringAmount],                                                                                                                                                                                                                                                        
                                       orderItem[id,                                                                                                                                                                                                                                                                                                  
                                                 order.userRecord[username],                                                                                                                                                                                                                                                                          
                                                 preset.keyName]],tagReferences[id,tag[name,id]],networkVlans[id,vlanNumber,networkSpace],dedicatedHost.id]                                                                                                                                                                                           
filter  None                                                                                                                                                                                                                                                                                                                                          
limit   None                                                                                                                                                                                                                                                                                                                                          
offset  None                                                                                                                                                                                                                                                                                                                                          
id      26750507
mask
filter  None
limit   None
offset  None

$ ./slcli -vvv vs detail 26750507 > out.txt
Calling: SoftLayer_Virtual_Guest::getObject(id=26750507)
Starting new HTTPS connection (1): api.softlayer.com
https://api.softlayer.com:443 "POST /xmlrpc/v3.1/SoftLayer_Virtual_Guest HTTP/1.1" 200 None
Returned Data:
{'createDate': '2016-12-12T12:14:23-06:00', 'dedicatedAccountHostOnlyFlag': False, 'domain': 'grunin.com', 'fullyQualifiedDomainName': 'testprov.grunin.com', 'hostname': 'testprov', 'id': 26750507, 'maxCpu': 2, 'maxMemory': 4096, 'modifyDate': '2018-02-20T23:05:41-06:00', 'provisionDate': '2016-12-12T12:17:42-06:00', 'globalIdentifier': '583c0b11-608c-4434-a6db-c15b1d0fd71d', 'hourlyBillingFlag': False, 'primaryBackendIpAddress': '10.142.63.82', 'primaryIpAddress': '169.54.234.110', 'privateNetworkOnlyFlag': False, 'activeTransaction': {'id': 82529705, 'transactionStatus': {'friendlyName': 'Cloud Reclaim Prep', 'name': 'CLOUD_RECLAIM_PREP'}}, 'blockDevices': [{'bootableFlag': 1, 'createDate': '2016-12-12T12:23:38-06:00', 'device': '0', 'diskImageId': 24693963, 'guestId': 26750507, 'hotPlugFlag': 0, 'id': 40069949, 'modifyDate': '2017-10-01T22:24:05-06:00', 'mountMode': 'RW', 'mountType': 'Disk', 'statusId': 1, 'uuid': 'f6e6c63a-1a1e-d91f-7e11-ffd2c102484f'}, {'bootableFlag': 0, 'createDate': '2017-10-01T22:24:10-06:00', 'device': '1', 'diskImageId': 42564603, 'guestId': 26750507, 'hotPlugFlag': 0, 'id': 62071899, 'modifyDate': '', 'mountMode': 'RW', 'mountType': 'Disk', 'statusId': 1, 'uuid': '75a16c81-29a8-24f6-e6a0-d42ca5d8cd11'}], 'datacenter': {'id': 449494, 'longName': 'Dallas 9', 'name': 'dal09', 'statusId': 2}, 'lastOperatingSystemReload': {'id': 44838967}, 'networkComponents': [{'id': 14390461, 'macAddress': '06:fe:83:e9:ca:3a', 'maxSpeed': 100, 'name': 'eth', 'port': 0, 'speed': 100, 'status': 'ACTIVE', 'primaryIpAddress': '10.142.63.82', 'primarySubnet': {'broadcastAddress': '10.142.63.127', 'cidr': 26, 'gateway': '10.142.63.65', 'id': 858931, 'isCustomerOwned': False, 'isCustomerRoutable': False, 'modifyDate': '2017-06-14T11:55:51-06:00', 'netmask': '255.255.255.192', 'networkIdentifier': '10.142.63.64', 'networkVlanId': 804491, 'sortOrder': '4', 'subnetType': 'PRIMARY', 'totalIpAddresses': '64', 'usableIpAddressCount': '61', 'version': 4, 'addressSpace': 'PRIVATE'}, 'securityGroupBindings': []}, {'id': 14390459, 'macAddress': '06:94:25:60:28:ad', 'maxSpeed': 100, 'name': 'eth', 'port': 1, 'speed': 100, 'status': 'ACTIVE', 'primaryIpAddress': '169.54.234.110', 'primarySubnet': {'broadcastAddress': '169.54.234.127', 'cidr': 27, 'gateway': '169.54.234.97', 'id': 858873, 'isCustomerOwned': False, 'isCustomerRoutable': False, 'modifyDate': '2017-06-14T11:55:51-06:00', 'netmask': '255.255.255.224', 'networkIdentifier': '169.54.234.96', 'networkVlanId': 804489, 'sortOrder': '4', 'subnetType': 'PRIMARY', 'totalIpAddresses': '32', 'usableIpAddressCount': '29', 'version': 4}, 'securityGroupBindings': []}], 'networkVlans': [{'id': 804489, 'vlanNumber': 915, 'networkSpace': 'PUBLIC'}, {'id': 804491, 'vlanNumber': 944, 'networkSpace': 'PRIVATE'}], 'operatingSystem': {'hardwareId': '', 'id': 13674565, 'manufacturerLicenseInstance': 'CB993274-2692-4058-BEAE-10B1D0A511CB', 'passwords': [{'password': 'LH3Yzv8w', 'username': 'root'}], 'softwareLicense': {'id': 2497, 'softwareDescriptionId': 1509, 'softwareDescription': {'manufacturer': 'Redhat', 'name': 'EL', 'referenceCode': 'REDHAT_7_64', 'version': '7.0-64 Minimal for VSI'}}}, 'powerState': {'keyName': 'RUNNING', 'name': 'Running'}, 'softwareComponents': [{'hardwareId': '', 'id': 13674565, 'manufacturerLicenseInstance': 'CB993274-2692-4058-BEAE-10B1D0A511CB', 'passwords': [{'password': 'LH3Yzv8w', 'username': 'root'}], 'softwareLicense': {'id': 2497, 'softwareDescriptionId': 1509, 'softwareDescription': {'manufacturer': 'Redhat', 'name': 'EL', 'referenceCode': 'REDHAT_7_64', 'version': '7.0-64 Minimal for VSI'}}}], 'status': {'keyName': 'ACTIVE', 'name': 'Active'}, 'tagReferences': [], 'userData': []}
Calling: SoftLayer_Virtual_Guest::getReverseDomainRecords(id=26750507)
Resetting dropped connection: api.softlayer.com
https://api.softlayer.com:443 "POST /xmlrpc/v3.1/SoftLayer_Virtual_Guest HTTP/1.1" 200 294
Returned Data:
[{'id': 1654863, 'name': '234.54.169.in-addr.arpa', 'serial': 2018022300, 'updateDate': '', 'networkAddress': '169.54.234.0', 'resourceRecords': []}]
execution_time    1.339166s
                            Virtual_Guest::getObject (0.900486s)
                  SoftLayer_Virtual_Guest::getReverseDomainRecords (0.358673s)
version           softlayer-python/v5.4.3
python_version    3.6.2 (v3.6.2:5fd33b5, Jul  8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)]
library_location  C:\Users\allmi\Source\softlayer-python\SoftLayer
id      26750507                                                                                                                                                                                                                                                                                                                                      
mask    mask[id,globalIdentifier,fullyQualifiedDomainName,hostname,domain,createDate,modifyDate,provisionDate,notes,dedicatedAccountHostOnlyFlag,privateNetworkOnlyFlag,primaryBackendIpAddress,primaryIpAddress,networkComponents[id, status, speed, maxSpeed, name,                                                                                 
                                             macAddress, primaryIpAddress, port,                                                                                                                                                                                                                                                                      
                                             primarySubnet,                                                                                                                                                                                                                                                                                           
                                             securityGroupBindings[                                                                                                                                                                                                                                                                                   
                                                securityGroup[id, name]]],lastKnownPowerState.name,powerState,status,maxCpu,maxMemory,datacenter,activeTransaction[id, transactionStatus[friendlyName,name]],lastOperatingSystemReload.id,blockDevices,blockDeviceTemplateGroup[id, name, globalIdentifier],postInstallScriptUri,operatingSystem[passwords[username,password],
                                           softwareLicense.softwareDescription[                                                                                                                                                                                                                                                                       
                                               manufacturer,name,version,                                                                                                                                                                                                                                                                             
                                               referenceCode]],softwareComponents[                                                                                                                                                                                                                                                                    
                            passwords[username,password,notes],                                                                                                                                                                                                                                                                                       
                            softwareLicense[softwareDescription[                                                                                                                                                                                                                                                                                      
                                                manufacturer,name,version,                                                                                                                                                                                                                                                                            
                                                referenceCode]]],hourlyBillingFlag,userData,billingItem[id,nextInvoiceTotalRecurringAmount,                                                                                                                                                                                                           
                                       children[categoryCode,nextInvoiceTotalRecurringAmount],                                                                                                                                                                                                                                                        
                                       orderItem[id,                                                                                                                                                                                                                                                                                                  
                                                 order.userRecord[username],                                                                                                                                                                                                                                                                          
                                                 preset.keyName]],tagReferences[id,tag[name,id]],networkVlans[id,vlanNumber,networkSpace],dedicatedHost.id]                                                                                                                                                                                           
filter  None                                                                                                                                                                                                                                                                                                                                          
limit   None                                                                                                                                                                                                                                                                                                                                          
offset  None                                                                                                                                                                                                                                                                                                                                          
id      26750507
mask
filter  None
limit   None
offset  None
============= testing.py =============
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
from xml.etree import ElementTree
client = requests.Session()
client.headers.update({'Content-Type': 'application/json', 'User-Agent': 'softlayer-python/testing',})
retry = Retry(connect=3, backoff_factor=3)
adapter = HTTPAdapter(max_retries=retry)
client.mount('https://', adapter)
url = 'https://api.softlayer.com/xmlrpc/v3.1/SoftLayer_Virtual_Guest'
payload = """<?xml version='1.0'?> <methodCall> <methodName>getObject</methodName> <params> <param> <value><struct> <member> <name>headers</name> <value><struct> <member> <name>authenticate</name> <value><struct> <member> <name>username</name> <value><string>sl307608-chgallo</string></value> </member> <member> <name>apiKey</name> <value><string>API_KEY_GOES_HERE</string></value> </member> </struct></value> </member> <member> <name>SoftLayer_Virtual_GuestInitParameters</name> <value><struct> <member> <name>id</name> <value><int>26750507</int></value> </member> </struct></value> </member> <member> <name>SoftLayer_ObjectMask</name> <value><struct> <member> <name>mask</name> <value><string>mask[id,globalIdentifier,fullyQualifiedDomainName,hostname,domain,createDate,modifyDate,provisionDate,notes,dedicatedAccountHostOnlyFlag,privateNetworkOnlyFlag,primaryBackendIpAddress,primaryIpAddress,networkComponents[id, status, speed, maxSpeed, name, macAddress, primaryIpAddress, port, primarySubnet, securityGroupBindings[ securityGroup[id, name]]],lastKnownPowerState.name,powerState,status,maxCpu,maxMemory,datacenter,activeTransaction[id, transactionStatus[friendlyName,name]],lastOperatingSystemReload.id,blockDevices,blockDeviceTemplateGroup[id, name, globalIdentifier],postInstallScriptUri,operatingSystem[passwords[username,password], softwareLicense.softwareDescription[ manufacturer,name,version, referenceCode]],softwareComponents[ passwords[username,password,notes], softwareLicense[softwareDescription[ manufacturer,name,version, referenceCode]]],hourlyBillingFlag,userData,billingItem[id,nextInvoiceTotalRecurringAmount, children[categoryCode,nextInvoiceTotalRecurringAmount], orderItem[id, order.userRecord[username], preset.keyName]],tagReferences[id,tag[name,id]],networkVlans[id,vlanNumber,networkSpace],dedicatedHost.id]</string></value> </member> </struct></value> </member> </struct></value> </member> </struct></value> </param> </params> </methodCall> """
transport_headers = {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, compress', 'Content-Type': 'application/xml', 'User-Agent': 'softlayer-python/v5.4.3'}
timeout = None
verify = True
cert = None
proxy = None
response = client.request('POST', url, data=payload, headers=transport_headers, timeout=timeout,
               verify=verify, cert=cert, proxies=proxy)
xml = ElementTree.fromstring(response.content)
ElementTree.dump(xml)
==========================
============= testing.py =============
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
from xml.etree import ElementTree
client = requests.Session()
client.headers.update({'Content-Type': 'application/json', 'User-Agent': 'softlayer-python/testing',})
retry = Retry(connect=3, backoff_factor=3)
adapter = HTTPAdapter(max_retries=retry)
client.mount('https://', adapter)
url = 'https://api.softlayer.com/xmlrpc/v3.1/SoftLayer_Virtual_Guest'
payload = """<?xml version='1.0'?> <methodCall> <methodName>getReverseDomainRecords</methodName> <params> <param> <value><struct> <member> <name>headers</name> <value><struct> <member> <name>authenticate</name> <value><struct> <member> <name>username</name> <value><string>sl307608-chgallo</string></value> </member> <member> <name>apiKey</name> <value><string>API_KEY_GOES_HERE</string></value> </member> </struct></value> </member> <member> <name>SoftLayer_Virtual_GuestInitParameters</name> <value><struct> <member> <name>id</name> <value><int>26750507</int></value> </member> </struct></value> </member> </struct></value> </member> </struct></value> </param> </params> </methodCall> """
transport_headers = {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, compress', 'Content-Type': 'application/xml', 'User-Agent': 'softlayer-python/v5.4.3'}
timeout = None
verify = True
cert = None
proxy = None
response = client.request('POST', url, data=payload, headers=transport_headers, timeout=timeout,
               verify=verify, cert=cert, proxies=proxy)
xml = ElementTree.fromstring(response.content)
ElementTree.dump(xml)
==========================

With rest, the example output looks like this

curl -u $SL_USER:$SL_APIKEY -X GET -H "Accept: */*" -H "Accept-Encoding: gzip, deflate, compress"  'https://api.softlayer.com/rest/v3.1/SoftLayer_Virtual_Guest/26750507/getObject.json?objectMask=mask%5Bid%2CglobalIdentifier%2CfullyQualifiedDomainName%2Chostname%2Cdomain%2CcreateDate%2CmodifyDate%2CprovisionDate%2Cnotes%2CdedicatedAccountHostOnlyFlag%2CprivateNetworkOnlyFlag%2CprimaryBackendIpAddress%2CprimaryIpAddress%2CnetworkComponents%5Bid%2C+status%2C+speed%2C+maxSpeed%2C+name%2C%0A+++++++++++++++++++++++++++++++++++++macAddress%2C+primaryIpAddress%2C+port%2C%0A+++++++++++++++++++++++++++++++++++++primarySubnet%2C%0A+++++++++++++++++++++++++++++++++++++securityGroupBindings%5B%0A++++++++++++++++++++++++++++++++++++++++securityGroup%5Bid%2C+name%5D%5D%5D%2ClastKnownPowerState.name%2CpowerState%2Cstatus%2CmaxCpu%2CmaxMemory%2Cdatacenter%2CactiveTransaction%5Bid%2C+transactionStatus%5BfriendlyName%2Cname%5D%5D%2ClastOperatingSystemReload.id%2CblockDevices%2CblockDeviceTemplateGroup%5Bid%2C+name%2C+globalIdentifier%5D%2CpostInstallScriptUri%2CoperatingSystem%5Bpasswords%5Busername%2Cpassword%5D%2C%0A+++++++++++++++++++++++++++++++++++softwareLicense.softwareDescription%5B%0A+++++++++++++++++++++++++++++++++++++++manufacturer%2Cname%2Cversion%2C%0A+++++++++++++++++++++++++++++++++++++++referenceCode%5D%5D%2CsoftwareComponents%5B%0A++++++++++++++++++++passwords%5Busername%2Cpassword%2Cnotes%5D%2C%0A++++++++++++++++++++softwareLicense%5BsoftwareDescription%5B%0A++++++++++++++++++++++++++++++++++++++++manufacturer%2Cname%2Cversion%2C%0A++++++++++++++++++++++++++++++++++++++++referenceCode%5D%5D%5D%2ChourlyBillingFlag%2CuserData%2CbillingItem%5Bid%2CnextInvoiceTotalRecurringAmount%2C%0A+++++++++++++++++++++++++++++++children%5BcategoryCode%2CnextInvoiceTotalRecurringAmount%5D%2C%0A+++++++++++++++++++++++++++++++orderItem%5Bid%2C%0A+++++++++++++++++++++++++++++++++++++++++order.userRecord%5Busername%5D%2C%0A+++++++++++++++++++++++++++++++++++++++++preset.keyName%5D%5D%2CtagReferences%5Bid%2Ctag%5Bname%2Cid%5D%5D%2CnetworkVlans%5Bid%2CvlanNumber%2CnetworkSpace%5D%2CdedicatedHost.id%5D'
curl -u $SL_USER:$SL_APIKEY -X GET -H "Accept: */*" -H "Accept-Encoding: gzip, deflate, compress"  'https://api.softlayer.com/rest/v3.1/SoftLayer_Virtual_Guest/26750507/getReverseDomainRecords.json'