Home
MClark edited this page Feb 10, 2017
·
4 revisions
In [337]: from nephoria.testcontroller import TestController
In [338]: from nephoria.usercontext import UserContext
A usercontext obj can be created with just Access, Secretkey and region... (if you already have this info)
In [339]: user = UserContext(aws_access_key='AKIAADKUYFA6DADUPO5H', aws_secret_key='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', region='h-14.autoqa.qa1.eucalyptus-systems.com')
Each of these has a nephoria test interface but also has both boto and boto3 interfaces within them. For example: user.ec2.connection as well as the user.ec2.boto2 interfaces are the boto ec2 boto interface. user.ec2.boto3 is the boto3 ec2 interface.
In [340]: user.ec2.get_volumes()
Out[340]: [EuVolume:vol-9d5aed2b, EuVolume:vol-4c2bf377]
In [341]: user.iam.show_all
user.iam.show_all_accounts user.iam.show_all_groups user.iam.show_all_roles user.iam.show_all_roles_for_account user.iam.show_all_users
In [341]: user.iam.show_all_accounts()
[2017-02-09 13:33:33,623][INFO][IAMops()]:
+----------------------------+--------------+
| ACCOUNT_NAME | ACCOUNT_ID |
+----------------------------+--------------+
| (eucalyptus)objectstorage | 000729537082 |
+----------------------------+--------------+
| (eucalyptus)database | 000098481631 |
+----------------------------+--------------+
| (eucalyptus)blockstorage | 000548020223 |
+----------------------------+--------------+
| (eucalyptus)cloudformation | 000614083463 |
+----------------------------+--------------+
| (eucalyptus)imaging | 000611806542 |
+----------------------------+--------------+
| (eucalyptus)loadbalancing | 000279278370 |
+----------------------------+--------------+
| (eucalyptus)aws-exec-read | 000975080808 |
+----------------------------+--------------+
| eucalyptus | 000445377749 |
+----------------------------+--------------+
| testrunner | 000365342570 |
+----------------------------+--------------+
If you dont want to store your users creds in a script, files, etc. 'and' you have root access to your CLC (either locally or via remote ssh) you can
use the testcontroller class to find and populate the creds and region for you. This is also a way to recover lost creds w/o creating new keys...
In [342]: tc = TestController('10.10.5.10', password='mypassword', log_level='DEBUG')
In [344]: tc.get_user_by_name('testrunner', 'admin')
Out[344]: UserContext:000365342570::testrunner::admin
In [346]: user.iam.show_user_summary()
[2017-02-09 13:34:27,007][INFO][IAMops(admin:testrunner)]:
+-------------------------------------------------------------------+
| USER SUMMARY: user:admin, account:testrunner |
+-------------------------------------------------------------------+
| +------------+-----------+-----------------------+--------------+ |
| | ACCOUNT: | USERNAME: | USER_ID | ACCT_ID | |
| +------------+-----------+-----------------------+--------------+ |
| | testrunner | admin | AIDAAOC343YGZHW5IHASJ | 000365342570 | |
| +------------+-----------+-----------------------+--------------+ |
| |
| +---------------------------------------------------------------+ |
| | POLICIES FOR USER: admin | |
| +---------------------------------------------------------------+ |
| | -- No Policies -- | |
| +---------------------------------------------------------------+ |
+-------------------------------------------------------------------+
In [347]: user.access_key
Out[347]: 'AKIAAZQFBCJ22MVVFKX3'
In [348]: user.secret_key
Out[348]: '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'
In [350]: user = tc.create_user_using_cloudadmin('matt', 'admin')
In [351]: user.iam.show_user_summary()
[2017-02-09 13:35:03,797][INFO][IAMops(admin:matt)]:
+-----------------------------------------------------------------+
| USER SUMMARY: user:admin, account:matt |
+-----------------------------------------------------------------+
| +----------+-----------+-----------------------+--------------+ |
| | ACCOUNT: | USERNAME: | USER_ID | ACCT_ID | |
| +----------+-----------+-----------------------+--------------+ |
| | matt | admin | AIDAATK5QBVTCC7ZWRFZ3 | 000959096634 | |
| +----------+-----------+-----------------------+--------------+ |
| |
| +-------------------------------------------------------------+ |
| | POLICIES FOR USER: admin | |
| +-------------------------------------------------------------+ |
| | -- No Policies -- | |
| +-------------------------------------------------------------+ |
+-----------------------------------------------------------------+
####fetch creds for an existing user
In [353]: user = tc.get_user_by_name('testrunner', 'admin')
In [354]: user.
user.CLASS_MAP user.aws_secret_access_key user.ec2_access_key user.get_eucarc_attrs user.secret_key_id
user.access_key user.aws_secret_access_key_id user.ec2_access_key_id user.get_existing_keys_from_clc user.service_port
user.access_key_id user.aws_secret_key user.ec2_account_id user.get_local_eucarc user.serviceconnection
user.account_id user.aws_secret_key_id user.ec2_account_name user.get_remote_eucarc user.session
user.account_name user.aws_simpleworkflow_url user.ec2_account_number user.get_urls user.show
user.assume_role_on_remote_clc user.aws_user_name user.ec2_cert user.iam user.simpleworkflow_url
user.auto_find_credentials user.bootstrap_url user.ec2_jvm_args user.iam_url user.sqs
user.auto_scaling_url user.boto2_api_version user.ec2_private_key user.info user.sqs_url
user.autoscaling user.cloudformation user.ec2_secret_key user.is_https user.sts
user.aws_access_key user.cloudformation_url user.ec2_secret_key_id user.key_id user.sts_url
user.aws_access_key_id user.cloudwatch user.ec2_url user.keys_dir user.swf
user.aws_account_id user.cloudwatch_url user.ec2_user_id user.log user.swf_url
user.aws_account_name user.connect_to_creds_machine user.elb user.properties_url user.test_resources
user.aws_auto_scaling_url user.create_ini_file user.elb_url user.region user.token_url
user.aws_cloudformation_url user.create_local_creds user.euare_url user.region_domain user.update_attrs_from_cloud_services
user.aws_cloudwatch_url user.creds_machine user.euca_bootstrap_url user.reporting_url user.update_region_and_domain
user.aws_credential_file user.critical user.euca_properties_url user.s3 user.user_id
user.aws_elb_url user.domain user.eucalyptus_cert user.s3_url user.user_info
user.aws_iam_url user.ec2 user.eustore_url user.secret_key user.user_name
This method can be used to create the old school runtime config files (ie eucarc files) for a usercontext. This can write to a local or remote directly I believe. .
In [354]: user.create_local_creds?
Type: instancemethod
String form: <bound method UserContext.create_local_creds of UserContext:000365342570::testrunner::admin>
File: /Library/Python/2.7/site-packages/adminapi-1.4.3.0.1.1-py2.7.egg/cloud_admin/access/autocreds.py
Definition: user.create_local_creds(self, local_destdir, machine=None, keydir=None, overwrite=False, zipfilename=None, ziponly=False)
Docstring:
Attempts to create a local set of files containing the current credential artifacts
in this AutoCreds obj. The following files will be written to the provided
'local_destdir' directory:
- A eucarc file containing the "export key=value" syntax to resolve service urls
and the location of any credentials related files.
- Any current attributes with an sftp:// uri will be downloaded to local_destdir. At this
time the AutoCred eucarc attributes will be updated to represent their now local filepath,
an the local eucarc written will also reflect the new location.
:param local_destdir: local directory to write cred files to.
Will create if does not exist.
:param machine: The Machine() obj to download any sftp:// files from
:param keydir: optional String representing path to key dir, otherwise auto-populated
:param overwrite: bool, if True will overwrite any existing items at 'local_destdir'
:param zipfilename: string representing the zip archive filename to be creat in the
'local_destdir' directory. If "None" a zip archive will not be created.
:param ziponly: boolean, if true only a zip archive will be created
:return: list of filepaths
Or any usercontext can dump it's info to stdout or a logfile (if one was provided to the constructor)
In [355]: user.show()
[2017-02-09 13:35:33,252][INFO][UserContext:000365342570::testrunner::admin]:
+--------------------------+--------------------------------------------------------------------------+
| access_key | AKIAAZQFBCJ22MVVFKX3 |
+--------------------------+--------------------------------------------------------------------------+
| access_key_id | AKIAAZQFBCJ22MVVFKX3 |
+--------------------------+--------------------------------------------------------------------------+
| account_id | 000365342570 |
+--------------------------+--------------------------------------------------------------------------+
| account_name | testrunner |
+--------------------------+--------------------------------------------------------------------------+
| auto_scaling_url | http://autoscaling.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| aws_access_key | AKIAAZQFBCJ22MVVFKX3 |
+--------------------------+--------------------------------------------------------------------------+
| aws_access_key_id | AKIAAZQFBCJ22MVVFKX3 |
+--------------------------+--------------------------------------------------------------------------+
| aws_account_id | 000365342570 |
+--------------------------+--------------------------------------------------------------------------+
| aws_account_name | testrunner |
+--------------------------+--------------------------------------------------------------------------+
| aws_auto_scaling_url | http://autoscaling.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| aws_cloudformation_url | http://cloudformation.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| aws_cloudwatch_url | http://monitoring.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| aws_credential_file | None |
+--------------------------+--------------------------------------------------------------------------+
| aws_elb_url | http://elasticloadbalancing.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| aws_iam_url | https://iam.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| aws_secret_access_key | Gh87lIU9iC5rOoYRFWDD4DjHdXBqWfQAndJj3Zuf |
+--------------------------+--------------------------------------------------------------------------+
| aws_secret_access_key_id | Gh87lIU9iC5rOoYRFWDD4DjHdXBqWfQAndJj3Zuf |
+--------------------------+--------------------------------------------------------------------------+
| aws_secret_key | Gh87lIU9iC5rOoYRFWDD4DjHdXBqWfQAndJj3Zuf |
+--------------------------+--------------------------------------------------------------------------+
| aws_secret_key_id | Gh87lIU9iC5rOoYRFWDD4DjHdXBqWfQAndJj3Zuf |
+--------------------------+--------------------------------------------------------------------------+
| aws_simpleworkflow_url | http://swf.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| aws_user_name | admin |
+--------------------------+--------------------------------------------------------------------------+
| bootstrap_url | http://bootstrap.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| boto2_api_version | 2015-10-01 |
+--------------------------+--------------------------------------------------------------------------+
| cloudformation_url | http://cloudformation.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| cloudwatch_url | http://monitoring.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| ec2_access_key | AKIAAZQFBCJ22MVVFKX3 |
+--------------------------+--------------------------------------------------------------------------+
| ec2_access_key_id | AKIAAZQFBCJ22MVVFKX3 |
+--------------------------+--------------------------------------------------------------------------+
| ec2_account_id | 000365342570 |
+--------------------------+--------------------------------------------------------------------------+
| ec2_account_name | testrunner |
+--------------------------+--------------------------------------------------------------------------+
| ec2_account_number | 000365342570 |
+--------------------------+--------------------------------------------------------------------------+
| ec2_cert | None |
+--------------------------+--------------------------------------------------------------------------+
| ec2_jvm_args | None |
+--------------------------+--------------------------------------------------------------------------+
| ec2_private_key | None |
+--------------------------+--------------------------------------------------------------------------+
| ec2_secret_key | Gh87lIU9iC5rOoYRFWDD4DjHdXBqWfQAndJj3Zuf |
+--------------------------+--------------------------------------------------------------------------+
| ec2_secret_key_id | Gh87lIU9iC5rOoYRFWDD4DjHdXBqWfQAndJj3Zuf |
+--------------------------+--------------------------------------------------------------------------+
| ec2_url | http://ec2.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| ec2_user_id | 000365342570 |
+--------------------------+--------------------------------------------------------------------------+
| elb_url | http://elasticloadbalancing.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| euare_url | https://iam.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| euca_bootstrap_url | http://bootstrap.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| euca_properties_url | http://properites.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| eucalyptus_cert | None |
+--------------------------+--------------------------------------------------------------------------+
| eustore_url | http://emis.eucalyptus.com/ |
+--------------------------+--------------------------------------------------------------------------+
| iam_url | https://iam.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| key_id | AKIAAZQFBCJ22MVVFKX3 |
+--------------------------+--------------------------------------------------------------------------+
| keys_dir | None |
+--------------------------+--------------------------------------------------------------------------+
| properties_url | http://properites.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| reporting_url | http://reporting.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| s3_url | http://s3.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| secret_key | Gh87lIU9iC5rOoYRFWDD4DjHdXBqWfQAndJj3Zuf |
+--------------------------+--------------------------------------------------------------------------+
| secret_key_id | Gh87lIU9iC5rOoYRFWDD4DjHdXBqWfQAndJj3Zuf |
+--------------------------+--------------------------------------------------------------------------+
| simpleworkflow_url | http://swf.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| sqs_url | http://sqs.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| sts_url | https://sts.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| swf_url | http://swf.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| token_url | https://sts.h-14.autoqa.qa1.eucalyptus-systems.com:8773/ |
+--------------------------+--------------------------------------------------------------------------+
| user_id | 000365342570 |
+--------------------------+--------------------------------------------------------------------------+
| user_name | admin |
+--------------------------+--------------------------------------------------------------------------+
| UNPARSED LINES | None |
+--------------------------+--------------------------------------------------------------------------+
The testcontroller.sysadmin interface is a combination host/linux and empyrean service interfaces. All of these are defined in the AdminApi project as they are not relevant to AWS, only Euca, Linux, system adminstration, etc..
In [33] from cloud_admin.services.serviceconnection import ServiceConnection
In [34]: from cloud_admin.systemconnection import SystemConnection
In [35]: sysadmin = SystemConnection?
Type: type
String form: <class 'cloud_admin.systemconnection.SystemConnection'>
File: /Library/Python/2.7/site-packages/adminapi-1.4.3.0.1.1-py2.7.egg/cloud_admin/systemconnection.py
Init definition: SystemConnection(self, hostname, username='root', password=None, keypath=None, domain=None, proxy_hostname=None, proxy_username=None, proxy_password=None, proxy_keypath=None, config_yml=None, config_qa=None, credpath=None, aws_access_key=None, aws_secret_key=None, log_level='INFO', boto_debug_level=0, euca_user='admin', euca_account='eucalyptus', https=True)
Docstring: <no docstring>
In [37]: sysadmin = SystemConnection('10.10.5.10', password='mypassword')
The sys admin interface can show the status of backend from a services and linux host perspective....
In [40]: tc.sysadmin
Out[40]: 10.10.1.72:SystemConnection
### show general info about a cloud's backend
In [41]: tc.sysadmin.show
tc.sysadmin.show_cloud_controllers tc.sysadmin.show_components_summary tc.sysadmin.show_objectstorage_gateways tc.sysadmin.show_service_types_verbose
tc.sysadmin.show_cloud_legacy_summary tc.sysadmin.show_hosts tc.sysadmin.show_properties tc.sysadmin.show_services
tc.sysadmin.show_cluster_controllers tc.sysadmin.show_machine_mappings tc.sysadmin.show_properties_narrow tc.sysadmin.show_storage_controllers
tc.sysadmin.show_cluster_mappings tc.sysadmin.show_nodes tc.sysadmin.show_service_types tc.sysadmin.show_walrus_backends
In [41]: tc.sysadmin.show_services()
[2017-02-09 13:55:59,583][INFO][UserContext::eucalyptus:admin]:
+-------------------+-------------------------------+-------+-------+----------------------------------------------------------------+
|TYPE |NAME |STATE |CLUSTER|URI |
+-------------------+-------------------------------+-------+-------+----------------------------------------------------------------+
|arbitrator |NOT REGISTERED? |MISSING| -- |SERVICE NOT REGISTERED |
|walrusbackend |walrus-0 |ENABLED| |http://10.10.5.138:8773/services/WalrusBackend |
|storage |one-sc-0 |ENABLED| one |http://10.10.5.138:8773/services/Storage |
|cluster |one-cc-0 |ENABLED| one |http://10.10.5.138:8774/axis2/services/EucalyptusCC |
|tokens |API_10.10.5.138.tokens |ENABLED| |http://10.10.5.138:8773/services/Tokens |
|simpleworkflow |API_10.10.5.138.simpleworkflow|ENABLED| |http://10.10.5.138:8773/services/SimpleWorkflow |
|objectstorage |API_10.10.5.138.objectstorage |ENABLED| |http://10.10.5.138:8773/services/objectstorage |
|loadbalancing |API_10.10.5.138.loadbalancing |ENABLED| |http://10.10.5.138:8773/services/LoadBalancing |
|imaging |API_10.10.5.138.imaging |ENABLED| |http://10.10.5.138:8773/services/Imaging |
|identity |API_10.10.5.138.identity |ENABLED| |http://10.10.5.138:8773/services/Identity |
|euare |API_10.10.5.138.euare |ENABLED| |http://10.10.5.138:8773/services/Euare |
|dns |API_10.10.5.138.dns |ENABLED| |http://10.10.5.138:8773/services/Dns |
|compute |API_10.10.5.138.compute |ENABLED| |http://10.10.5.138:8773/services/compute |
|cloudwatch |API_10.10.5.138.cloudwatch |ENABLED| |http://10.10.5.138:8773/services/CloudWatch |
|cloudformation |API_10.10.5.138.cloudformation|ENABLED| |http://10.10.5.138:8773/services/CloudFormation |
|autoscaling |API_10.10.5.138.autoscaling |ENABLED| |http://10.10.5.138:8773/services/AutoScaling |
|user-api |API_10.10.5.138 |ENABLED| |http://10.10.5.138:8773/services/User-API |
|node |10.10.1.94 |ENABLED| one |http://10.10.1.94:8775/axis2/services/EucalyptusNC |
|eucalyptus |10.10.1.72 |ENABLED| |http://10.10.1.72:8773/services/Eucalyptus |
|node |10.10.1.208 |ENABLED| one |http://10.10.1.208:8775/axis2/services/EucalyptusNC |
|component |10.10.1.72 |ENABLED| |http://10.10.1.72:8773/services/Component |
|reporting |10.10.1.72 |ENABLED| |http://10.10.1.72:8773/services/Reporting |
|bootstrap |10.10.1.72 |ENABLED| |http://10.10.1.72:8773/services/Empyrean |
|properties |10.10.1.72 |ENABLED| |http://10.10.1.72:8773/services/Properties |
|imagingbackend |10.10.1.72 |ENABLED| |http://10.10.1.72:8773/services/ImagingBackend |
|configuration |10.10.1.72 |ENABLED| |http://10.10.1.72:8773/services/Configuration |
|ldap |10.10.1.72 |ENABLED| |http://10.10.1.72:8773/services/Ldap |
|cloudwatchbackend |10.10.1.72 |ENABLED| |http://10.10.1.72:8773/services/CloudWatchBackend |
|notifications |10.10.1.72 |ENABLED| |http://10.10.1.72:8773/services/Notifications |
|autoscalingbackend |10.10.1.72 |ENABLED| |http://10.10.1.72:8773/services/AutoScalingBackend |
|db |10.10.1.72 |ENABLED| |postgresql://10.10.1.72:8777/eucalyptus?ssl=true&sslfacto |
| | | | |ry=com.eucalyptus.postgresql.PostgreSQLSSLSocketFactory |
|pollednotifications|10.10.1.72 |ENABLED| |http://10.10.1.72:8773/services/PolledNotifications |
+-------------------+-------------------------------+-------+-------+----------------------------------------------------------------+
In [42]: tc.sysadmin.show_prop
tc.sysadmin.show_properties tc.sysadmin.show_properties_narrow
In [42]: tc.sysadmin.show_properties('network')
[2017-02-09 13:56:13,012][INFO][UserContext::eucalyptus:admin]:
+------------------------------------------------------+-------------------------------+-------+--+----------------------------------------+
|PROPERTY NAME |PROPERTY VALUE |DEFAULT|RO|DESCRIPTION |
+------------------------------------------------------+-------------------------------+-------+--+----------------------------------------+
|cloud.network.address_pending_timeout |35 |35 |F |Minutes before a pending system public |
| | | | |address allocation timesout and is |
| | | | |released. |
+------------------------------------------------------+-------------------------------+-------+--+----------------------------------------+
|cloud.network.ec2_classic_additional_protocols_allowed| | |F |Comma delimited list of protocol numbers|
| | | | |to support in EDGE mode for security |
| | | | |group rules beyond the EC2-classic |
| | | | |defaults (tcp,udp,icmp) |
+------------------------------------------------------+-------------------------------+-------+--+----------------------------------------+
|cloud.network.max_broadcast_apply |120 |120 |F |Maximum time to apply network |
| | | | |information (seconds). |
+------------------------------------------------------+-------------------------------+-------+--+----------------------------------------+
|cloud.network.min_broadcast_interval |5 |5 |F |Minimum interval between broadcasts of |
| | | | |network information (seconds). |
+------------------------------------------------------+-------------------------------+-------+--+----------------------------------------+
|cloud.network.network_configuration |{ | |F |Network configuration document. |
| | "Clusters": [ | | | |
| | { | | | |
| | "Name": "one", | | | |
| | "PrivateIps": [ | | | |
| | "10.10.75.62", | | | |
| | "10.10.75.63", | | | |
| | "10.10.75.76", | | | |
| | "10.10.75.77", | | | |
| | "10.10.75.78", | | | |
| | "10.10.75.79", | | | |
| | "10.10.75.80", | | | |
| | "10.10.75.81" | | | |
| | ], | | | |
| | "Subnet": { | | | |
| | "Gateway": "10.10.0.1", | | | |
| | "Name": "10.10.0.0", | | | |
| | "Netmask": "255.255.0.0",| | | |
| | "Subnet": "10.10.0.0" | | | |
| | } | | | |
| | } | | | |
| |], | | | |
| | "InstanceDnsServers": [ | | | |
| | "10.10.5.138" | | | |
| | ], | | | |
| | "Mode": "EDGE", | | | |
| | "PublicIps": [ | | | |
| | "10.10.75.52", | | | |
| | "10.10.75.53", | | | |
| | "10.10.75.56", | | | |
| | "10.10.75.57", | | | |
| | "10.10.75.58", | | | |
| | "10.10.75.59", | | | |
| | "10.10.75.60", | | | |
| | "10.10.75.61" | | | |
| | ] | | | |
| |} | | | |
+------------------------------------------------------+-------------------------------+-------+--+----------------------------------------+
|cloud.network.network_index_pending_timeout |35 |35 |F |Minutes before a pending index |
| | | | |allocation timesout and is released. |
+------------------------------------------------------+-------------------------------+-------+--+----------------------------------------+
|cloud.vmstate.network_metadata_refresh_time |15 |3600000|F |Maximum amount of time (in seconds) that|
| | | | |the network topology service takes to |
| | | | |propagate state changes. |
+------------------------------------------------------+-------------------------------+-------+--+----------------------------------------+
|cloud.vpc.networkaclspervpc |200 |200 |F |Maximum number of network ACLs for each |
| | | | |VPC. |
+------------------------------------------------------+-------------------------------+-------+--+----------------------------------------+
|cloud.vpc.rulespernetworkacl |20 |20 |F |Maximum number of rules per direction |
| | | | |for each network ACL. |
+------------------------------------------------------+-------------------------------+-------+--+----------------------------------------+
|cloud.vpc.securitygroupspernetworkinterface |5 |5 |F |Maximum number of associated security |
| | | | |groups for each network interface . |
+------------------------------------------------------+-------------------------------+-------+--+----------------------------------------+
|one.cluster.networkmode |EDGE | |T |Currently configured network mode |
+------------------------------------------------------+-------------------------------+-------+--+----------------------------------------+
In [43]: prop = tc.sysadmin.get_property('objectstorage.max_metadata_request_size')
In [44]: prop.show()
[2017-02-09 13:56:40,446][INFO][UserContext::eucalyptus:admin]:
+---------------------------------------+--------------+-------+--+----------------------------------------+
|PROPERTY NAME |PROPERTY VALUE|DEFAULT|RO|DESCRIPTION |
+---------------------------------------+--------------+-------+--+----------------------------------------+
|objectstorage.max_metadata_request_size|307200 | |F |Maximum allowed size of metadata request|
| | | | |bodies |
+---------------------------------------+--------------+-------+--+----------------------------------------+
In [45]: prop.modify_value('311111')
[2017-02-09 13:56:57,992][INFO][UserContext::eucalyptus:admin]:
+---------------------------------------+--------------+-------+--+
|PROPERTY NAME |PROPERTY VALUE|DEFAULT|RO|
+---------------------------------------+--------------+-------+--+
|objectstorage.max_metadata_request_size|311111 |None |? |
+---------------------------------------+--------------+-------+--+
Out[45]: EucaProperty:objectstorage.max_metadata_request_size
A eucahost combines the Adminapi machine class with eucalyptus service utilities specific to the host's role or services it's serving. An ssh connection is setup automatically along with relevant AWS service connections to represent the 'eucahost' obj..
from cloud_admin.hosts.eucahost import EucaHost
from cloud_utils.system_utils.machine import Machine
Show all hosts, the host info (cpu, mem, disk) as well as the service states, VMs on that machine, relevant processes.
In [46]: tc.sysadmin.show_hosts()
[2017-02-09 13:57:14,924][INFO][UserContext::eucalyptus:admin]:
+--------------------------------+-----------------------------------------------------------------------------+
| HOST:10.10.1.72 | EUCALYPTUS SERVICES:[ CLC ] |
+--------------------------------+-----------------------------------------------------------------------------+
| Euca Versions: | TYPE NAME STATE CLUSTER |
| Cloud: 4.4.0 | eucalyptus 10.10.1.72 ENABLED |
| 2ools: 3.4.1-1-gf960997 | |
| Hostname: | ------------------------------------------------------------- |
| 10.10.1.72 | HOST SERVICE COMMAND %CPU %MEM PS_UPTIME |
| Mem: | ------------------------------------------------------------- |
| Used: 4416 56.5% | eucalyptus: |
| Free: 202 2.6% | ---> eucalyptus-clou 5.2 50.1 6-17:49:19 |
| Swap: 206 2.6% | ------------------------------------------------------------- |
| CPU: | |
| #0: 1.94% | |
| #1: 2.55% | |
| #2: 2.57% | |
| #3: 2.9% | |
| DISK: | |
| md0 509m 27% | |
| vg01-lv_root 200g 4% | |
| tmpfs 782m 0% | |
| devtmpfs 3.9g 0% | |
+--------------------------------+-----------------------------------------------------------------------------+
| HOST:10.10.1.208 | EUCALYPTUS SERVICES:[ NC ] |
+--------------------------------+-----------------------------------------------------------------------------+
| Euca Versions: | TYPE NAME STATE CLUSTER |
| Cloud: 4.4.0 | node 10.10.1.208 ENABLED one |
| 2ools: 3.4.1-1-gf960997 | ----------------------------------------------------- |
| Hostname: | INSTANCES STATE: VMTYPE: ROOT_DEV: |
| 10.10.1.208 | ----------------------------------------------------- |
| Mem: | i-6a8d83ae running m1.small instance-store |
| Used: 1045 13.4% | i-c2c06544 running c1.medium ebs |
| Free: 2544 32.6% | i-ec465278 running c1.medium ebs |
| Swap: 0 0.0% | i-90335731 running c1.medium ebs |
| CPU: | ----------------------------------------------------- |
| #0: 0.38% | ("10.10.1.208"'s VM AVAILABILITY @ None) |
| #1: 0.4% | ------------------------------------------------------------ |
| #2: 0.4% | VMTYPE AVAIL CPU(None/None) MEM(None/None) DISK(None/None) |
| #3: 0.54% | ------------------------------------------------------------ |
| DISK: | ------------------------------------------------------------ |
| md0 509m 27% | -------------------------------------------------------- |
| vg01-lv_root 200g 3% | HOST SERVICE COMMAND %CPU %MEM PS_UPTIME |
| tmpfs 782m 0% | -------------------------------------------------------- |
| devtmpfs 3.9g 0% | eucalyptus-nc: |
| | ---> httpd 0.3 0.6 6-17:40:33 |
| | eucanetd: |
| | ---> eucanetd 0.0 0.0 6-17:40:34 |
| | -------------------------------------------------------- |
+--------------------------------+-----------------------------------------------------------------------------+
| HOST:10.10.1.94 | EUCALYPTUS SERVICES:[ NC ] |
+--------------------------------+-----------------------------------------------------------------------------+
| Euca Versions: | TYPE NAME STATE CLUSTER |
| Cloud: 4.4.0 | node 10.10.1.94 ENABLED one |
| 2ools: 3.4.1-1-gf960997 | ----------------------------------------------------- |
| Hostname: | INSTANCES STATE: VMTYPE: ROOT_DEV: |
| 10.10.1.94 | ----------------------------------------------------- |
| Mem: | i-d161ea01 running c1.medium ebs |
| Used: 1024 13.1% | i-33398373 running m1.small instance-store |
| Free: 852 10.9% | i-d94fda93 running c1.medium ebs |
| Swap: 0 0.0% | i-9c050fa2 running c1.medium ebs |
| CPU: | ----------------------------------------------------- |
| #0: 0.71% | ("10.10.1.94"'s VM AVAILABILITY @ None) |
| #1: 0.49% | ------------------------------------------------------------ |
| #2: 0.46% | VMTYPE AVAIL CPU(None/None) MEM(None/None) DISK(None/None) |
| #3: 0.39% | ------------------------------------------------------------ |
| DISK: | ------------------------------------------------------------ |
| md0 509m 27% | -------------------------------------------------------- |
| vg01-lv_root 200g 4% | HOST SERVICE COMMAND %CPU %MEM PS_UPTIME |
| tmpfs 782m 0% | -------------------------------------------------------- |
| devtmpfs 3.9g 0% | eucalyptus-nc: |
| | ---> httpd 0.4 0.6 6-17:40:58 |
| | eucanetd: |
| | ---> eucanetd 0.0 0.0 6-17:41:00 |
| | -------------------------------------------------------- |
+--------------------------------+-----------------------------------------------------------------------------+
| HOST:10.10.5.138 | EUCALYPTUS SERVICES:[ UFS SC CC WS ] |
+--------------------------------+-----------------------------------------------------------------------------+
| Euca Versions: | TYPE NAME STATE CLUSTER |
| Cloud: 4.4.0 | user-api API_10.10.5.138 ENABLED |
| 2ools: 3.4.1-1-gf960997 | autoscaling API_10.10.5.138.autoscaling ENABLED |
| Hostname: | cloudformation API_10.10.5.138.cloudformation ENABLED |
| 10.10.5.138 | cloudwatch API_10.10.5.138.cloudwatch ENABLED |
| Mem: | compute API_10.10.5.138.compute ENABLED |
| Used: 5478 70.1% | dns API_10.10.5.138.dns ENABLED |
| Free: 183 2.3% | euare API_10.10.5.138.euare ENABLED |
| Swap: 476 6.1% | identity API_10.10.5.138.identity ENABLED |
| CPU: | imaging API_10.10.5.138.imaging ENABLED |
| #0: 2.51% | loadbalancing API_10.10.5.138.loadbalancing ENABLED |
| #1: 3.07% | objectstorage API_10.10.5.138.objectstorage ENABLED |
| #2: 3.28% | simpleworkflow API_10.10.5.138.simpleworkflow ENABLED |
| #3: 3.71% | tokens API_10.10.5.138.tokens ENABLED |
| DISK: | storage one-sc-0 ENABLED one |
| md0 509m 27% | cluster one-cc-0 ENABLED one |
| vg01-lv_root 200g 10% | walrusbackend walrus-0 ENABLED |
| tmpfs 782m 0% | |
| devtmpfs 3.9g 0% | -------------------------------------------------------------- |
| | HOST SERVICE COMMAND %CPU %MEM PS_UPTIME |
| | -------------------------------------------------------------- |
| | eucalytus-cc: |
| | ---> httpd 0.0 0.5 6-16:51:32 |
| | eucalyptus: |
| | ---> eucalyptus-clou 11.0 63.0 6-17:43:44 |
| | -------------------------------------------------------------- |
+--------------------------------+-----------------------------------------------------------------------------+
This can be done by 'tc.sysadmin.get_hosts_by_service_type(servicetype) or by a few canned service methods. Example below is a canned method for fetching all the node controller host objs..
In [48]: node = tc.sysadmin.get_hosts_for_node_controllers(instanceid='i-33398373')[0]
In [50]: instance_xml = node.helpers.node_controller.get_instance_xml_text('i-33398373')
In [51]: print instance_xml
<domain type='kvm' id='4'>
<name>i-33398373</name>
<uuid>30c8a998-50e7-44a4-b8a3-376646f258d1</uuid>
<description>Eucalyptus instance i-33398373</description>
<memory unit='KiB'>262144</memory>
<currentMemory unit='KiB'>262144</currentMemory>
<vcpu placement='static'>1</vcpu>
<resource>
<partition>/machine</partition>
</resource>
<os>
<type arch='x86_64' machine='pc-i440fx-rhel7.3.0'>hvm</type>
<boot dev='hd'/>
</os>
<features>
<acpi/>
</features>
<cpu>
<topology sockets='1' cores='1' threads='1'/>
</cpu>
<clock offset='localtime'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/libexec/qemu-kvm</emulator>
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source dev='/var/lib/eucalyptus/instances/work/AIDAATPHVGYOL5PQUR5IL/i-33398373/link-to-vda'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
<serial>euca-bdm-machine-dev-vda</serial>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</disk>
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source dev='/var/lib/eucalyptus/instances/work/AIDAATPHVGYOL5PQUR5IL/i-33398373/link-to-vdb'/>
<backingStore/>
<target dev='vdb' bus='virtio'/>
<serial>euca-bdm-ephemeral0-dev-vdb</serial>
<alias name='virtio-disk1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</disk>
<disk type='file' device='floppy'>
<driver name='qemu' type='raw' cache='none'/>
<source file='/var/lib/eucalyptus/instances/work/AIDAATPHVGYOL5PQUR5IL/i-33398373/floppy'/>
<backingStore/>
<target dev='fda' bus='fdc'/>
<alias name='fdc0-0-0'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
<controller type='usb' index='0'>
<alias name='usb'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
</controller>
<controller type='pci' index='0' model='pci-root'>
<alias name='pci.0'/>
</controller>
<controller type='fdc' index='0'>
<alias name='fdc0'/>
</controller>
<interface type='bridge'>
<mac address='d0:0d:33:39:83:73'/>
<source bridge='br0'/>
<target dev='vn_i-33398373'/>
<model type='virtio'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</interface>
<serial type='file'>
<source path='/var/lib/eucalyptus/instances/work/AIDAATPHVGYOL5PQUR5IL/i-33398373/console.log'/>
<target port='1'/>
<alias name='serial0'/>
</serial>
<console type='file'>
<source path='/var/lib/eucalyptus/instances/work/AIDAATPHVGYOL5PQUR5IL/i-33398373/console.log'/>
<target type='serial' port='1'/>
<alias name='serial0'/>
</console>
<input type='mouse' bus='ps2'>
<alias name='input0'/>
</input>
<input type='keyboard' bus='ps2'>
<alias name='input1'/>
</input>
<memballoon model='virtio'>
<alias name='balloon0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
</memballoon>
</devices>
<seclabel type='dynamic' model='selinux' relabel='yes'>
<label>system_u:system_r:svirt_t:s0:c66,c577</label>
<imagelabel>system_u:object_r:svirt_image_t:s0:c66,c577</imagelabel>
</seclabel>
<seclabel type='dynamic' model='dac' relabel='yes'>
<label>+107:+107</label>
<imagelabel>+107:+107</imagelabel>
</seclabel>
</domain>
In [54]: for host in tc.sysadmin.get_hosts_for_storage_controllers():
print host.sys('echo "My hostname is $(hostname)"', code=0)
....:
['My hostname is d-14.qa1.eucalyptus-systems.com']
In [59]: for host in tc.sysadmin.get_hosts_for_node_controllers():
print host.sys('echo "My hostname is $(hostname)"', code=0)
....:
['My hostname is d-24.qa1.eucalyptus-systems.com']
['My hostname is e-17.qa1.eucalyptus-systems.com']
In [60]: host.sys('ls filedoesnotexist')
Out[60]: ['ls: cannot access filedoesnotexist: No such file or directory']
In [61]: host.sys('ls filedoesnotexist', code=0)
CommandExitCodeException: "Cmd:ls filedoesnotexist failed with status code:2, output:['ls: cannot access filedoesnotexist: No such file or directory']"
tc.sysadmin._eucahosts is a dict of hosts with the {ip: hostobj}. Example:
In [78]: for ip, host in tc.sysadmin._eucahosts.iteritems():
print "ip:{0} ---> {1}".format(ip, host.get_eucalyptus_version())
....:
In [79]: for ip, host in tc.sysadmin._eucahosts.iteritems():
host.log.info("ip:{0} ---> {1}".format(ip, host.get_eucalyptus_version()))
....:
[2017-02-09 17:48:59,460][INFO][10.111.5.217:(['CLC'])]:
ip:10.111.5.217 ---> 4.4.0
[2017-02-09 17:48:59,596][INFO][10.111.1.22:(['UFS', 'SC', 'CC', 'WS'])]:
ip:10.111.1.22 ---> 4.4.0
[2017-02-09 17:48:59,761][INFO][10.111.1.2:(['SC', 'CC'])]:
ip:10.111.1.2 ---> 4.4.0
[2017-02-09 17:48:59,928][INFO][10.111.1.19:(['NC'])]:
ip:10.111.1.19 ---> 4.4.0
[2017-02-09 17:49:00,096][INFO][10.111.5.190:(['NC'])]:
ip:10.111.5.190 ---> 4.4.0
Fetch an image, show image, filter by the name or location string of the image and root_device_type...
In [87]: emi = user.ec2.get_emi(root_device_type='instance-store', location='precise')
In [88]: emi
Out[88]: Image:emi-2a68fc7f
In [89]: user.ec2.show_image(emi)
[2017-02-09 17:53:40,620][INFO][EC2ops(admin:testrunner)]:
+----------------------------------------------------------------------------------------------------------+
|IMAGE ID: emi-2a68fc7f, IMAGE NAME:preciseservercloudimgamd64disk1img-1486095746X |
+----------------------------------------------------------------------------------------------------------+
|IMAGE SUMMARY: |
|+---------+------+------------+---------+----------+--------+--------------+---------+ |
||VIRT TYPE|PUBLIC| OWNER ID |KERNEL ID|RAMDISK ID|PLATFORM|ROOT DEV TYPE | STATE | |
|+---------+------+------------+---------+----------+--------+--------------+---------+ |
|| hvm | True |000772500664| None | None | LINUX |instance-store|available| |
|+---------+------+------------+---------+----------+--------+--------------+---------+ |
| |
|IMAGE MANIFEST PATH: |
|+--------------------------------------------------------------------------------------------------------+|
|| IMAGE LOCATION: ||
|+--------------------------------------------------------------------------------------------------------+|
|| testrunner-precise-server-cloudimg-amd64-disk1img/precise-server-cloudimg-amd64-disk1.img.manifest.xml ||
|+--------------------------------------------------------------------------------------------------------+|
| |
|IMAGE BLOCK DEVICE MAPPING: N/A |
| |
|IMAGE TAGS: |
|+-----------------------------------+-----------------------------------------------------------------+ |
||TAG NAME |TAG VALUE | |
|+-----------------------------------+-----------------------------------------------------------------+ |
||URL |http://images.walrus.cloud.qa1.eucalyptus-systems.com:8773 | |
|| |/precise-server-cloudimg-amd64-disk1.img | |
|+-----------------------------------+-----------------------------------------------------------------+ |
||source |http://images.walrus.cloud.qa1.eucalyptus-systems.com:8773 | |
|| |/precise-server-cloudimg-amd64-disk1.img | |
|+-----------------------------------+-----------------------------------------------------------------+ |
||eutester-created | | |
|+-----------------------------------+-----------------------------------------------------------------+ |
||Nephoria Test Image: Fri Feb 3 | | |
||04:22:27 2017 | | |
|+-----------------------------------+-----------------------------------------------------------------+ |
||size |(3, '/disk1/storage/precise-server-cloudimg-amd64-disk1_img | |
|| |/precise-server-cloudimg-amd64-disk1.img') | |
|+-----------------------------------+-----------------------------------------------------------------+ |
| |
+----------------------------------------------------------------------------------------------------------+
Create a user, key, and security group. Run an instance with that user, connect and run commands on that instance.
# Create the testcontroller obj...
In [81]: from nephoria.testcontroller import TestController
In [82]: tc = TestController('10.111.1.72', password='foobar', log_level='DEBUG')
# Create a user (if this user already exists it will just fetch and return that user, else create a new euca account/user)
In [83]: user = tc.create_user_using_cloudadmin('testrunner', 'admin')
# Fetch an image to run...
In [84]: emi = user.ec2.get_emi(root_device_type='instance-store', not_platform='windows')
# Create a key. This creates the keypair artifact in the cloud as well as saves the localcert locally with a .pem extension.
In [90]: key = user.ec2.create_keypair_and_localcert('mytestkey')
In [92]: ll | grep mytestkey
-rw-------+ 1 joeuser staff 1757 Feb 9 17:55 mytestkey.pem
# Create a security group and authorize the group for ssh and icmp...
In [93]: securitygroup = user.ec2.add_group('newgroup')
In [95]: user.ec2.authorize_group(securitygroup, protocol='tcp', port=22)
Out[95]: True
In [96]: user.ec2.authorize_group(securitygroup, protocol='icmp', port=-1)
Out[96]: True
Now run an instance. The run_image method verifies certain cloud artifacts are in place, show information about the state of the ec2 service, existing VMs, etc. The method will also attempt to provide a root connection to the VM when it is available if the 'autoconnect' flag is provided. By default this is set to 'True'.
ins = user.ec2.run_image(image=emi, keypair=key, zone='one', group=securitygroup, vmtype='m1.small', min=1)[0]
# The instance obj is an extension of the Adminapi Machine class plus an ec2 interface and utils.
# Some example utilities...
# Run a command over ssh
In [100]: ins.sys('free')
[2017-02-09 18:03:49,926][DEBUG][i-6a8d83ae]: [root@10.111.75.57]# free
[2017-02-09 18:03:50,101][DEBUG][i-6a8d83ae]:
total used free shared buffers cached
Mem: 245316 184800 60516 0 20140 123972
-/+ buffers/cache: 40688 204628
Swap: 0 0 0
#Transfer a file to/from...
ins.ssh.sftp_get()
ins.ssh.sftp_put()
#work with a remote file obj on the instance as a local file obj
In [102]: rem_file = ins.ssh.sftp.open('newfile', 'w')
In [103]: rem_file.write('woot')
In [104]: rem_file.close()
In [105]: ins.sys('ls; cat newfile')
[2017-02-09 18:06:01,658][DEBUG][i-6a8d83ae]: [root@10.111.75.57]# ls; cat newfile
[2017-02-09 18:06:01,861][DEBUG][i-6a8d83ae]:
newfile
woot
[2017-02-09 18:06:01,861][DEBUG][i-6a8d83ae]: done with exec
Out[105]: ['newfile', 'woot']
# fetch and/or show network stats, interface info...
In [106]: ins.show_network_interfaces_delta()
[2017-02-09 18:06:19,582][INFO][i-6a8d83ae]: Time Elapsed Since Last Update: None
+-----+----+----+-----------+----+----+----+-------+----+--------+----+----+----+----+----+------+----+
|IFACE|RXFR|RXMU|RXBYTES |RXFI|RXER|TXDR|TXBYTES|TXCO|RXPACKET|TXCO|RXDR|RXCO|TXFI|TXER|TXPACK|TXCA|
+-----+----+----+-----------+----+----+----+-------+----+--------+----+----+----+----+----+------+----+
|lo |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 |0 |
+-----+----+----+-----------+----+----+----+-------+----+--------+----+----+----+----+----+------+----+
|eth0 |0 |0 |14531306169|0 |0 |0 |1186810|0 |14745371|0 |0 |0 |0 |0 |15980 |0 |
+-----+----+----+-----------+----+----+----+-------+----+--------+----+----+----+----+----+------+----+
#Work with block devices..
In [107]: ins.show_block_device_mapping()
[2017-02-09 18:06:51,119][INFO][i-6a8d83ae]:
+------------------------------------------------------------------------+
|BLOCK DEVICE MAP |
+------------------------------------------------------------------------+
|+------+---------+-------+------+----+---------+------+---------+------+|
||DEVICE|VOLUME_ID|SNAP_ID|D.O.T.|SIZE|EPHEMERAL|NO DEV|ATTACH TM|STATUS||
|+------+---------+-------+------+----+---------+------+---------+------+|
|+------+---------+-------+------+----+---------+------+---------+------+|
+------------------------------------------------------------------------+