Cloudera Director - Amazon Web Services integration
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
provider
tests
.gitignore
CONTRIBUTING.md
Cloudera CCLA_25APR2018.pdf
Cloudera ICLA_25APR2018.pdf
LICENSE.txt
NOTICE
README.md
pom.xml

README.md

Amazon Web Services (AWS) plugin for Cloudera Altus Director

Introduction

This project defines an open source plugin implementing the Cloudera Altus Director Service Provider Interface (Director SPI) for Amazon Web Services (AWS).

The plugin implements a compute provider that allocates Amazon Elastic Compute Cloud (EC2) instances, and a database server provider that allocates Amazon Relational Database Service (RDS) instances.

Getting started

Because the AWS plugin ships with Cloudera Altus Director, these instructions are primarily geared towards plugin authors, who may be interested in using this plugin as an example or a base for their own plugins. They assume familiarity with git, github, maven, and other software development technologies; AWS concepts and terminology; and some familiarity with Cloudera Altus Director and the Director SPI.

Additional details can be found in the source code, in the AWS Plugin javadoc, and in the Cloudera Altus Director User Guide.

Prior to using the plugin, you will need to have AWS credentials.

Building the plugin

You can build the plugin with mvn clean install. Maven will need access to the Director SPI, either via the Cloudera Repository referenced in pom.xml, or via a local build.

Running the live tests

By default, the unit tests do not require communication with AWS. To run live tests that allocate and destroy EC2 and RDS instances, you must specify a system property (test.aws.live), and arrange for your AWS credentials to be available to the default credential provider chain via one of the methods described here.

The live test also requires some AWS network configurations to be specified in a property file, see the sample file. The default location to look for this property file should be overridden and is specified as a system property (test.aws.live.file).

Example command to run live tests : mvn -Dtest.aws.live=true -Dtest.aws.live.file=/path/to/livetest.properties clean install

Implementation details

The plugin follows the implementation pattern described in the Director SPI documentation.

The major classes are:

The configuration properties are implemented as nested enums of configuration property tokens. A representative example is EC2InstanceTemplate.EC2InstanceTemplateConfigurationPropertyToken.

The display properties are implemented as nested enums of display property tokens. A representative example is EC2Instance.EC2InstanceDisplayPropertyToken.

Validation is implemented via composite validators, each containing a default validator and a custom validator. A representative custom validator example is EC2InstanceTemplateConfigurationValidator.

Localization uses the default localization support provided by the Director SPI.

Logging uses the SLF4J logging API, as required by the Director SPI.

Additional filesystem configuration is specified in HOCON and loaded using the typesafe config library.

Important notice

Copyright © 2015 Cloudera, Inc. Licensed under the Apache License, Version 2.0.

Cloudera, the Cloudera logo, and any other product or service names or slogans contained in this document are trademarks of Cloudera and its suppliers or licensors, and may not be copied, imitated or used, in whole or in part, without the prior written permission of Cloudera or the applicable trademark holder.

Hadoop and the Hadoop elephant logo are trademarks of the Apache Software Foundation. Amazon Web Services, the "Powered by Amazon Web Services" logo, Amazon Elastic Compute Cloud, EC2, Amazon Relational Database Service, and RDS are trademarks of Amazon.com, Inc. or its affiliates in the United States and/or other countries. All other trademarks, registered trademarks, product names and company names or logos mentioned in this document are the property of their respective owners. Reference to any products, services, processes or other information, by trade name, trademark, manufacturer, supplier or otherwise does not constitute or imply endorsement, sponsorship or recommendation thereof by us.

Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Cloudera.

Cloudera may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Cloudera, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. For information about patents covering Cloudera products, see http://tiny.cloudera.com/patents.

The information in this document is subject to change without notice. Cloudera shall not be liable for any damages resulting from technical errors or omissions which may be present in this document, or from use of this document.