Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS Deployment improvements #522

Merged
merged 1 commit into from Apr 17, 2018

Conversation

RyanConnell
Copy link
Member

Multiple improvements and changes to the AWS deployment script.

Added the following flags:

  • worker-limit => Sets a limit on the amount of workers to deploy to (useful for benchmarking)
  • workers-only => Only create and deploy to workers.
  • master => The IP of a master to connect to instead of one tagged in the users AWS cluster.
  • deployment-credentials => Credentials to be used by the deployed container.

@RyanConnell RyanConnell added this to the Sprint 7 milestone Apr 17, 2018
@RyanConnell RyanConnell self-assigned this Apr 17, 2018
metavar='DATA_ABSTRACTION_LAYER', action='store',
help="Deploys containers on all running servers. Valid abstraction layers " \
"are DFS and S3")
parser.add_argument('-w', '--worker_limit', type=int, action='store', default=0)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is inconsistent with workers-only and deployment-credentials which use -, not _

aws/aws.py Outdated
help="Deployed workers will connect to this master instead of deploying " \
"a new one on AWS.")
parser.add_argument('--workers-only', action='store_true', default=False,
help="When this flag is set we will only create and deploy to workers" )

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is workers only needed when we have master-ip?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When creating new instances we don't need a master IP. So if you wanted to create 20 new instances without providing a master yet you would have to run aws.py -c 20 -m <random-unused-data>. I felt that it was neater to have it as aws.py -c 20 --workers-only, but i'm fine with removing it if you disagree.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah okay, makes sense.

@RyanConnell RyanConnell force-pushed the feature/aws-deployment-improvements branch from 4b5d47b to 9f782d1 Compare April 17, 2018 21:29
@RyanConnell RyanConnell merged commit 2d86f36 into develop Apr 17, 2018
@RyanConnell RyanConnell deleted the feature/aws-deployment-improvements branch April 17, 2018 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants