Continuous Delivery for Microservices on EC2 with
Learn how to deploy a webapp to AWS EC2 using mu. If you want to get into microservices, but aren't ready to take the container plunge, you can deploy applications/services directly to AWS EC2 instances. Using mu's EC2 mode, you can leverage the benefits of microservices without containerization, and speed up development feedback loops.
The web app in this article is the smallest possible Flask application. You can adapt this example to deploy other applications that include their own web servers, like Django, NodeJS with Express, Ruby with puma/thin/passenger, Jetty or even Spring Boot.
Stelligent mu is a great open source tool to deploy Microservices to t]he AWS cloud. It has three modes of deployment:
* AWS ECS - (Elastic Container Service, which runs Docker containers on EC2).</li> * AWS Fargate - (A simpler “serverless” version of ECS)</li> * AWS EC2 instances without Docker containers. This is the mode we're covering in this article.</li>
In this blog post, we're going to:
* Fork this example repository on Github.com
* Change the mu.yml to point to your fork
* Create the mu pipeline
* Walk through the resources
mu creates for you, and
* Explain the role each file plays in deploying the app.
- Amazon Web Services
- AWS Cloudformation
- AWS CodePipeline
- AWS CodeBuild
- AWS CodeDeploy
- AWS Elastic LoadBalancers, VPC, Subnets
- Upstart, which manages services on Amazon Linux & other CentOS 6.x-based systems.
- An AWS Account
- a Github Account
- a Github Token - create one here
- AWS Access keys set up in $HOME/.aws
Make your own repository
Fork this repository in github.
Clone your forked repository to your workstation:
git clone firstname.lastname@example.org:<your-github-name>/mu-minimal-ec2.git
Go to the newly cloned directory
repo:line 17 of
mu.yml, to match your new repository's URL.
Commit and push your changes
git commit -a -m "modify the repo: attribute in mu.yml" git push
Create the mu pipeline
mu pipeline up -t <your-github-token>
Walkthrough of the mu pipeline
mu pipeline up command creates the CloudFormation stacks for the CodePipeline, CodeBuild and CodeDeploy services.
Navigate to the CloudFormation console, and watch the pipeline stacks form. They'll look something like this:
Once the pipeline stacks are created, open the CodePipeline console. You should have one pipeline listed. Click on it.
The mu pipeline looks like this:
- Source – the CodeBuild service retrieves a copy of your source code from GitHub or AWS CodeCommit.
- Build - your software is built, tested, packaged according to instructions in your repo's buildspec.yml file. This uses AWS CodeBuild.
- Deploy to Acceptance - a new environment called "acceptance" is created (or updated) with your software. This uses AWS CodeDeploy.
- Deploy to Production - after the one-and-only manual step, which is to offer Red and Green buttons to stop the pipeline, or deploy the software that is in Acceptance to the last environment, which is Production. This also uses AWS CodeDeploy.
Once the pipeline stacks are complete, you can watch the CodePipeline execute. The CodePipeline/CodeDeploy/CodeBuild services start building your software within a few moments.
When your pipeline reaches the Deploy stage, the CodeDeploy service creates Infrastructure stacks for each Environment, and will update them with each subsequent build. This is another core concept in DevOps known as infrastructure-as-code, and is fully implemented by
The additional environment stacks look like this:
The infrastructure created by these additional stacks include:
- networking: VPC, subnets, security groups
- compute: EC2 instances (alternatively, ECS and Fargate are also supported in
- storage: S3 buckets supporting the build artifacts
- resiliency: AutoScaling groups and the Elastic Load Balancer definitions come with
This means that if you want to change your infrastructure (AutoScaling parameters, your EC2 or RDS instance sizes, etc), you can implement those changes the same way you do your code changes: Change a file, run
git commit, and
Walkthrough of the files
1 namespace: min-ec2 2 environments: 3 - name: acceptance 4 provider: ec2 5 - name: production 6 provider: ec2 7 8 service: 9 name: min-ec2 10 port: 8080 11 healthEndpoint: / 12 pathPatterns: 13 - /* 14 pipeline: 15 source: 16 provider: GitHub 17 repo: stelligent/mu-minimal-ec2
environments: section from lines 2 to 6 (above), are a simple list of the environment
names you wish to have (acceptance, staging, sandbox, demo, production, whatever). The
provider value can have one of three different values (
To learn more about other configuring
mu environments, refer to the wiki page for Mu Environments.
services: section is from lines 8 to 17 (above), and specifies just the bare necessities. We're going to accept incoming connections on port 80, route them to our application on port 8080, checking the / endpoint for
200 OK responses, and use the code in the
repo specified on line 17.
To learn more about configuring
mu services, refer to the wiki page for Mu Services
1 version: 0.2 2 3 phases: 4 build: 5 commands: 6 - pip install Flask 7 8 artifacts: 9 files: 10 - "**/*"
mu uses AWS CodePipeline, CodeBuild, and CodeDeploy to turn your github repo into a deployed service. The buildspec.yml contains instructions (lines 3-6), on how to install prerequisites, how to invoke the build, do any postprocessing (minification, compression, archiving, etc).
Finally, you specify what files are to be deployed via the
artifacts/files: section from line 8 to 10.
You can (and should) also use buildspec.yml to run unit tests. Make sure all your commands return non-zero if there are any failures.
1 from flask import Flask 2 app = Flask(__name__) 3 4 @app.route("/") 5 def hello(): 6 return "Mu minimal ec2 example, v2!\n"
In this example repo, we have included the smallest possible Flask application possible, predictably named hello.py. It listens on a port number, and responds to incoming
GET / requests with a hardcoded string.
So, now we just needs to tell the system how to install it, start it, and stop this service.
1 version: 0.0 2 os: linux 3 files: 4 - source: ./hello.py 5 destination: /home/ec2-home/hello/ 6 - source: ./etc/init/mu-minimal-ec2.conf 7 destination: /etc/init/ 8 9 hooks: 10 BeforeInstall: 11 - location: codedeploy/remove_previous_install.sh 12 timeout: 30 13 ApplicationStop: 14 - location: codedeploy/stop_server.sh 15 timeout: 30 16 ApplicationStart: 17 - location: codedeploy/start_server.sh 18 timeout: 30
appspec.yml file contains the commands that install, start and stop the service.
filessection tells the system which files to move, and where they should reside.
- There are
hookssection, with hooks called
ApplicationStop(line 11), and
ApplicationStart(line 14). Each hook has a script location and a timeout in seconds. These scripts reside in a
codedeploydirectory, primarily to keep the root directory a little cleaner.
codedeploy/remove_previous_install.shremoves /home/ec2-user/hello/hello.py and /etc/init/mu-minimal-ec2.conf.
1 #!/bin/bash -xe 2 /bin/rm -rf /home/ec2-home/hello* /etc/init/mu-minimal-ec2.conf 3 exit 0
codedeploy/start_server.shtells upstart to start the service.
1 #!/bin/bash -x 2 set +e 3 stop mu-minimal-ec2 4 start mu-minimal-ec2 5 exit 0
upstartto stop the service.
1 #!/bin/bash -xe 2 stop mu-minimal-ec2 3 exit 0
Use your new application
mu env show acceptance to display key information about your app. One of the attributes is the ELB endpoint.
This repository demonstrates using
mu to deploy software to EC2. You should also know that this is just one of several deployment models that
mu can also dockerize your application and deploy it to AWS Elastic Container Service (Amazon ECS) and AWS Fargate. For the latest developments in
mu, you can watch the mu repository.