Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding some docs #5

Merged
merged 2 commits into from
May 7, 2015
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
24 changes: 24 additions & 0 deletions CLOUDFORMATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
## Recommended practices for Cloudformation
Cloudformation is a necessary evil when working with AWS. It uses JSON which has a number of staggering limitations. You will soon learn that it is overly rigid in its formatting. Additionally it lacks commenting, which, as you know, is a rather atrocious limitation. In an effort to limit your exposure to JSON we have adopted a nested stack model. Basically you will create a stack template which will use these ready made nested stacks. For an example check out this section of the [nubis-mediawiki template](https://github.com/Nubisproject/nubis-mediawiki/blob/master/nubis/cloudformation/main.json#L70).

### Nested Stacks
Nested stacks are in of themselves simply stacks that you include in a higher level, or container stack. We have created a number of stack templates to cover the most common use cases. You can take a look at them [here](https://github.com/Nubisproject/nubis-stacks). For each stack template we have included a README which includes usage code that you can copy into your stack template. Following the previous example from the nubis-mediawiki project you can see the EC2Stack nested stack template [here](https://github.com/Nubisproject/nubis-stacks/blob/master/ec2.template).

### Stack Outputs
We have created a small [function](https://github.com/Nubisproject/nubis-stacks/blob/master/lambda/LookupStackOutputs/LookupStackOutputs.README.md) that runs in [Lambda](http://aws.amazon.com/lambda/) (an AWS compute service) which makes the outputs of other stacks available for reference in your template. You will find us using this function in nearly every nested stack, sometimes multiple times. While you may not find a need for this in your template it is necessary knowledge for understanding the nested stack templates. For example, in the EC2Stack example above we are calling the function as [VpcInfo](https://github.com/Nubisproject/nubis-stacks/blob/master/ec2.template#L48) and using the VpcId output of the $region-$environment-vpc stack [here](https://github.com/Nubisproject/nubis-stacks/blob/master/ec2.template#L73).

### Parameterization
By utilizing stack outputs we are able to minimize the number of parameters (AWS name for input variables) we need. This simplifies deployments, especially when multiple developers are working on the same project. Back in the nubis-mediawiki project you will find the [parameters.json-dist file](https://github.com/Nubisproject/nubis-mediawiki/blob/master/nubis/cloudformation/parameters.json-dist) to contain only the absolute minimum[^minimum] number of parameters. These are the parameters that are necessary for every project that utilizes the Nubis project.

| Parameter | Description |
|---------------|-------------|
|ServiceName | Name of service from [here](https://inventory.mozilla.org/en-US/core/service/)
|Environment | Sandbox or Dev or Prod
|KeyName | Name of ssh key to install on ec2 instances
|TechnicalOwner | Email address or distribution list
|AmiId | ID output from nubis-builder

### Credentials
When deploying your stack using the [AWS cli tools](http://aws.amazon.com/cli/) you will be using an API keypair. You will need to take extra percaution to ensure that these secrets remain, well secret. This includes dressing up your .gitignore file, taking care with pastebins and the like.

[^minimum]: Well, not really since technically environment can be discovered.
42 changes: 42 additions & 0 deletions GETTING_STARTED.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
## Getting started with the Nubis Project
Welcome to the Nubis Project. We hope you will find that it meats your requirements and is easy to use. In this document I will introduce you to the Nubis Project and give you a number of links to other documents that will help you along.

The Nubis Project is in essence a framework for deploying applications to the cloud. At this time we support only Amazon Web Services (AWS). For an overview of our design principles I recommend you read out [manifesto](https://github.com/Nubisproject/nubis-docs/blob/master/MANIFESTO.md).

### Familiarize yourself with the Nubis Project
Now, to get you up to speed with everything you will need to know to use the Nubis Project, I will provide for you a reading list. Not to worry, while this list looks long most of the documents are quite short.
* [Nubis Overview](WRITE_ME) Will give you an outline of the pieces of the project.
* [Git & GitHub](https://github.com/Nubisproject/nubis-docs/blob/master/GIT_GITHUB.md) provides some advice specific to Nubis.
* [CloudFormation](https://github.com/Nubisproject/nubis-docs/blob/master/CLOUDFORMATION.md) walks through some recomendations on structure and workflow.
* [Project Onbording](https://github.com/Nubisproject/nubis-docs/blob/master/PROJECT_ONBOARDING.md) will guide you through onboarding your first application.

### Deployment
Now that you are familiar with the project and the process it is time to get coding. The first step is to assemble your deployment repository. Then it will be time to deploy into the sandbox.

As we have seen in various examples through these documents, you will need to create a deployment repository. Take a look at the [System Overview](link) document for details.

Once your repository is all set up the next step it to deploy into the sandbox. You can deploy bu following the procedures outlined in the [Project Onbording](https://github.com/Nubisproject/nubis-docs/blob/master/PROJECT_ONBOARDING.md#Application Build Out) doc. Some example commands can be found in our trusty [nubis-mediawiki](https://github.com/Nubisproject/nubis-mediawiki/blob/master/nubis/cloudformation/README.md) repository.


### Bugs, Contributions and more
We are super excited to have you hear. If you have stumbled into an issue there are several ways to address it.

First, you can fix the issue yourself and file a pull request. You will find a guild in our [Contributing Doc]((https://github.com/Nubisproject/nubis-docs/blob/master/CONTRIBUTING.md).

Next, you can file an issue. Simply navigate to the Nubis Project space on Github [here](https://github.com/Nubisproject), select the appropriate repository and click on the issues link. For example to file an issue against nubis-stacks you would go [here](https://github.com/Nubisproject/nubis-stacks/issues)

Finally if you are looking for a new feature to be supported, simply follow the [Feature Requests](https://github.com/Nubisproject/nubis-docs/blob/master/FEATURE_REQUESTS.md) guide.

---
## TODO
Document these things
* set up git repo
* add nubis directory
* link to structure doc
* discuss packer and nubis-builder
* discuss packers use of puppet
* describe cloudformation template system
* link to cloudformation layout doc?
* discuss what is and is not appropritae to place in the bin directory
* walk through deployment of application
* need to link to set up for Nubis doc (set up aws, git, github, etc...)
28 changes: 28 additions & 0 deletions GIT_GITHUB.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
## Recommended Practices for Git & GitHub
This document will walk you through some best practices that we recommend for working with the Nubis project. I will not cover much about the basic operation of git or GitHub as there are a large number of tutorials online that cover these topics. Instead I will concentrate on the specifics that will help you to get the most out of the Nubis project, but most importantly will help you to avoid some pitfalls along the way.

### Deployment Repository
Lets start with what we will call the "Deployment Repository". This is a git repository, typically available on GitHub, that contains all of the pieces necessary to deploy your Application. This includes two things, your application code and a collection of Nubis files. It is important that you follow this layout as our automation tools expect to find things in specific locations. For an example, check out the example [nubis-mediawiki](https://github.com/Nubisproject/nubis-mediawiki) repository.

#### Application Code
Your application code can be contained within this repository or it can simply be included as a git submodule. The choice to embed your code directly in the repository simplifies your application by having everything in one location. On the other hand if you separate out your application code from your deployment repository you can have different people responsible for different aspects of your code. Additionally this allows your application code repository to remain deployment agnostic. The choice is yours, but I generally recommend you use the submodule method for clean separation of responsibility and technology.

#### Nubis Files
The nubis files are all contained in a single folder called, not surprisingly, nubis. Typically there will be three or more folders contained within the nubis folder; puppet, builder and cloudformation. You can learn more about this layout over in the [Nubis Overview](link) document.

### Branching
We recommend using topic (feature) branches while developing new features. This allows you to switch easily between different development work-flows without all that stashing nonsense. You can learn more about branching (and merging) [here](http://git-scm.com/book/en/v2/Git-Branching-Basic-Branching-and-Merging).

### Issues & Pull Requests
For anything related to the Nubis Project itself, you can either file an [issue](https://guides.github.com/features/issues/) for us or submit a patch by using GtiHubs [pull request](https://help.github.com/articles/using-pull-requests/) method. Either way we will be sure to work with you to solve your issue.

You might be interested in checking out [Hub](https://hub.github.com/), it makes working with GitHub from the command line a snap.

### Code Reviews
For everything relating to the Nubis Project itself, we require a code review before landing anything. We do not currently have a strict process for this, however one of the core maintainers (or module owner) must review code before it is merged to any production branch. We define a production branch as any branch that can affect any running systems. For example, branches that deploy the Sandbox are considered "Production" as they affect the productivity of people using this system.

We recommend that you adopt a code review process for all aspects of your application. This helps to reduce production downtime, helps to maintain cohesiveness and ensures code continues to follow your style guidelines.

### TODO
* Describe versioning
* Details on directory layout may be in another doc and should be linked here.
60 changes: 60 additions & 0 deletions PROJECT_ONBOARDING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
## Project Onboarding

This document is hear to guide us through the process of on-boarding your application in AWS using the [Nubis](https://github.com/Nubisproject) project.

### Checklist
The steps we will take during this process are:
1. You [gather information](Gather Information)
1. We [create your AWS account](Create Account) in the Sandbox
1. You [generate your AWS API keys](Generate API Keys)
1. Everyone [meets](Meeting Time) to discuss architectural requirements / design
1. You [build out](Application Build Out) your application in the Sandbox
1. You initiate the [promotion to Dev](Promote to Dev) process
1. You initiate the [promotion to Prod](Promote to Prod) process


### Gather Information
We need to know a few pieces of information about your application. This is used to track resources, for troubleshooting and for billing purposes. If you do not have all of this information, do not worry, we can help you figure it out. Once you have gathered all of this information you simply fill out [this form](link to Bugzilla form) to kick off the rest of the process.

> Link to Bugzilla form (or list here the instructions for them to send the information to us). I am suggesting we create a Bugzilla form with inputs specifically for this information. When the user fills out this form it will both initiate the on-boarding process as well as provide us with the information we need to get started. I suggest two additional Bugzilla forms later in this document.

The information we need from you is:
1. Name of your project (AKA the "Service Name") [found here] (https://inventory.mozilla.org/en-US/core/service/)
1. Email address of the ["Technical Owner"](link to email address requirements)
1. [Cost center](link to wherever we look up cost centers) (Do we need this?)
1. List of people who should attend the (Name?) Meeting
1. [ list more requirements here as we discover them ]


### Create Account
As soon as you submit the above information to us we will create your AWS account.
> Will we use the Technical Owner email address for this? If not provide details here about how we will provide the information to them.

Once we have done that we will notify you by...
> (how?).

### Generate API Keys
Once you are logged into your new account you will need to generate an API key pair by following the instructions [here](link to API key section of account creation document). You will use this key pair to deploy resources (such as EC2 instances) into the sandbox. You should keep in mind that these keys are [secret](https://github.com/Nubisproject/nubis-docs/blob/master/SECURITY.md) and should not be shared with anyone.

### Meeting Time
We will schedule a meeting with all the necessary folks so we can all sit down and determine how we can help you to succeed. In this meeting we will discuss; design requirements, architectural needs, best practices and so on. Not to worry, we have a presentation all set up and will try to make this as painless as possible for you and your team.

### Application Build Out
Now that we have a design it is time to build the resources necessary to support your application. To assist you with your application build out we have prepared a number of documents.

* First you will want to check out our [System Overview](https://github.com/Nubisproject/nubis-docs/blob/master/SYSTEM_OVERVIEW.md) document. It will give you a general sense of how all the pieces fit together.
* Then I recommend you take a peek at our [Git & GitHub](https://github.com/Nubisproject/nubis-docs/blob/master/GIT_GITHUB.md) doc. This will aid you in setting up your repository for deploying with the Nubis Project.
* Next you should peruse our [Nubis Builder](https://github.com/Nubisproject/nubis-builder/blob/master/README.md) document which covers building AMIs using [Packer](https://www.packer.io/) and [Puppet](https://puppetlabs.com/).
* Finally you should take a gander at out [CloudFormation](https://github.com/Nubisproject/nubis-docs/blob/master/CLOUDFORMATION.md) document which covers things like using nested stacks to simplify your CloudFormation templates.

### Promote to Dev
The next step on the road to getting your application into production is to initiate the process to get it deployed into Dev. This should be a super simple process as long as you followed the best practices mentioned above. If so, you simply fill out [this little form](link to another Bugzilla form for promotion process) and we will do the rest. We will be setting up a Continuous Integration (CI) system to deploy your project into Dev. This CI system will deploy your application using the exact same CloudFormation templates that you use to deploy in the Sandbox. The process will go something like [this](https://mana.mozilla.org/wiki/display/EA/Environment+promotion).

### Promote to Prod
Once your application is running in Dev it is time for you to do your User Acceptance Testing (UAT). Once you have completed your UAT and you are ready to promote your application into production, simply fill out this [form](link to Bugzilla form for production promotion). We will then set up the CI system to deploy your application into Production. We will also schedule the following meetings
> what meetings go here? make list CAB, what else?

During these meetings we will work with you to schedule the actual go-live event. This typically includes things like coordinating with the Mozilla Operations Center (MOC) and scheduling final content sync and DNS cut-over.

### Winning
That is all there is to it. If you have any feedback on this document, this process, or anything Nubis Project related please feel free to drop us a line. We are in #infra-aws on [irc](irc.mozilla.org) or you can shoot us an [email](mailto:infra-aws@mozilla.org). Happy clouding.