Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates for lint checks #137

Merged
merged 1 commit into from
Jun 13, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
227 changes: 151 additions & 76 deletions CONTRIBUTING.md

Large diffs are not rendered by default.

129 changes: 89 additions & 40 deletions DEPLOYMENT_OVERVIEW.md
Original file line number Diff line number Diff line change
@@ -1,61 +1,92 @@
# Deployment Overview
A Nubis Account Deployment consists of a number of standard services and security integrations. This document provides and overview of the account and services provided. Each service is self-contained and links are provided to each services' documentation which details that specific service.


# Deployment Overview

A Nubis Account Deployment consists of a number of standard services and
security integrations. This document provides and overview of the account and
services provided. Each service is self-contained and links are provided to each
services' documentation which details that specific service.

## Nubis Account Diagram

![Nubis Account Diagram](media/Nubis_Account_Diagram.png "Nubis Account Diagram")

NOTE: Details for the deployment including; naming conventions, relationships, permissions, etcetera, can be found in the [Terraform template](https://github.com/nubisproject/nubis-deploy/blob/master/main.tf) used for deployment.
NOTE: Details for the deployment including; naming conventions, relationships,
permissions, etcetera, can be found in the [Terraform template](https://github.com/nubisproject/nubis-deploy/blob/master/main.tf)
used for deployment.

### Services Provided

This is a list of all of the services available in a Nubis Account.

**TODO**: Add missing documentation links

- [VPC](https://github.com/nubisproject/nubis-deploy/blob/master/README.md#vpc-deployment)
- [Consul](https://github.com/nubisproject/nubis-consul/blob/master/README.md#consul-deployment)
- [Jumphost](https://github.com/nubisproject/nubis-jumphost/blob/master/README.md#jumphost-deployment)
- [Fluent](https://github.com/nubisproject/nubis-fluent-collector/blob/master/README.md#fluent-deployment)
- [Opsec / CloudTrail](https://github.com/nubisproject/nubis-deploy/blob/master/README.md#opsec-deployment)
- [CI](https://github.com/nubisproject/nubis-ci/blob/master/README.md#ci-deployment)
- [VPN](https://github.com/nubisproject/nubis-deploy/blob/master/README.md#vpc-deployment)
- User Management
- [NAT / Proxy](https://github.com/nubisproject/nubis-nat/blob/master/README.md#nat-deployment)
- [Prometheus](https://github.com/nubisproject/nubis-prometheus/blob/master/README.md#prometheus-deployment)
- [ELK](https://github.com/nubisproject/nubis-fluent-collector/blob/master/README.md#deployment-notes)

It is important to note that not all services are deployed in every account. To determine which services are deployed in a specific account you will need to consult the deployment configuration file for that account. For example, you can find the configuration files for the Nubis' Teams accounts in the [nubis-accounts-nubis](https://github.com/nubisproject/nubis-accounts-nubis) repository.

Within each configuration file are a set of feature flags, these flags are used to enable or disable specific services and are discussed [below](#feature-flags).
* [VPC](https://github.com/nubisproject/nubis-deploy/blob/master/README.md#vpc-deployment)
* [Consul](https://github.com/nubisproject/nubis-consul/blob/master/README.md#consul-deployment)
* [Jumphost](https://github.com/nubisproject/nubis-jumphost/blob/master/README.md#jumphost-deployment)
* [Fluent](https://github.com/nubisproject/nubis-fluent-collector/blob/master/README.md#fluent-deployment)
* [Opsec / CloudTrail](https://github.com/nubisproject/nubis-deploy/blob/master/README.md#opsec-deployment)
* [CI](https://github.com/nubisproject/nubis-ci/blob/master/README.md#ci-deployment)
* [VPN](https://github.com/nubisproject/nubis-deploy/blob/master/README.md#vpc-deployment)
* User Management
* [NAT / Proxy](https://github.com/nubisproject/nubis-nat/blob/master/README.md#nat-deployment)
* [Prometheus](https://github.com/nubisproject/nubis-prometheus/blob/master/README.md#prometheus-deployment)
* [ELK](https://github.com/nubisproject/nubis-fluent-collector/blob/master/README.md#deployment-notes)

It is important to note that not all services are deployed in every account. To
determine which services are deployed in a specific account you will need to
consult the deployment configuration file for that account. For example, you can
find the configuration files for the Nubis' Teams accounts in the
[nubis-accounts-nubis](https://github.com/nubisproject/nubis-accounts-nubis) repository.

Within each configuration file are a set of feature flags, these flags are used
to enable or disable specific services and are discussed [below](#feature-flags).

### Decryption Keys
You will need your GPG key added to the authorized configuration to view these files. [Git-crypt](https://github.com/AGWA/git-crypt) is used to manage encrypting the files. You will need to contact the team responsible for the deployment repository to gain decryption abilities.

You will need your GPG key added to the authorized configuration to view these
files. [Git-crypt](https://github.com/AGWA/git-crypt) is used to manage encrypting
the files. You will need to contact the team responsible for the deployment
repository to gain decryption abilities.

To determine which team to contact you will need to:
- Log into ServiceNow (The Hub)
- You will need the 'Amazon Web Services (AWS)' module enabled (Yellow Arrow)
- Select 'AWS Assets' (Red Circle)
- Locate the account by name or number (Purple Arrows)
- Locate the 'Account Email Address' (Sea-Green Square)
- Send an email to the address requesting access

**NOTE:** Only Nubis project accounts contain 'nubis' in the name. Application accounts are named after the deployed application.
* Log into ServiceNow (The Hub)
* You will need the 'Amazon Web Services (AWS)' module enabled (Yellow Arrow)
* Select 'AWS Assets' (Red Circle)
* Locate the account by name or number (Purple Arrows)
* Locate the 'Account Email Address' (Sea-Green Square)
* Send an email to the address requesting access

**NOTE:** Only Nubis project accounts contain 'nubis' in the name. Application
accounts are named after the deployed application.

![Service Now Screenshot](media/Service_Now_Screenshot.png "Service Now Screenshot")

### Security Integrations
There are a number of security integrations deployed into a Nubis Account. These are not available via feature flags and are always deployed in an account. Note that specific services contain additional security integrations which are detailed with the documentation for the service.

There are a number of security integrations deployed into a Nubis Account. These
are not available via feature flags and are always deployed in an account. Note
that specific services contain additional security integrations which are
detailed with the documentation for the service.

**TODO**: List security integrations
- SSH security group
- MIG
- NSM
- IP Blocklist
- HTTP(S) Proxy
- Cloud Trail
-

* SSH security group
* MIG
* NSM
* IP Blocklist
* HTTP(S) Proxy
* Cloud Trail
* ?

### Feature Flags
Within the account deployment variables file are a number of feature flags. These flags are used to select which services to deploy into the account. For a complete list of services depoyed into a particular account you will need to consult that accounts variables file. Here is an example of some of the feature flags available:

Within the account deployment variables file are a number of feature flags.
These flags are used to select which services to deploy into the account. For a
complete list of services depoyed into a particular account you will need to
consult that accounts variables file. Here is an example of some of the feature
flags available:

```bash
features.consul = 1
Expand All @@ -69,14 +100,32 @@ features.user_management_consul = 0
```

## Deployment Workflow
All deployment methods use Terraform as the descriptive language. The process varies somewhat depending on weather you are deploying a Nubis account or an Application in a Nubis account.

All deployment methods use Terraform as the descriptive language. The process
varies somewhat depending on weather you are deploying a Nubis account or an
Application in a Nubis account.

### Account Deployment Workflow
Account deployments are quite simple in practice. To deploy an account you need access to the encrypted variables file discussed above. You also need to have admin (*.*) privileges in AWS. The actual deployment is accomplished with Terraform and is described in greater detail in the [nubis-deployment repository](https://github.com/nubisproject/nubis-deploy/blob/master/README.md).

Account deployments are quite simple in practice. To deploy an account you need
access to the encrypted variables file discussed above. You also need to have
admin (*.*) privileges in AWS. The actual deployment is accomplished with
Terraform and is described in greater detail in the
[nubis-deployment repository](https://github.com/nubisproject/nubis-deploy/blob/master/README.md).

### Application Deployment Workflow
Application deployment is a bit more complex. If you are working in a Sandbox account that you will likely be using a manual process utilizing Terraform. When working in a production account the deployment is automated by using Jenkins as the continuous integration (CI) system

The CI system monitors the application's deployment repository hosted in git, typically GitHub. When a change lands in the repository, CI triggers AMI builds and (if successful) deploys the new image into the Stage environment (VPC). Deployment to production is typically triggered manually, through the CI system. The Prod deployment does not build an AMI, instead it uses the latest successfully built AMI from the Stage environment. This helps to ensure that only working AMIs are deployed into production.
Application deployment is a bit more complex. If you are working in a Sandbox
account that you will likely be using a manual process utilizing Terraform. When
working in a production account the deployment is automated by using Jenkins as
the continuous integration (CI) system

The CI system monitors the application's deployment repository hosted in git,
typically GitHub. When a change lands in the repository, CI triggers AMI builds
and (if successful) deploys the new image into the Stage environment (VPC).
Deployment to production is typically triggered manually, through the CI system.
The Prod deployment does not build an AMI, instead it uses the latest
successfully built AMI from the Stage environment. This helps to ensure that
only working AMIs are deployed into production.

![Application Deployment Workflow](media/Application_Deployment_Workflow.png "Application Deployment Workflow")
![Application Deployment Workflow](media/Application_Deployment_Workflow.png "Flow")
1 change: 1 addition & 0 deletions FEATURE_REQUESTS.md
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@

# Nubis - Feature Requests
81 changes: 58 additions & 23 deletions GETTING_STARTED.md
Original file line number Diff line number Diff line change
@@ -1,43 +1,78 @@
## Getting started with the Nubis Project
Welcome to the Nubis Project. We hope you will find that it meets your requirements and is easy to use. In this document I will introduce you to the Nubis Project and give you a number of links to other documents that will help you along.


The Nubis Project is in essence a framework for deploying applications to the cloud. At this time we support only Amazon Web Services (AWS). For an overview of our design principles I recommend you read our [manifesto](https://github.com/Nubisproject/nubis-docs/blob/master/MANIFESTO.md).
# Getting started with the Nubis Project

### Familiarize yourself with the Nubis Project
Now, to get you up to speed with everything you will need to know to use the Nubis Project, I will provide for you a reading list. Not to worry, while this list looks long, most of the documents are quite short.
* [Nubis Overview](https://github.com/Nubisproject/nubis-docs/blob/master/SYSTEM_OVERVIEW.md) will give you an outline of the pieces of the project.
* [Git & GitHub](https://github.com/Nubisproject/nubis-docs/blob/master/GIT_GITHUB.md) provides some advice specific to Nubis.
* [CloudFormation](https://github.com/Nubisproject/nubis-docs/blob/master/CLOUDFORMATION.md) walks through some recomendations on structure and workflow.
* [Prerequisites](https://github.com/Nubisproject/nubis-docs/blob/master/PREREQUISITES.md) will get you set up with all the necessary tools.
* [Project Onbording](https://github.com/Nubisproject/nubis-docs/blob/master/PROJECT_ONBOARDING.md) will guide you through onboarding your first application.
Welcome to the Nubis Project. We hope you will find that it meets your
requirements and is easy to use. In this document I will introduce you to the
Nubis Project and give you a number of links to other documents that will help
you along.

### Deployment
Now that you are familiar with the project and the process, it is time to get coding. The first step is to assemble your deployment repository. Then it will be time to deploy into the sandbox.
The Nubis Project is in essence a framework for deploying applications to the
cloud. At this time we support only Amazon Web Services (AWS). For an overview
of our design principles I recommend you read our [manifesto](https://github.com/Nubisproject/nubis-docs/blob/master/MANIFESTO.md).

As we have seen in various examples throughout these documents, you will need to create a deployment repository. Take a look at the [System Overview](link) document for details.
## Familiarize yourself with the Nubis Project

Once your repository is all set up the next step it to deploy into the sandbox. You can deploy following the procedures outlined in the [Project Onbording](https://github.com/Nubisproject/nubis-docs/blob/master/PROJECT_ONBOARDING.md#Application Build Out) doc. Some example commands can be found in our trusty [nubis-mediawiki](https://github.com/Nubisproject/nubis-mediawiki/blob/master/nubis/cloudformation/README.md) repository.
Now, to get you up to speed with everything you will need to know to use the
Nubis Project, I will provide for you a reading list. Not to worry, while this
list looks long, most of the documents are quite short.

* [Nubis Overview](https://github.com/Nubisproject/nubis-docs/blob/master/SYSTEM_OVERVIEW.md)
will give you an outline of the pieces of the project.
* [Git & GitHub](https://github.com/Nubisproject/nubis-docs/blob/master/GIT_GITHUB.md)
provides some advice specific to Nubis.
* [CloudFormation](https://github.com/Nubisproject/nubis-docs/blob/master/CLOUDFORMATION.md)
walks through some recomendations on structure and workflow.
* [Prerequisites](https://github.com/Nubisproject/nubis-docs/blob/master/PREREQUISITES.md)
will get you set up with all the necessary tools.
* [Project Onbording](https://github.com/Nubisproject/nubis-docs/blob/master/PROJECT_ONBOARDING.md)
will guide you through onboarding your first application.

### Bugs, Contributions and more
We are super excited to have you here! If you have stumbled into an issue there are several ways to address it.
## Deployment

First, you can fix the issue yourself and file a pull request. You will find a guide in our [Contributing Doc](https://github.com/Nubisproject/nubis-docs/blob/master/CONTRIBUTING.md).
Now that you are familiar with the project and the process, it is time to get
coding. The first step is to assemble your deployment repository. Then it will
be time to deploy into the sandbox.

Next, you can file an issue. Simply navigate to the Nubis Project space on Github [here](https://github.com/Nubisproject), select the appropriate repository and click on the issues link. For example, to file an issue against nubis-stacks you would go [here](https://github.com/Nubisproject/nubis-stacks/issues)
As we have seen in various examples throughout these documents, you will need to
create a deployment repository. Take a look at the [System Overview](link)
document for details.

Finally if you are looking for a new feature to be supported, simply follow the [Feature Requests](https://github.com/Nubisproject/nubis-docs/blob/master/FEATURE_REQUESTS.md) guide.
Once your repository is all set up the next step it to deploy into the sandbox.
You can deploy following the procedures outlined in the [Project Onbording](https://github.com/Nubisproject/nubis-docs/blob/master/PROJECT_ONBOARDING.md#application-build-out)
doc. Some example commands can be found in our trusty [nubis-mediawiki](https://github.com/Nubisproject/nubis-mediawiki/blob/master/nubis/cloudformation/README.md)
repository.

## Bugs, Contributions and more

We are super excited to have you here! If you have stumbled into an issue there
are several ways to address it.

First, you can fix the issue yourself and file a pull request. You will find a
guide in our [Contributing Doc](https://github.com/Nubisproject/nubis-docs/blob/master/CONTRIBUTING.md).

Next, you can file an issue. Simply navigate to the Nubis Project space on
Github [here](https://github.com/Nubisproject), select the appropriate
repository and click on the issues link. For example, to file an issue against
nubis-stacks you would go [here](https://github.com/Nubisproject/nubis-stacks/issues)

Finally if you are looking for a new feature to be supported, simply follow the
[Feature Requests](https://github.com/Nubisproject/nubis-docs/blob/master/FEATURE_REQUESTS.md)
guide.

---

## TODO

Document these things

* set up git repo
* add nubis directory
* add nubis directory
* link to structure doc
* discuss packer and nubis-builder
* discuss packers use of puppet
* discuss packer and nubis-builder
* discuss packers use of puppet
* describe cloudformation template system
* link to cloudformation layout doc?
* link to cloudformation layout doc?
* discuss what is and is not appropritae to place in the bin directory
* walk through deployment of application
* need to link to set up for Nubis doc (set up aws, git, github, etc...)