-
Notifications
You must be signed in to change notification settings - Fork 8
Feature/deployment #613
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/deployment #613
Changes from all commits
fb98b92
bb3606f
0162298
c4e9961
6e713be
3ef925f
667fee9
7c99107
702e41c
ac2cd83
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| 9.3.0 | ||
| 10.17.0 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| web: npm run deploy |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| { | ||
| "name": "HackMcGill HackerAPI", | ||
| "description": "API for registration, live-site", | ||
| "repository": "https://github.com/hackmcgill/hackerAPI", | ||
| "keywords": ["node", "express", "mongo", "react", "hackathon"], | ||
| "env": { | ||
| "FRONTEND_ADDRESS_DEPLOY": { | ||
| "description": "URL for application front end" | ||
| }, | ||
| "DB_ADDRESS_DEPLOY": { | ||
| "description": "Connection string for MongoDB" | ||
| }, | ||
| "DB_USER_DEPLOY": { | ||
| "description": "DB Username" | ||
| }, | ||
| "DB_PASS_DEPLOY": { | ||
| "description": "DB Password" | ||
| }, | ||
| "JWT_INVITE_SECRET": { | ||
| "generator": "secret" | ||
| }, | ||
| "JWT_RESET_PWD_SECRET": { | ||
| "generator": "secret" | ||
| }, | ||
| "JWT_CONFIRM_ACC_SECRET": { | ||
| "generator": "secret" | ||
| }, | ||
| "SENDGRID_API_KEY": { | ||
| "description": "Sendgrid API Key" | ||
| }, | ||
| "NO_REPLY_EMAIL": { | ||
| "description": "Noreply email address" | ||
| }, | ||
| "NO_REPLY_EMAIL": { | ||
| "description": "GCP Storage Bucket Name" | ||
| }, | ||
| "TYPE": { | ||
| "description": "GCP Account Type" | ||
| }, | ||
| "PROJECT_ID": { | ||
| "description": "GCP Project ID" | ||
| }, | ||
| "PRIVATE_KEY_ID": { | ||
| "description": "GCP Private Key ID" | ||
| }, | ||
| "PRIVATE_KEY": { | ||
| "description": "GCP Private Key" | ||
| }, | ||
| "CLIENT_EMAIL": { | ||
| "description": "GCP Client Email" | ||
| }, | ||
| "AUTH_URI": { | ||
| "description": "GCP Auth URI" | ||
| }, | ||
| "TOKEN_URI": { | ||
| "description": "GCP TOken URI" | ||
| }, | ||
| "AUTH_PROVIDER_X509_CERT_URL": { | ||
| "description": "GCP Provider Cert URL" | ||
| }, | ||
| "CLIENT_X509_CERT_URL": { | ||
| "description": "GCP Cert URL" | ||
| } | ||
| }, | ||
| "formation": { | ||
| "web": { | ||
| "quantity": 1, | ||
| "size": "free" | ||
| } | ||
| }, | ||
| "image": "heroku/nodejs", | ||
| "buildpacks": [ | ||
| { | ||
| "url": "https://github.com/heroku/heroku-buildpack-nodejs" | ||
| } | ||
| ] | ||
| } |
This file was deleted.
This file was deleted.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,71 +1,42 @@ | ||
| # How to deploy | ||
|
|
||
| The following document acts as a quickstart guide on how to get your machine set up for deployment to the Google Cloud Container Cluster we have set up for the project. | ||
| *** Please note that the very first step before performing any of the steps below is to get permission from** @Kyle Rubenok **to be added to the Google Cloud project. | ||
| ** The steps below are written with an Ubuntu OS in mind, however a Mac OS should also be able to perform the majority (if not all) of this given the UNIX environment.** | ||
|
|
||
| ## Installing Google Cloud SDK | ||
|
|
||
| We begin by installing the Google Cloud CLI so we can easily deploy the application directly from the command line. Head over to [this link](https://cloud.google.com/sdk/) and click on the "Install for [OS]" button. | ||
|
|
||
| ## Setting up Google Cloud SDK | ||
|
|
||
| Once you have the SDK installed, you can head over to your terminal and type in `gcloud init` to get started with the initial configuration of the SDK. This process is fairly intuitive and it takes you through a few steps to get your Google account linked to the CLI by logging into the web interface, as well as choosing a default project. **Note that it's a good idea to have your Google account already added to the project on Google Cloud by someone who has access before doing this, so that you can just select the hackboard project as your default instead of re-setting it later.** | ||
| In any case, we will be covering what commands you need to set our project as your default project. Type in the following lines in your terminal. | ||
|
|
||
| ```bash | ||
| > gcloud config set project hackboard6 | ||
| > gcloud config set compute/zone us-east1-b | ||
| ``` | ||
|
|
||
| ## Installing Kubernetes | ||
|
|
||
| Kubernetes is essentially Google's answer to efficient cloud clustering and scaling. We'll use both Kubernetes and Docker here. Kubernetes is essentially what we will use to manage our cluster of nodes that will run the application and allow us to scale it efficiently and easily. We get into the Docker details later. We will **not** be installing Kubernetes through the official installer, but rather use `gcloud` CLI to install it such that it automatically integrates with your credentials and save us some time setting it up. Run the following in your terminal. | ||
|
|
||
| ```bash | ||
| > gcloud components install kubectl | ||
| ``` | ||
|
|
||
| Hopefully you'll manage to walk through the simple setup quickly and have it up and running. The next step would be creating a cluster and setting it up with your application, but since we have already taken care of that, you'll want to run the below command to have your CLI connected to the cluster we have already created so that you can deploy to it. This command will attempt to retrieve the credentials for the 'hackboard' cluster we have set up and allow you to deploy to it. | ||
|
|
||
| ```bash | ||
| > gcloud container clusters get-credentials hackboard | ||
| ``` | ||
|
|
||
| ## Intalling Docker CE | ||
|
|
||
| We will be using the Community Edition of Docker for our application. Head over to [this link](https://www.docker.com/community-edition#download) and choose and specific installation for your own operating system; the setup is rather intuitive and skipped in this guide. Once you've got Docker installed, try running `docker —version` to make sure it's up and running. Note that docker has a 'daemon' service which should be running for the CLI to communicate with. I suggest following the steps in [this page](https://docs.docker.com/install/linux/linux-postinstall/#configure-docker-to-start-on-boot) to make sure the daemon is configured to start with your machine on start-up. Also, the previously mentioned note is mostly directed towards Linux users; the Windows and MacOS application (I believe) takes care of starting up the daemon for you once you see the green light in your indicators. | ||
|
|
||
| **NOTE**: **The following steps are optional and meant for you to understand how the deployment actually works. You will not have to (and it is not recommended) to run all these steps manually everytime, as there will be a bash script setup to take care of all this.** | ||
|
|
||
| ## Building the Docker Image | ||
|
|
||
| Every Docker container requires an image to run, which in turn requires a 'Dockerfile' as a set of instructions for creating it. Images can extend off of other images or be created from scratch. Due to the fact that the Docker community is very well established and has an 'almost' exhaustive list of base images on it already, we will be using an official Node JS image that provides the runtime for our application to run, and add to it as we wish. Below is the basic Dockerfile we have created so far: | ||
|
|
||
| ```bash | ||
| FROM node:carbon | ||
| ENV PORT 8080 | ||
| WORKDIR /usr/src/app | ||
| COPY package*.json ./ | ||
| ADD VERSION . | ||
| RUN npm install | ||
| COPY . . | ||
| EXPOSE 8080 | ||
| CMD [ "npm", "start" ] | ||
| This application can be most easily deployed via Heroku. A "deploy to heroku" button is available in the `README.md` file. Once in Heroku, several environment variables will need to be filled out including: | ||
|
|
||
| ```plain | ||
krubenok marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ##front-end server | ||
| FRONTEND_ADDRESS_DEPLOY= | ||
|
|
||
| #The info for the deployment database | ||
| DB_ADDRESS_DEPLOY= | ||
| DB_USER_DEPLOY= | ||
| DB_PASS_DEPLOY= | ||
|
|
||
| #Secret key for the cookies | ||
| COOKIE_SECRET= | ||
|
|
||
| #Secret key for the invite tokens | ||
| JWT_INVITE_SECRET= | ||
|
|
||
| #Reset password secret | ||
| JWT_RESET_PWD_SECRET= | ||
|
|
||
| #Secret key fo account confirmation token | ||
| JWT_CONFIRM_ACC_SECRET= | ||
|
|
||
| #mail server information | ||
| SENDGRID_API_KEY= | ||
| NO_REPLY_EMAIL= | ||
|
|
||
| #Google Cloud Storage information | ||
| BUCKET_NAME= | ||
| TYPE= | ||
| PROJECT_ID=-api | ||
| PRIVATE_KEY_ID= | ||
| PRIVATE_KEY= | ||
| CLIENT_EMAIL= | ||
| CLIENT_ID= | ||
| AUTH_URI= | ||
| TOKEN_URI= | ||
| AUTH_PROVIDER_X509_CERT_URL= | ||
| CLIENT_X509_CERT_URL= | ||
| ``` | ||
|
|
||
| *If you have any questions regarding what Docker is and the difference between an image and a container, or anything else, I suggest reading online about it in detail before getting your hands dirty.* | ||
| To explain what is happening here, we are extending off of the LTS (carbon) version of the official Node image available on [Docker Hub](https://hub.docker.com/). Next we create an environment variable called PORT for our application. Environment variables can be created like this, or all added in a separate file. Next we declare the path `/usr/src/app` to be working directory (i.e. starting directory) of our image. Any operations from here onwards in the Dockerfile are performed in the set working directory. We will also copy the `package.json` files and the `VERSION` file with each build to make it easier to figure out which build is live via the application itself. Next we run the `npm install` command to install all dependencies, copy everything else left to copy to the image, and mark port `8080` to be exposed. This is an arbitrary value for now and can be replaced with any other port desired. Finally, set `npm start` as the starting command of the application which corresponds to `node ./bin/www` in our `package.json` file; this pretty much fires up the backend server. | ||
|
|
||
| To be able to push the docker image into the Google Cloud Container Registry so that our clusters can access the images easily, we first need a defined naming/tagging scheme for the images. Below is the scheme we are using (which is the default and recommended way of naming images on Container Registry). The host name for Google Cloud Container Registry is `[gcr.io](http://gcr.io)` , our project-ID is `hackboard6` , image name is `hackboard` and the tag will correspond to the version of the current build `6.x.x` which we will also have it programatically increment and synchronize with every release on the master branch on GitHub. | ||
|
|
||
| [HOSTNAME]/[PROJECT-ID]/[IMAGE][:TAG|@DIGEST] #template | ||
| gcr.io/hackboard6/hackboard:latest #example | ||
|
|
||
| Every image has a `latest` version as well as all the other versions it was tagged as. The `latest` tag is automatically applied to (obviously) the latest build of the image, while the legacy builds are accessible via their specific version numbers. | ||
|
|
||
| First you want to make sure there are no changes pushed to the master branch by anyone else before you deploy your local version. So perform a `git pull origin master`. | ||
|
|
||
| We will first increment our `VERSION` file using a 3rd-party Docker image. | ||
|
|
||
| *TO BE CONTINUED WHEN I HAVE THE MOOD.* | ||
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,56 +1,59 @@ | ||
| { | ||
| "name": "hackboard", | ||
| "version": "0.0.0", | ||
| "private": true, | ||
| "scripts": { | ||
| "start": "DEBUG=hackboard:* NODE_ENV=development nodemon --ignore gcp_creds.json ./bin/www.js", | ||
| "build": "NODE_ENV=deployment node ./bin/www.js", | ||
| "debug": "DEBUG=hackboard:* NODE_ENV=deployment nodemon --ignore gcp_creds.json ./bin/www.js", | ||
| "test": "DEBUG=hackboard:* NODE_ENV=test mocha --reporter spec tests/**.js --exit", | ||
| "seed": "NODE_ENV=development node ./seed/index.js", | ||
| "docs": "apidoc -i ./routes -o ./docs/api/", | ||
| "format": "prettier --write '**/*.js'", | ||
| "lint": "eslint --fix '**/*.js'" | ||
| }, | ||
| "dependencies": { | ||
| "@google-cloud/logging-winston": "^0.11.1", | ||
| "@google-cloud/storage": "^3.5.0", | ||
| "@sendgrid/mail": "^6.4.0", | ||
| "bcrypt": "^3.0.7", | ||
| "cookie-parser": "~1.4.4", | ||
| "cookie-session": "^2.0.0-beta.3", | ||
| "cors": "^2.8.5", | ||
| "cryptiles": "^4.1.3", | ||
| "debug": "~4.1.1", | ||
| "dotenv": "^8.2.0", | ||
| "express": "~4.17.1", | ||
| "express-validator": "^6.3.0", | ||
| "express-winston": "^2.6.0", | ||
| "handlebars": "^4.5.3", | ||
| "jsonwebtoken": "^8.5.1", | ||
| "memory-cache": "^0.2.0", | ||
| "mongoose": "^5.7.14", | ||
| "multer": "^1.4.2", | ||
| "passport": "^0.4.0", | ||
| "passport-local": "^1.0.0", | ||
| "q": "^1.5.1", | ||
| "qrcode": "^1.4.4", | ||
| "winston": "^2.4.4" | ||
| }, | ||
| "devDependencies": { | ||
| "@types/express": "^4.17.2", | ||
| "@types/google-cloud__storage": "^1.7.2", | ||
| "@types/mongodb": "^3.3.12", | ||
| "@types/mongoose": "^5.5.32", | ||
| "@types/multer": "^1.3.10", | ||
| "apidoc": "^0.17.7", | ||
| "chai": "^4.2.0", | ||
| "chai-http": "^4.3.0", | ||
| "eslint": "6.7.2", | ||
| "eslint-config-prettier": "6.7.0", | ||
| "eslint-plugin-prettier": "3.1.1", | ||
| "mocha": "^6.2.2", | ||
| "nodemon": "^1.19.4", | ||
| "prettier": "1.19.1" | ||
| } | ||
| "name": "hackerAPI", | ||
| "version": "1.6.2", | ||
| "private": true, | ||
| "scripts": { | ||
| "start": "DEBUG=hackboard:* NODE_ENV=development nodemon --ignore gcp_creds.json ./bin/www.js", | ||
| "deploy": "NODE_ENV=deployment node ./bin/www.js", | ||
| "debug": "DEBUG=hackboard:* NODE_ENV=deployment nodemon --ignore gcp_creds.json ./bin/www.js", | ||
| "test": "DEBUG=hackboard:* NODE_ENV=test mocha --reporter spec tests/**.js --exit", | ||
| "seed": "NODE_ENV=development node ./seed/index.js", | ||
| "docs": "apidoc -i ./routes -o ./docs/api/", | ||
| "format": "prettier --write '**/*.js'", | ||
| "lint": "eslint --fix '**/*.js'" | ||
| }, | ||
| "engines": { | ||
| "node": "10.17.0" | ||
| }, | ||
| "dependencies": { | ||
| "@google-cloud/logging-winston": "^0.11.1", | ||
| "@google-cloud/storage": "^3.5.0", | ||
| "@sendgrid/mail": "^6.4.0", | ||
| "bcrypt": "^3.0.7", | ||
| "cookie-parser": "~1.4.4", | ||
| "cookie-session": "^2.0.0-beta.3", | ||
| "cors": "^2.8.5", | ||
| "cryptiles": "^4.1.3", | ||
| "debug": "~4.1.1", | ||
| "dotenv": "^8.2.0", | ||
| "express": "~4.17.1", | ||
| "express-validator": "^6.3.0", | ||
| "express-winston": "^2.6.0", | ||
| "handlebars": "^4.5.3", | ||
| "jsonwebtoken": "^8.5.1", | ||
| "memory-cache": "^0.2.0", | ||
| "mongoose": "^5.7.14", | ||
| "multer": "^1.4.2", | ||
| "passport": "^0.4.0", | ||
| "passport-local": "^1.0.0", | ||
| "q": "^1.5.1", | ||
| "qrcode": "^1.4.4", | ||
| "winston": "^2.4.4" | ||
| }, | ||
| "devDependencies": { | ||
| "@types/express": "^4.17.2", | ||
| "@types/google-cloud__storage": "^1.7.2", | ||
| "@types/mongodb": "^3.3.12", | ||
| "@types/mongoose": "^5.5.32", | ||
| "@types/multer": "^1.3.10", | ||
| "apidoc": "^0.17.7", | ||
| "chai": "^4.2.0", | ||
| "chai-http": "^4.3.0", | ||
| "eslint": "6.7.2", | ||
| "eslint-config-prettier": "6.7.0", | ||
| "eslint-plugin-prettier": "3.1.1", | ||
| "mocha": "^6.2.2", | ||
| "nodemon": "^1.19.4", | ||
| "prettier": "1.19.1" | ||
| } | ||
|
Comment on lines
+2
to
+58
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The VS Code format on save settings forced the 4 tab spacing to be applied to this file hence the large diff. |
||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you remove this for now? I think it's better to discuss w everyone first and we can always add it back easily