Homeday GitHub Stats - Backend
This is the backend part of the Homeday GitHub Stats app. Documentation covers all of the deployment requirements.
Unless you expect high usage, you should be covered by the free plan of each service.
Creating a GitHub OAuth APP
Go to GitHub -> Settings -> Developer Settings -> OAuth Apps. Create a new OAuth App (give it a recognizable name as people will see it when authenticating with GitHub). Homepage URL will be the url of your Auth0 account, which you will get when signing up with Auth0.
Creating a Auth0 App
After creating an account with Auth0, under "Connections -> Social", set up the GitHub authentication (here you will need Client ID and Client Secret from your GitHub App). Permissions you need are the following:
Under Applications, create a new "SINGLE PAGE APPLICATION" (this application will be used for the Authentication on the frontend side of your app). In the "Allowed Callback URLs" enter any urls your app might run on (including localhost). Copy-paste those into the "Allowed Web Origins" and "Allowed Logout URLs" fields as well. Under "Connections", make sure to disable the default Username-Password-Authentication and allow the newly created GitHub one.
Create another Application, this time of type "MACHINE TO MACHINE" (will be used to verify authentication on the backend). Enable Auth0 Management API for it (which was probably already created for you under "APIs").
You don't need anything except serverless installed locally on your machine:
npm install -g serverless
You will need tow S# buckets, one for staging and one for production. Make sure you enable "Static website hosting" for them. Also, under Permissions -> CORS configuration, we need to enable CORS:
<?xml version="1.0" encoding="UTF-8"?> <CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> <CORSRule> <AllowedOrigin>*</AllowedOrigin> <AllowedMethod>HEAD</AllowedMethod> <AllowedMethod>GET</AllowedMethod> <AllowedMethod>PUT</AllowedMethod> <AllowedMethod>POST</AllowedMethod> <AllowedMethod>DELETE</AllowedMethod> <ExposeHeader>ETag</ExposeHeader> <AllowedHeader>*</AllowedHeader> </CORSRule> </CORSConfiguration>
This policy allows CORS for all origins which was ok for the purposes of our app, you might wish to tweak it to allow only your own domains.
Lambdas are managed by the Serverless framework, so in theory, other than for debugging, you should not have totouch them.
The final piece of the puzzle is Travis. There are two parts to it,
.travis.variables.sh file in this repo where we set some variables based on the environment (there you will want to change your S3 bucket and app domain), and the configuration of the repository.
You will need the following list of environment variables defined in the repo settings of the Travis APP:
API_ENDPOINT: Public, fixed value of
AUTH0_BACKEND_CLIENT_ID: Public, Client ID of your Auth0 "MACHINE TO MACHINE" app
AUTH0_BACKEND_CLIENT_SECRET: Private, Client Secret of your Auth0 "MACHINE TO MACHINE" app
AUTH0_BACKEND_DOMAIN: Public, your Auth0 domain, will be your main Auth0 domain assigned to your account
AUTH0_CLIENT_ID: Public, Client ID of your Auth0 "SINGLE PAGE APPLICATION" app
AUTH0_DOMAIN: Public, probably the same as AUTH0_BACKEND_DOMAIN unless you used two separate Auth0 accounts
AWS_ACCESS_KEY_ID: Private, access key of the AWS account with permissions to write to S3 and execute lambdas
AWS_SECRET_ACCESS_KEY: Private, secret access key of the AWS account with permissions to write to S3 and execute lambdas
How it works
Authentication is performed with Auth0. We then do the exchange of a Auth0 token for a GitHub access token in the lambda so as not to leak the GitHub access token. That exchange is done with each API call.
Once we have GitHub access code, we perform data fetching and calculations. Due to the hardcoded limit of 29 seconds for any lambda connected to the API Gateway, the approach taken here is to have two lambdas for doing the analysis work (in total 3 lambdas, 1 for fetching GitHub repositories and 2 for analyzing them).
The first analyzing lambda simply authenticatest he user and receives the request, and immediately responds with a path on which a frontend can expect JSON results to be (it's a path to S3
results directory). The second analyzing lambda then does the actual work and writes the results into the S3 bucket under the defined path. On the frontend, we poll the url until it responds with the actual data.
(Hopefully) Useful notes
Changing the name of the app
If you change the name of the "app" in
package.json, you will want to update it in the
index.js file since it's used as a prefix for the lambda name