Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create a staging deployment environment #185

Closed
csakoda opened this issue Mar 24, 2020 · 16 comments
Closed

Create a staging deployment environment #185

csakoda opened this issue Mar 24, 2020 · 16 comments

Comments

@csakoda
Copy link
Contributor

csakoda commented Mar 24, 2020 •
edited

To aid in the group development of the frontend and backend assets, we should create a staging environment that is discrete from production resources so we can test deploys of new versions without risking continuous access to the data we've already made available.

Related or perhaps alternatively, if we can document a pattern for a dev setting their own Google Sheet to play with, we may be able to defer the need here.
@raindrift
Copy link
Collaborator

For the frontend, it seems like loading the production data.json would make for the most realistic test. As someone who's broken the site twice now, I would appreciate such an environment. :)

@drwestco drwestco mentioned this issue Mar 25, 2020
Closed
@awong-dev
Copy link
Collaborator

So there are 2 major forks of work:
(a) Staging of the frontend code (what most of y'all have been working on)
(b) Staging of the cloud functions.

For (a), there is a clear path forward. The set of steps are as follows
(1) Create a production branch off master. Change deploy.sh to pull from production instead of master.
(2) Change deploy.sh to so it does the git sync/rsync steps a second time. But this time pulling from master, and pushing ti ~/findthemasks.com/staging.

That should handle the frontend staging server.

For (b), this is harder. I need someone to understand cloud functions and come up with a way to both figure out staging AND local development. It might be worth tapping external expertise.

@patricknelson
Copy link
Contributor

patricknelson commented Mar 25, 2020 •
edited

Could we graph out the current server architecture? Here's what I know so far from what I've picked up or hunted down on my own to get the "lay of the land":

  • findthemasks.com
    • CDN/WAF: Cloudflare
    • Server: Dreamhost (LAMP) shared hosting, so Apache web server with
    • Node/NPM installed server-side with scripts that interface with Firebase (installed under /functions)
    • Deploy PHP scripts that sync git
  • Google Sheets database
  • Firebase account that hits Google Sheets (scripts run on server noted above)

Those are the major components I'm already aware of. Is there anything else not listed here?

@csakoda
Copy link
Contributor Author

csakoda commented Mar 25, 2020

Firebase account that hits Google Shits (scripts run on server noted above)

Sheets? 😂

@csakoda
Copy link
Contributor Author

csakoda commented Mar 25, 2020

(1) Create a production branch off master. Change deploy.sh to pull from production instead of master.

Isn't the other way around a bit more natural? master branch is the up to date blessed code running production, and feature branches are made from master and then merged via PR. And then we can deploy any arbitrary branch, not just the code that's been merged into the staging branch.

@csakoda
Copy link
Contributor Author

csakoda commented Mar 25, 2020

For (b), this is harder. I need someone to understand cloud functions and come up with a way to both figure out staging AND local development. It might be worth tapping external expertise.

Happy to figure this out, I had a glimpse of how local can work and from there staging is just a hosted copy of that with a deploy mechanism wrapped around it.

I'll plan to put some time in on this tonight.

@patricknelson
Copy link
Contributor

patricknelson commented Mar 25, 2020 •
edited

Isn't the other way around a bit more natural? master branch is the up to date blessed code running production, and feature branches are made from master and then merged via PR. And then we can deploy any arbitrary branch, not just the code that's been merged into the staging branch.

This is what I have my team doing. We have 4 pre-production environments and each one serves different purposes. In this case we probably only need just one, but that environment (known here as stage?) would be where you can deploy any arbitrary branch. That said: Ideally you'd have a branch (maybe a throwaway branch that you can rewrite from time to time, discard, branch back off master again) where you can review in staging.

Git workflow: So this is where we dig into the git workflow. For example:

  • master: Blessed code; this is roughly equivalent to production, maybe one commit/tag ahead of what is currently deployed or will be deployed soon. Ideally we'd tag it, e.g. +0.0.1 for bug fixes and defects, +0.1.0 for features and +1.0.0 for project-wide backward incompatible breaking changes (basically semver).
  • issue-123-short-name: Feature branches that are always branched directly off of master (and never any other branch; ever). They are either rebased against master or have master merged back into them as developers work
  • stage (or qa, dev, etc): Also branched off of master but used as a testing/integration brach where you can merge in feature branches to preview them in the staging environment.

Local development: Then we should probably put some consideration into what is needed to facilitate segmentation (or isolation) for development purposes of the following:

  • Website code (a.k.a. "front-end" or static code); this must be accompanied by some kind of basic web server (e.g. nginx, node.js or similar, doesn't matter right now).
  • Bash + Node.js command line to facilitate execution of
  • Nice to have: Local emulation of as much as possible, at minimum a standard web server that we can all spin up and use. I call this one "nice to have" since I think most of us already have something we're happy with, but standardizing this (or at least offering something that new folks can pick up) will help new folks coming in. I propose setting up Dockerfile + docker-compose.yml (just throwing it out there).
  • Shared development Google Sheets spreadsheet

Documentation: Also, we should dig into documenting:

  • A git workflow of some sort (proposed one above)
  • The current systems architecture (my hypothesis also above)
  • How to get setup with our isolated/dev environments (e.g. running locally with docker, shared Google Sheets "database").

What do you all think?

@awong-dev
Copy link
Collaborator

Patrick's description of architecture is pretty much spot on. The design philosophy was to produce a 100% static site. We don't even use php for anything user facing.

As for branching strategy, we can do whatever folks are comfortable with. What I've been used to is having master be volatile, staging be pushed to something like staging.findthemasks.com where one can examine potential issues with authorizations, cors, etc. on the live site, and then that's okay, merge to production for a final push. Totally willing to go another way though.

However, the current hacked up deploy method doesn't really have a great way to create bunches of test sites per branch. Why do people want to deploy a development branch? Just to see it?

As for docker, since this is designed as a static site, there's no binary dependency other than the firebase bits, which only 2-3 people touch anyways. I don't see a benefit to having folks download the image and deal with the various containerization kernel bugs that show up over various platforms.

@patricknelson
Copy link
Contributor

patricknelson commented Mar 25, 2020 •
edited

Just asking for a quick clarification, but:

having master be volatile

staging be pushed to something like staging.findthemasks.com

merge to production for a final push

Those are the staging and production branches, correct? Also, if master is volatile, it's still considered "blessed" canon (so to speak), correct? That is, it's been thoroughly tested (in staging branch) but not merged into master until we're relatively certain it's golden, right? Just clarifying your intent. If so, we're still fairly aligned.

Also, in that workflow, everything is the same (i.e. importantly, feature branches based on master and only merged back once entirely complete and as bug-free as we can hope for). I think in this case, the main difference would be that the production branch servers the purpose I was using in lieu of tags, and may work better with the "wild west" quick iteration style (with tags, you can quickly revert as needed and always be certain that the tag is always on master and thus a validated release). When I deploy to production, I simply run ./scripts/deploy 1.2.3 and it immediately gets to work deploying that tag to my cluster. The quick version of that is simply git checkout 1.2.3. In this case, instead you'd just do git pull production and instead of tagging, you simply merge into production once master gets up to a state you want in production.

Does that sound about right so far?

@csakoda
Copy link
Contributor Author

csakoda commented Mar 25, 2020

Though I spoke against master being volatile I have no serious opinion, happy to work in the system @patricknelson has proposed.

@patricknelson
Copy link
Contributor

patricknelson commented Mar 25, 2020 •
edited

p.s.

As for docker, since this is designed as a static site, there's no binary dependency other than the firebase bits, which only 2-3 people touch anyways. I don't see a benefit to having folks download the image and deal with the various containerization kernel bugs that show up over various platforms.

Yeah, it can get complicated if you don't already have it up and running (for me it's dead simple, e.g. steps 7 + 8 here) but some folks may have problems accessing shared files. I just like the ability to quickly consolidate everything you need with relatively little (or no) work given how self contained it is.

The node based server implemented in #196 works well! I mean, it's currently not working (need to submit a bug for @csakoda) but it's super simple.

@patricknelson patricknelson mentioned this issue Mar 25, 2020
Open
@patricknelson
Copy link
Contributor

cc This thread for external visibility. It's possible we may have a Heroku pipeline to help with staging master + PR's, so more to come soon, thanks to @alex on that.

image

@alex
Copy link
Collaborator

alex commented Mar 26, 2020 via email

@alex
Copy link
Collaborator

alex commented Mar 26, 2020 via email

@awong-dev
Copy link
Collaborator

I think the PR apps have solved the immediate problem. Closing for now. Thanks for all the discussion everyone!

@patricknelson
Copy link
Contributor

Heroku FTW! Thanks @alex. :shipit:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@alex @awong-dev @raindrift @csakoda @patricknelson