Create new production environment

[bfirsh]

Deploy a new production environment, probably on Heroku.

Blockers

• Test staging environment with real data #323
• Failsafe mailgun config #351 or Switch backend to anymail #350
• Fix profile thumbnails on S3 #462

Pre-migration tasks

• Move embassynetwork.com DNS to Cloudflare (just for DNS - Cloudflare stuff can be disabled)
  • This can be done ahead of time, which will make the final migration much less scary.
• Backups Offsite backups #435
  • Heroku DB
  • Off-site DB (Tarsnap?) (see Optimize Tarsnap backups #293)
  • Off-site S3 (Tarsnap?)
• Configure Cloudflare to use a canonical domain
  • Currently embassynetwork.com works but www.embassynetwork.com doesn't. Both should work and redirect to a single place.
  • We need to decide whether we want https://embassynetwork.com or https://www.embassynetwork.com
• Fix scheduled tasks (as per Should we replace Celery with Heroku scheduler? #326)
• Tests for key workflows (headless as per Headless browser tests for key workflows #339 or otherwise)
• Run migration of database on staging to ensure the migration works
• Enable force HTTPS on Cloudflare
• Test generating thumbnails on staging once Fix thumbnails on S3 #474 has been merged
  • heroku run -a embassynetwork-staging ./manage.py generateimages
• Create production Heroku app (clone staging app), app name "embassynetwork-production" http://embassynetwork-production.herokuapp.com/
  • Create new clone of production database, recording commands executed in ... sections below
  • Migrate database to check migration works
  • Generate new secret key
  • Copy over production API keys (pending mailgun)
  • Upgrade to full dynos
  • Upgrade to standard database plan
  • Enable Heroku DB backups
  • heroku pg:backups:schedule -a embassynetwork-production DATABASE_URL --at '02:00 America/Los_Angeles'
  • Set up off-site backups
    • Follow instructions on https://github.com/bfirsh/heroku-tarsnap-backups
    • Distribute and store Tarsnap key to at least two people moved to Post Production Migration Tasks #483
  • Do initial move of static files
    • s3cmd sync --region us-east-1 -rv webapp/media/* s3://media.embassynetwork.com/public/
  • Do initial generation of thumbnails
    • heroku run -a embassynetwork-production ./manage.py generateimages
  • Set up media.embassynetwork.com
    • Cloudflare
    • heroku config:set -a embassynetwork-production MEDIA_URL=https://media.embassynetwork.com/public/
  • Test new production app works (click around!)
  • Restore and verify backup is intact. (Actually restore database locally.)
• Finalize workflow to deploy to production
• 2FA all the things moved to Post Production Migration Tasks #483

Migration steps

• Notify folks on Slack that site will be down for 30 mins, and emails might not work over next few days
• Turn off Celery on old site
• Take down current site somehow
• Dump database (as per)
  • PGPASSWORD=XXXX pg_dump -Fc --no-acl --no-owner -h localhost -U postgres modernomadb > modernomadb.dump
• scp db dump to local machine and then upload to S3
  • aws s3 cp modernomadb.dump s3://jessykate-scratch
• Import database dump on heroku
  • upload DB export to S3, get signed URL
  • heroku pg:backups:restore -a embassynetwork-production <s3 signed url> DATABASE_URL
• Run migrations on database
  • heroku run -a embassynetwork-production ./manage.py migrate
• Do final move of static files
  • s3cmd sync --region us-east-1 -rv webapp/media/* s3://media.embassynetwork.com/public/
• Do final generation of thumbnails
• heroku run -a embassynetwork-production ./manage.py generateimages
• Click around and check everything works on https://embassynetwork-production.herokuapp.com/
• Swap embassynetwork.com and www.embassynetwork.com on Cloudflare to CNAME embassynetwork-production.herokuapp.com, and switch on Cloudflare
• Click around and check everything works on https://embassynetwork.com
• Roll Stripe keys (done after migration so it doesn't break existing environment)
• Roll Mailgun keys
• Create scheduled task in Heroku scheduler app
  • 11:00 UTC daily: ./manage.py run_daily_tasks
• Enable email sending
• Remove .herokuapp.com from ALLOWED_HOSTS
  • heroku config:set -a embassynetwork-production ALLOWED_HOSTS=embassynetwork.com
• 🎉

[bfirsh]

@jessykate @jonathan-s Filled out everything in the checklist I could think of off the top of my head. Can you think of anything else?

[jonathan-s]

@bfirsh Can't really think of anything else.

[bfirsh]

@jonathan-s

[jonathan-s]

Laughing out loud! This made my morning. Wish I was there :)

On Thu 7. Mar 2019 at 03:40, Ben Firshman ***@***.***> wrote: @jonathan-s <https://github.com/jonathan-s> [image: untitled] <https://user-images.githubusercontent.com/40906/53928322-8a108800-403e-11e9-966f-1a7fbc65286d.jpg> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#324 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACBsEpRUYGZ5wR8E73yhU0uiOqefC_meks5vUHrggaJpZM4ZT_c9> .

-- Sent from Gmail Mobile