Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create separate GitHub environments for each Elastic Beanstalk environment #820

Closed
seanh opened this issue Oct 24, 2022 · 1 comment
Closed
Assignees

Comments

@seanh
Copy link
Contributor

seanh commented Oct 24, 2022

Our deployment workflows currently create two GitHub environments named qa and prod for each one of our GitHub application repos. This is incorrect usage of GitHub environments: instead for each app we should create separate GitHub environments for each Elastic Beanstalk environment that the app is deployed to. For example:

  • For Via we might have GitHub environments named QA, LMS QA, Production and LMS Production, corresponding to Via's four Elastic Beanstalk environments
  • For LMS we might have GitHub environments named QA, Production and Canada, corresponding to that app's three Elastic Beanstalk environments
  • Bouncer would have only QA and Production, since it only has two Elastic Beanstalk environments
  • And so on

Creating this 1:1 GitHub environment:Elastic Beanstalk environment mapping will cause various deployment-related GitHub UI and API features that're currently broken to work correctly:

  • GitHub lets you associate a single URL with each deployment to an environment and links to that URL in various places where the environment or deployments to the environment appear in GitHub's UI and API.

    For example we'd use https://via.hypothes.is/ as the URL for Via's prod environment and https://qa-via.hypothes.is/ for Via's qa environment, and so on for the prod-lms and qa-lms Via environments.

    GitHub's docs about this: https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#using-an-environment

    We can't make use of this feature when we share a single GitHub environment between multiple Elastic Beanstalk environments: each GitHub environment can only have one deployment URL not multiple.

    I created a separate issue to add URLs to the environments once we've created separate GitHub environments for each Elastic Beanstalk environment: Add GitHub environment URLs #821

  • The Environments section in the right-hand sidebar on each GitHub repo's front page will helpfully list all of that repo's Elastic Beanstalk deployments. For example just QA and Production for Bouncer, but QA, Production, WebSocket QA, WebSocket Production and Canada for h. I think the sidebar will also link each environment to its URL. Very handy for finding each app's QA environment when you need to test it.

    Currently this Environment sidebar is unhelpful/uninformative: it just lists the same two qa and prod environments (not real environments) for each repo and doesn't link them to URLs. It also sometimes incorrectly labels one or the other of the environments as "inactive":

    Screenshot 2022-10-21 at 18 03 49
  • I think GitHub will also handily link the environments/deployments to their URLs when they appear on Actions workflow pages, on deployment history pages, etc.

  • I think the deployment history page will work properly:

    https://github.com/hypothesis/via/deployments/activity_log

    For one thing the section at the top of that page will show all an app's Elastic Beanstalk environments separately and which PR is currently deployed to each environment. Currently it just lists the qa and prod environments:

    Screenshot 2022-10-21 at 18 04 45

    The information shown here is incorrect. Taking Via as an example: usually, if all is going well, the same PR will be deployed to both of Via's production Elastic Beanstalk environments: via-prod and lms-via-prod. But this won't always be true. For example the job to deploy a PR to via-prod might succeed whilst the job to deploy the same PR to lms-via-prod fails for some reason. Now two different PRs are deployed to these two different environments. So the above UI needs to present the two different environments separately. It's not valid to state that a given PR is deployed to "prod" because there isn't really a single prod environment.

    The filter menu on that page will also allow us to filter each Elastic Beanstalk environment separately, instead of just qa and prod:

    Screenshot 2022-10-21 at 18 08 43

    The deployment history will also be correct: currently for a given pull request (for example: remove-jenkinsfile, Remove Jenkinsfile #808) the deployment history page shows two apparently duplicate deployments of the same PR to qa, one marked as "active" and the other "inactive", and two apparently duplicate deployments of the same PR to prod, again one "active" and one "inactive". The reason there are two deployments to QA and two to prod is that there are two QA Elastic Beanstalk environments (via-qa and lms-via-qa) and two Elastic Beanstalk prod environments (via-prod and lms-via-prod) and our GitHub Actions workflow has two jobs that deploy to the two Elastic Beanstalk QA environments (in parallel) and two jobs that deploy to the two Elastic Beanstalk prod environments (also in parallel). But you can't see this in GitHub's deployment history UI: the two deployments to qa look exactly the same as each other and the two deployments to prod look the same. They look like duplicate entries:

    Screenshot 2022-10-21 at 18 09 41

    The reason is that we've told GitHub that there is just one qa environment and one prod environment. So what happens is:

    1. Parallel jobs start deploying to the via-prod and lms-via-prod Elastic Beanstalk environments. GitHub thinks that these are two parallel jobs deploying the same commit to the same single prod environment
    2. The via-prod deployment finishes first and GitHub's deployment history briefly marks this deployment as "active"
    3. Then the lms-via-prod deployment finishes and since GitHub thinks both jobs are deploying to a single prod environment it now marks the lms-via-prod deployment as "active" and the via-prod one as "inactive". It thinks that the lms-via-prod deployment has replaced the via-prod deployment in the single prod environment.

    What should happen is that both deployments should remain marked as active because they are both currently deployed to two different Elastic Beanstalk environments, and the history should show the two different environment names instead of calling them both prod.

@seanh seanh self-assigned this Oct 24, 2022
seanh referenced this issue in hypothesis/workflows Oct 24, 2022
Part of https://github.com/hypothesis/workflows/issues/3 and
https://github.com/hypothesis/workflows/issues/4.

Add a new `deploy.yml` shared workflow which is similar to the existing
`eb-update.yml` workflow but allows the caller to provide the GitHub
environment name and URL as inputs.

This will enable us to change the caller workflows in each of our app
repos one-by-one to call the new `deploy.yml` workflow instead of
`eb-update.yml`, correcting the GitHub environment names and adding
environment URLs to each app repo one by one. Once all app repos have
been moved over we can then finally delete the original `eb-update.yml`.
seanh referenced this issue in hypothesis/workflows Oct 24, 2022
Part of https://github.com/hypothesis/workflows/issues/3 and
https://github.com/hypothesis/workflows/issues/4.

Add a new `deploy.yml` shared workflow which is similar to the existing
`eb-update.yml` workflow but allows the caller to provide the GitHub
environment name and URL as inputs.

This will enable us to change the caller workflows in each of our app
repos one-by-one to call the new `deploy.yml` workflow instead of
`eb-update.yml`, correcting the GitHub environment names and adding
environment URLs to each app repo one by one. Once all app repos have
been moved over we can then finally delete the original `eb-update.yml`.
seanh referenced this issue in hypothesis/workflows Oct 25, 2022
Part of https://github.com/hypothesis/workflows/issues/3 and
https://github.com/hypothesis/workflows/issues/4.

Add a new `deploy.yml` shared workflow which is similar to the existing
`eb-update.yml` workflow but allows the caller to provide the GitHub
environment name and URL as inputs.

This will enable us to change the caller workflows in each of our app
repos one-by-one to call the new `deploy.yml` workflow instead of
`eb-update.yml`, correcting the GitHub environment names and adding
environment URLs to each app repo one by one. Once all app repos have
been moved over we can then finally delete the original `eb-update.yml`.
@seanh seanh transferred this issue from hypothesis/workflows Oct 27, 2022
@seanh
Copy link
Contributor Author

seanh commented Oct 27, 2022

This was fixed by #818

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant