Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy 2023.03.1 to test #1306

Merged
merged 46 commits into from
Mar 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
70e2677
chore(deps): bump django from 4.1.6 to 4.1.7 in /appcontainer
dependabot[bot] Feb 17, 2023
5b12221
chore(deps-dev): bump cypress from 12.5.1 to 12.6.0 in /tests/cypress
dependabot[bot] Feb 20, 2023
40d4c46
fix: put back links that were accidentally removed in f8ebdece
angela-tran Feb 22, 2023
6ba355e
fix: formatting for HEALTHCHECK_USER_AGENTS callout
angela-tran Feb 22, 2023
cbff7d4
fix: typo in middleware name for HEALTHCHECK_USER_AGENTS
angela-tran Feb 22, 2023
db9ebfb
Fix: small issues on environment variable docs (#1291)
angela-tran Feb 22, 2023
e619097
chore: add support for setting SacRT merchant ID through env var
angela-tran Feb 22, 2023
af5372b
chore: add a separate type and verifier for SacRT senior discount
angela-tran Feb 22, 2023
963cf0b
docs: add documentation and scripts for setting Key Vault secrets
angela-tran Feb 23, 2023
657198d
refactor: move argument check above variable assignment
angela-tran Feb 24, 2023
bef677b
fix: use variables instead of argument index, which fixes a bug
angela-tran Feb 24, 2023
0031ae7
chore(deps): bump django from 4.1.6 to 4.1.7 in /appcontainer (#1274)
angela-tran Feb 24, 2023
a2208de
chore(deps-dev): bump cypress from 12.5.1 to 12.6.0 in /tests/cypress…
machikoyasuda Feb 27, 2023
26869e5
chore(pre-commit): autoupdate hooks
pre-commit-ci[bot] Feb 27, 2023
05d1930
chore(deps): bump sentry-sdk from 1.15.0 to 1.16.0 in /appcontainer
dependabot[bot] Feb 27, 2023
562df63
chore(deps-dev): bump cypress from 12.6.0 to 12.7.0 in /tests/cypress
dependabot[bot] Feb 27, 2023
62580da
chore(deps-dev): bump cypress from 12.6.0 to 12.7.0 in /tests/cypress…
afeld Feb 27, 2023
726ad18
chore(deps): bump sentry-sdk from 1.15.0 to 1.16.0 in /appcontainer (…
afeld Feb 27, 2023
02d6a45
fix(docs): fix numbering for Cypress setup steps
afeld Feb 28, 2023
9e6375e
fix(docs): fix+simplify Cypress setup steps
afeld Feb 28, 2023
e425759
chore: add incident response checklist
afeld Feb 28, 2023
bdd26d9
chore(pre-commit): autoupdate hooks (#1295)
thekaveman Feb 28, 2023
1d04d3d
Fix Cypress setup instructions (#1299)
afeld Feb 28, 2023
88dcc89
fix: allow for spaces in file paths when setting secrets
angela-tran Feb 28, 2023
0258ed7
fix: update incident response checklist based on feedback
afeld Mar 1, 2023
29212c7
chore: add incident response checklist (#1300)
afeld Mar 1, 2023
87129cb
Feat: set Key Vault values (#1293)
angela-tran Mar 2, 2023
575e660
fix(terraform): add purge_protection_enabled setting to key vault
angela-tran Mar 2, 2023
3e7966d
Fix: represent Key Vault purge protection in Terraform (#1301)
angela-tran Mar 3, 2023
5ce356d
feat: specify app setting needed to mount docker container
angela-tran Feb 24, 2023
ce479d8
chore(pre-commit): autoupdate hooks
pre-commit-ci[bot] Mar 6, 2023
f6e6682
chore(pre-commit): autoupdate hooks (#1304)
angela-tran Mar 7, 2023
3cf1ff7
feat: add data migration environment variables as app settings
angela-tran Feb 24, 2023
6709e96
feat: add more app settings that are the same across environments
angela-tran Feb 24, 2023
fd06e34
feat: add other secret app settings (e.g. Django, API keys, Sentry)
angela-tran Feb 24, 2023
835a631
chore: add throwaway app setting used solely for refreshing secrets
angela-tran Feb 24, 2023
2ec1e17
docs: app settings are now managed in Terraform
angela-tran Mar 3, 2023
4e48b6f
fix(docs): fix broken link to data migrations docs
angela-tran Mar 3, 2023
4a6fa7e
docs: add documentation about refreshing secrets
angela-tran Mar 3, 2023
8c2f7c6
refactor: use conditional expressions + null to selectively add settings
angela-tran Mar 3, 2023
00287f5
refactor: extract common string out into a local value
angela-tran Mar 7, 2023
5cdc6a9
refactor: use key vault reference for Django log level so it can vary
angela-tran Mar 7, 2023
3e9a2f7
feat: add helper script for viewing secret values from CLI
angela-tran Mar 7, 2023
5d1fa35
Feat: specify app settings as code (#1294)
angela-tran Mar 7, 2023
f1a486f
chore(release): bump version to 2023.03.1
angela-tran Mar 7, 2023
fa13fa2
Prepare Release 2023.03.1 (#1305)
angela-tran Mar 7, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
59 changes: 59 additions & 0 deletions .github/ISSUE_TEMPLATE/incident-response.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
name: Incident response
about: Use to track the steps that should happen during an incident, such as a data breach or unexpected downtime, even if it's just a suspicion.
---

_**Do not put sensitive information in this issue or Slack.** Use a file on Google Drive with access restricted._

**Severity:** _High/Medium/Low_

## Initiate

1. [x] Create an issue from this template.
1. [ ] Declare an incident in the relevant Slack channel, such as [#benefits-general][benefits-general].
- Include brief details about the concern.
1. [ ] Start a video call.
1. [ ] Share a link to the video call in Slack, asking relevant parties to join.
1. [ ] Delegate subsequent tasks.

## Assess

- [ ] Determine the impact.
- [ ] Assign the severity above:
- **High:** Possible/confirmed breach of sensitive information, such as production system credentials or personally-identifiable information (PII)
- **Medium:** Full Benefits downtime for more than 30 minutes
- **Low:** Partial service degredation, zero-day vulnerabilities, data loss

### Medium/High incidents

- [ ] Notify [#benefits-general][benefits-general].
- [ ] If the incident is lasting more than 60 minutes or their customers' data has been breached, notify the impacted transit agencies.
- [ ] If it's believed to be an ongoing attack (such as DDoS), notify DevSecOps.

## Remediate

- [ ] Delegate someone to take notes in the Slack thread.
- [ ] Check the [troubleshooting documentation](https://docs.calitp.org/benefits/deployment/troubleshooting/) for relevant information.
- [ ] Post in the Slack thread when the incident has been resolved.
- [ ] Retain any relevant materials.

### Medium/High incidents

- [ ] Provide updates to [#benefits-general][benefits-general] every 30 minutes.
- [ ] [Release hotfixes](https://docs.calitp.org/benefits/deployment/release/) as necessary.
- [ ] Notify [#benefits-general][benefits-general] when the incident has been resolved.

## Follow-up

- [ ] For Medium/High incidents, write an incident report. [Past examples.](https://drive.google.com/drive/folders/1MVixX2jFioaSiM3xtGB15rKyU99fScqH)
1. [ ] Write a draft.
- Link to relevant Slack messages, etc.
1. [ ] Get thumbs-up from those involved in the incident.
1. [ ] Share with relevant stakeholders.
- [ ] Create issues for follow-up tasks, such as:
- Adding monitoring
- Updating documentation
- Having the system fail more gracefully
- Scheduling a retrospective/post-mortem

[benefits-general]: https://cal-itp.slack.com/archives/C013W8RUAMU
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,12 @@ repos:
files: .py$

- repo: https://github.com/pre-commit/mirrors-prettier
rev: v3.0.0-alpha.4
rev: v3.0.0-alpha.6
hooks:
- id: prettier
types_or: [javascript, css]

- repo: https://github.com/Riverside-Healthcare/djLint
rev: v1.19.15
rev: v1.19.16
hooks:
- id: djlint-django
4 changes: 2 additions & 2 deletions appcontainer/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Authlib==1.2.0
Django==4.1.6
Django==4.1.7
django-csp==3.7
eligibility-api==2023.01.1
requests==2.28.2
sentry-sdk==1.15.0
sentry-sdk==1.16.0
six==1.16.0
2 changes: 1 addition & 1 deletion benefits/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
__version__ = "2023.02.1"
__version__ = "2023.03.1"

VERSION = __version__
55 changes: 41 additions & 14 deletions benefits/core/migrations/0002_sample_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,16 @@ def load_sample_data(app, *args, **kwargs):

EligibilityType = app.get_model("core", "EligibilityType")

senior_type = EligibilityType.objects.create(
name="senior", label="Senior", group_id=os.environ.get("MST_SENIOR_GROUP_ID", "group1")
mst_senior_type = EligibilityType.objects.create(
name="senior", label="Senior Discount (MST)", group_id=os.environ.get("MST_SENIOR_GROUP_ID", "group1")
)
courtesy_card_type = EligibilityType.objects.create(
name="courtesy_card", label="Courtesy Card", group_id=os.environ.get("MST_COURTESY_CARD_GROUP_ID", "group2")
mst_courtesy_card_type = EligibilityType.objects.create(
name="courtesy_card",
label="Courtesy Card Discount (MST)",
group_id=os.environ.get("MST_COURTESY_CARD_GROUP_ID", "group2"),
)
sacrt_senior_type = EligibilityType.objects.create(
name="senior", label="Senior Discount (SacRT)", group_id=os.environ.get("SACRT_SENIOR_GROUP_ID", "group3")
)

PemData = app.get_model("core", "PemData")
Expand Down Expand Up @@ -119,9 +124,9 @@ def load_sample_data(app, *args, **kwargs):

EligibilityVerifier = app.get_model("core", "EligibilityVerifier")

oauth_claims_verifier = EligibilityVerifier.objects.create(
name=os.environ.get("OAUTH_VERIFIER_NAME", "OAuth claims via Login.gov"),
eligibility_type=senior_type,
mst_oauth_claims_verifier = EligibilityVerifier.objects.create(
name=os.environ.get("MST_OAUTH_VERIFIER_NAME", "OAuth claims via Login.gov (MST)"),
eligibility_type=mst_senior_type,
auth_provider=auth_provider,
selection_label=_("eligibility.pages.index.login_gov.label"),
selection_label_description=_("eligibility.pages.index.login_gov.description"),
Expand All @@ -141,12 +146,12 @@ def load_sample_data(app, *args, **kwargs):
enrollment_success_expiry_item_details=None,
)

courtesy_card_verifier = EligibilityVerifier.objects.create(
mst_courtesy_card_verifier = EligibilityVerifier.objects.create(
name=os.environ.get("COURTESY_CARD_VERIFIER", "Eligibility Server Verifier"),
api_url=os.environ.get("COURTESY_CARD_VERIFIER_API_URL", "http://server:8000/verify"),
api_auth_header=os.environ.get("COURTESY_CARD_VERIFIER_API_AUTH_HEADER", "X-Server-API-Key"),
api_auth_key=os.environ.get("COURTESY_CARD_VERIFIER_API_AUTH_KEY", "server-auth-token"),
eligibility_type=courtesy_card_type,
eligibility_type=mst_courtesy_card_type,
public_key=server_public_key,
jwe_cek_enc=os.environ.get("COURTESY_CARD_VERIFIER_JWE_CEK_ENC", "A256CBC-HS512"),
jwe_encryption_alg=os.environ.get("COURTESY_CARD_VERIFIER_JWE_ENCRYPTION_ALG", "RSA-OAEP"),
Expand Down Expand Up @@ -181,6 +186,28 @@ def load_sample_data(app, *args, **kwargs):
enrollment_success_expiry_item_details=_("enrollment.pages.success.mst_cc.expiry_item.details"),
)

sacrt_oauth_claims_verifier = EligibilityVerifier.objects.create(
name=os.environ.get("SACRT_OAUTH_VERIFIER_NAME", "OAuth claims via Login.gov (SacRT)"),
eligibility_type=sacrt_senior_type,
auth_provider=auth_provider,
selection_label=_("eligibility.pages.index.login_gov.label"),
selection_label_description=_("eligibility.pages.index.login_gov.description"),
start_title=_("eligibility.pages.start.login_gov.title"),
start_headline=_("eligibility.pages.start.login_gov.headline"),
start_item_heading=_("eligibility.pages.start.login_gov.start_item.heading"),
start_item_details=_("eligibility.pages.start.login_gov.start_item.details"),
start_help_anchor="login-gov",
unverified_title=_("eligibility.pages.unverified.login_gov.title"),
unverified_blurb=_("eligibility.pages.unverified.login_gov.p[0]"),
eligibility_confirmed_item_heading=_("enrollment.pages.index.login_gov.eligibility_confirmed_item.heading"),
eligibility_confirmed_item_details=_(
"enrollment.pages.index.login_gov.eligibility_confirmed_item.details%(transit_agency_short_name)s"
),
enrollment_success_confirm_item_details=_("enrollment.pages.success.login_gov.confirm_item.details"),
enrollment_success_expiry_item_heading=None,
enrollment_success_expiry_item_details=None,
)

PaymentProcessor = app.get_model("core", "PaymentProcessor")

payment_processor = PaymentProcessor.objects.create(
Expand Down Expand Up @@ -222,15 +249,15 @@ def load_sample_data(app, *args, **kwargs):
payment_processor=payment_processor,
eligibility_index_intro=_("eligibility.pages.index.p[0].mst"),
)
mst_agency.eligibility_types.set([senior_type, courtesy_card_type])
mst_agency.eligibility_verifiers.set([oauth_claims_verifier, courtesy_card_verifier])
mst_agency.eligibility_types.set([mst_senior_type, mst_courtesy_card_type])
mst_agency.eligibility_verifiers.set([mst_oauth_claims_verifier, mst_courtesy_card_verifier])

sacrt_agency = TransitAgency.objects.create(
slug="sacrt",
short_name=os.environ.get("SACRT_AGENCY_SHORT_NAME", "SacRT (sample)"),
long_name=os.environ.get("SACRT_AGENCY_LONG_NAME", "Sacramento Regional Transit (sample)"),
agency_id="sacrt",
merchant_id="sacrt",
merchant_id=os.environ.get("SACRT_AGENCY_MERCHANT_ID", "sacrt"),
info_url="https://sacrt.com/",
phone="916-321-2877",
active=True,
Expand All @@ -240,8 +267,8 @@ def load_sample_data(app, *args, **kwargs):
payment_processor=payment_processor,
eligibility_index_intro=_("eligibility.pages.index.p[0].sacrt"),
)
sacrt_agency.eligibility_types.set([senior_type])
sacrt_agency.eligibility_verifiers.set([oauth_claims_verifier])
sacrt_agency.eligibility_types.set([sacrt_senior_type])
sacrt_agency.eligibility_verifiers.set([sacrt_oauth_claims_verifier])


class Migration(migrations.Migration):
Expand Down
8 changes: 6 additions & 2 deletions docs/configuration/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,9 +149,9 @@ Comma-separated list of hosts which are trusted origins for unsafe requests (e.g

!!! warning "Deployment configuration"

You must change this setting when deploying the app to a non-localhost domain
You must change this setting when deploying the app to a non-localhost domain

Comma-separated list of User-Agent strings which, when present as an HTTP header, should only receive healthcheck responses. Used by our `HealthcheckUserAgent` middleware.
Comma-separated list of User-Agent strings which, when present as an HTTP header, should only receive healthcheck responses. Used by our `HealthcheckUserAgents` middleware.

## Cypress tests

Expand Down Expand Up @@ -183,3 +183,7 @@ Enables [sending events to Sentry](../../deployment/troubleshooting/#error-monit
[`environment` config value](https://docs.sentry.io/platforms/python/configuration/options/#environment)

Segments errors by which deployment they occur in. This defaults to `local`, and can be set to match one of the [environment names](../../deployment/infrastructure/#environments).

[app-service-config]: https://docs.microsoft.com/en-us/azure/app-service/configure-common?tabs=portal
[benefits-secrets]: https://github.com/cal-itp/benefits-secrets
[getting-started_create-env]: ../getting-started/README.md#create-an-environment-file
7 changes: 7 additions & 0 deletions docs/deployment/.pages
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
nav:
- README.md
- infrastructure.md
- secrets.md
- release.md
- troubleshooting.md
- workflows.md
2 changes: 1 addition & 1 deletion docs/deployment/infrastructure.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ Within the `CDT Digital CA` directory ([how to switch](https://learn.microsoft.c

All resources in these Resource Groups should be reflected in Terraform in this repository. The exceptions are:

- Secrets, such as values under [Key Vault](https://azure.microsoft.com/en-us/services/key-vault/) and [App Service application settings](https://docs.microsoft.com/en-us/azure/app-service/configure-common#configure-app-settings). [`prevent_destroy`](https://developer.hashicorp.com/terraform/tutorials/state/resource-lifecycle#prevent-resource-deletion) is used on these Resources.
- Secrets, such as values under [Key Vault](https://azure.microsoft.com/en-us/services/key-vault/). [`prevent_destroy`](https://developer.hashicorp.com/terraform/tutorials/state/resource-lifecycle#prevent-resource-deletion) is used on these Resources.
- [Things managed by DevSecOps](#ownership)

You'll see these referenced in Terraform as [data sources](https://developer.hashicorp.com/terraform/language/data-sources).
Expand Down
47 changes: 47 additions & 0 deletions docs/deployment/secrets.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Setting secrets

Secret values used by the Benefits application (such as API keys, private keys, certificates, etc.) are stored in an Azure Key Vault for each environment.

To set a secret, you can use the [Azure portal](https://learn.microsoft.com/en-us/azure/key-vault/secrets/quick-create-portal?source=recommendations) or the [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/keyvault/secret?view=azure-cli-latest#az-keyvault-secret-set).

There are helper scripts under `terraform/secrets` which build up the Azure CLI command, given some inputs. The usage is as follows:

First, make sure you are set up for [local development](../infrastructure/#local-development) and that you are in the `terraform/secrets` directory.

```bash
cd terraform/secrets
```

To set a secret by providing a value:

```bash
./value.sh <environment_letter> <secret_name> <secret_value>
```

where `environment_letter` is `D` for development, `T` for test, and `P` for production.

To set a secret by providing the path of a file containing the secret (useful for [multi-line secrets](https://learn.microsoft.com/en-us/azure/key-vault/secrets/multiline-secrets)):

```bash
./file.sh <environment_letter> <secret_name> <file_path>
```

To verify the value of a secret, you can use the helper script named `read.sh`.

```bash
./read.sh <environment_letter> <secret_name>
```

## Refreshing secrets

To make sure the Benefits application uses the latest secret values in Key Vault, you will need to make a change to the app service's configuration. If you don't do this step, the application will instead use cached values, which may not be what you expect. See the [Azure docs](https://learn.microsoft.com/en-us/azure/app-service/app-service-key-vault-references?tabs=azure-cli#rotation) for more details.

The steps are:

1. After setting new secret values, go to the App Service configuration in Azure Portal, and change the value of the setting named `change_me_to_refresh_secrets`.
1. Save your changes.

The effects of following those steps should be:

- A restart of the App Service is triggered.
- The next time that our Azure infrastructure pipeline is run, the value of `change_me_to_refresh_secrets` is set back to the value defined in our Terraform file for the App Service resource.
2 changes: 1 addition & 1 deletion docs/getting-started/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,4 +73,4 @@ docker compose down
[docker]: https://www.docker.com/products/docker-desktop
[sample-data]: https://github.com/cal-itp/benefits/tree/dev/benefits/core/migrations/0002_sample_data.py

[data-migration](https://github.com/cal-itp/benefits/tree/dev/benefits/core/migrations)
[data-migration]: (https://github.com/cal-itp/benefits/tree/dev/benefits/core/migrations)
40 changes: 18 additions & 22 deletions docs/tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,40 +14,36 @@ will install `cypress` and its dependencies on your machine. Make sure to run th

1. Ensure you have Node.js and NPM available on your local machine:

```bash
node -v
npm -v
```
```bash
node -v
npm -v
```

If not, [install Node.js](https://nodejs.org/en/download/) locally.
If not, [install Node.js](https://nodejs.org/en/download/) locally.

2. Start the the application container
2. Start the the application container:

```bash
cd .devcontainer/
docker compose up -d client
```
```bash
docker compose up -d client
```

3. Change into the `cypress` directory:

```bash
cd ..
cd tests/cypress
```
```bash
cd tests/cypress
```

4. Install all packages and `cypress`. Verify `cypress` installation succeeds:

```bash
npm install
npx cypress install
npx cypress verify
```
```bash
npm install
```

5. Run `cypress` with test environment variables and configuration variables:

```bash
CYPRESS_baseUrl=http://localhost:8000 npm run cypress:open
```
```bash
CYPRESS_baseUrl=http://localhost:8000 npm run cypress:open
```

See `tests/cypress/package.json` for more cypress scripts.

Expand Down
1 change: 1 addition & 0 deletions terraform/.terraform.lock.hcl

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.