[develop] v1.23.0

[zwliew]

Features:

• Allow activation of SNS as a fallback for SMS
• Support whitelisting of domains through a new agencies table

Improvements:

• Upgrade worker and backend Docker instances to use Python 3
• Add frontend tests for critical workflows
• Only prepend "Copy of" when a campaign name doesn't already have it
• Upgrade dependencies

Bug Fixes:

• Fix parsing of SendGrid email callbacks and disable custom from for SendGrid emails
• Prevent dashboard rows and campaign titles from overflowing
• Update message delivery statuses only if error does not exist

Tests

Test frontend tests: #1086
Test campaign title overflow fixes: #1147
Test agencies table: #1141
Test SNS fallback: #1019
Test SendGrid fallback: #1026
Test message delivery statuses: #1150

Frontend tests

• Frontend tests run and pass on Travis

Campaign title overflow fixes

• A long campaign title doesn't cause dashboard rows to overflow
• A long campaign title gets truncated in the edit campaign page
• "Copy of" is only prepended to a campaign title that doesn't already have it

agencies whitelisting title (tested on staging only)

• Adding @gmail.com to the table allows OTPs to be sent to those emails
• Removing @gmail.com from the table disallows OTPs from being sent to those emails
• OTPs can be sent to .gov.sg emails

SNS SMS fallback (tested on staging only)

• All SMSes are sent with SNS with a sender ID of Postman during fallback (SNS receives it, but it isn't delivered due to exceeding the quota)
• Validating a new credential should error during fallback
• Validating an existing credential should error during fallback
• Adding a new credential to a campaign should error during fallback

SendGrid email fallback (tested on staging only)

• All emails are sent with default from during fallback
• Sends email campaign with SendGrid
• SendGrid delivery statuses are set correctly (status and error_code)
• Sends email campaign with SES
• SES delivery statuses are set correctly (status and error_code)
• Storing a template with a custom from address should error during fallback
• Verifying a custom from address should error during fallback

SMS delivery status (skipped)

• Calling /callback/sms with an error then a success should log the status as an ERROR
• Calling /callback/sms with a success than an error should log the status as an ERROR

Smoke tests

• Send email campaign
• Send protected email campaign
• View a protected email
• Unsubscribe from a mailing list
• Send Telegram campaign
• Send SMS campaign

Deploy Notes

Frontend tests have been added and should be run on every Travis and Amplify build. If any test fails, the entire frontend build should fail. In particular, Amplify should not publish the failed build.

New environment variables:

In both backend and worker:

• SMS_FALLBACK_ACTIVATE : Switch to true to use SNS as fallback for all SMS campaigns
• SMS_FALLBACK_SENDER_ID: Sender ID to use for SNS SMSes
• EMAIL_FALLBACK_ACTIVATE: Switch to true to use SendGrid as fallback for all emails. Ensure that the SMTP settings are properly configured as well.

New dependencies:

• @aws-sdk/sns-client : AWS SNS Client

New dev dependencies:

In frontend:

• msw: to mock backend API responses
• @testing-library/react: for idiomatic react testing APIs
• @testing-library/jest-dom: for idiomatic DOM assertion APIs
• @testing-library/user-event: for idiomatic DOM user event APIs
• jest-canvas-mock: to mock Canvas for Lottie as jsdom doesn't implement it
• @peculiar/webcrypto: as jsdom doesn't implement the WebCrypto API

Upgraded dependencies:

Dependency	From	To
redis	3.0.2	3.1.2
react-draft-wysiwyg	1.14.5	1.14.6
validator	13.0.0	13.6.0
@types/cheerio	0.22.24	0.22.28
@types/node	14.14.31	14.14.42
twilio	3.55.1	3.58.0
moment	2.24.0	2.29.1
@types/papaparse	5.0.3	5.2.5

New scripts:

• test: Runs the frontend tests. This should be run on every Travis build. If any test fails, the entire frontend build should fail.

IAM policy changes

• Attach postmangovsg-SNSPublishSMS to production Beanstalk instance role
• Attach postmangovsg-SNSPublishSMS to production ECS task role