build: release v6.114.0

[KenLSM]

New

• fix: prefill myinfo dropdown field options when faking sample submission data #7225
• feat(mrf): attachments #7181
• fix(mrf): design review add workflow page #7218
• fix(mrf): skip verifying OTP signatures on decryption for export #7192
• chore: update mrf copy on create-page #7217
• refactor: move override code closer to other overrides #7216
• fix(mrf): remove any response limits for MRF #7214
• chore(mrf): prevent form activation if email confirmation is present #7201
• feat: Disabling MRF for Forms with SingPass #7200
• feat(mrf): backend validation for field locking #7143
• chore: switch advertising for checkpoint to mrf #7191
• feat: disabling submit another response button (MRF) #7186
• chore: remove unused deps #7150
• build: merge release v6.113.0 back to develop #7149
• feat: disabling response limit toggle for MRF #7180
• revert: "chore(deps-dev): bump jest-mock-axios from 4.7.2 to 4.7.3" #7179
• fix: remove number and decimal scrolling #7171
• build: release v6.113.0 #7148
• chore: bump version to v6.114.0 993e31e

Dependencies

• fix(deps): bump mongodb-memory-server-core from 9.1.6 to 9.1.7 #7211
• fix(deps): bump aws-sdk from 2.1354.0 to 2.1585.0 #7209
• fix(deps): bump neverthrow from 6.0.0 to 6.1.0 #7208
• fix(deps): bump @babel/runtime from 7.22.3 to 7.24.1 #7207
• fix(deps): bump libphonenumber-js from 1.10.49 to 1.10.59 #7204
• fix(deps): bump http-status-codes from 2.2.0 to 2.3.0 #7203
• fix(deps): bump express from 4.18.2 to 4.19.2 #7199
• fix(deps): bump libphonenumber-js from 1.10.58 to 1.10.59 in /shared #7195
• fix(deps): bump type-fest from 4.13.1 to 4.14.0 in /shared #7190
• chore(deps): Upgrade bson from 5.4.0 to 5.5.1 #7152
• fix(deps): bump type-fest from 4.8.3 to 4.13.1 in /shared #7178
• fix(deps): bump celebrate from 15.0.1 to 15.0.3 #7175
• fix(deps): bump zod from 3.22.3 to 3.22.4 #7174
• fix(deps): bump libphonenumber-js from 1.10.52 to 1.10.58 in /shared #7168
• fix(deps): bump @faker-js/faker from 8.0.2 to 8.4.1 #7161
• fix(deps): bump ts-essentials from 9.3.1 to 9.4.1 #7158
• fix(deps): bump the npm_and_yarn group group with 2 updates #7151
• chore(deps): bump fast-xml-parser, @aws-sdk/client-secrets-manager and @aws-sdk/client-ssm in /functions/form-payment-reconciliation #6490

Dev-Dependencies

• chore(deps-dev): bump @babel/core from 7.23.9 to 7.24.3 #7206
• chore(deps-dev): bump eslint-plugin-playwright from 0.12.0 to 1.5.4 #7189
• chore(deps-dev): bump express from 4.18.2 to 4.19.2 in /frontend #7198
• chore(deps-dev): bump rimraf from 4.4.0 to 5.0.5 #7173
• chore(deps-dev): bump @types/convict from 6.1.1 to 6.1.6 #7176
• chore(deps-dev): bump ts-node from 10.9.1 to 10.9.2 #7172
• chore(deps-dev): bump @types/lodash from 4.14.202 to 4.17.0 in /shared #7169
• chore(deps-dev): bump @types/uuid from 9.0.0 to 9.0.8 #7165
• chore(deps-dev): bump @types/cookie-parser from 1.4.3 to 1.4.7 #7163
• chore(deps-dev): bump @opengovsg/mockpass from 4.0.8 to 4.1.0 #7160
• chore(deps-dev): bump @types/html-escaper from 3.0.0 to 3.0.2 #7159
• chore(deps-dev): bump @types/busboy from 1.5.0 to 1.5.3 #7157
• chore(deps-dev): bump jest-mock-axios from 4.7.2 to 4.7.3 #7154

Tests

feat(mrf): attachments #7181

@kathleenkhy

Preparation:

• Create an MRF form
  • Add 2x attachment field
  • Assign 3 workflow steps
    • Mark AttachmentField Improve Email mode messaging (smol ticket) #1 required for Step 1
    • Mark AttachmentField Update documentation for site banners #2 required for Step 2
    • Mark AttachmentField Improve Email mode messaging (smol ticket) #1 + Update documentation for site banners #2 required for Step 3

Respondent 1

• Attach file 1 (Respondent Improve Email mode messaging (smol ticket) #1) to field 1
• Ensure that field 2 is disabled
• Ensure that form can be submitted

Respondent 2

• Attach file 2 (Respondent Update documentation for site banners #2) to field 2
• Ensure that field 1 is disabled
• Ensure that download of file 1 is allowed and content is correct
• Ensure that form can be submitted

Respondent 3

• Ensure that download of file 1 is allowed and content is correct
• Ensure that download of file 2 is allowed and content is correct
• Attach file 3 (Respondent chore: update documentation for banner environment variables #3) to field 1
• Attach file 4 (Respondent chore: update documentation for banner environment variables #3) to field 2
• Ensure that form can be submitted

Attachments downloaded on admin response page should be the last submitted content

• Ensure that the attachment for field 1 is file 3
• Ensure that the attachment for field 2 is file 4

Ensure that scanner works

• Using the MRF form prepared earlier
• Download the eicar test file
• Attach eicar file to field 1
• Ensure that the form fails to submit and rejects with file has failed virus scan

fix(mrf): skip verifying OTP signatures on decryption for export #7192

@KenLSM

• Create an MRF form with email and mobile otp for respondent 1 and another email and mobile otp for respondent 2. Fill the form for both respondents, then attempt to export the results as CSV. This should work.

feat(mrf): backend validation for field locking #7143

@KenLSM

Create a multirespondent form with a simple workflow (e.g. two respondents). Make sure there are a mix of shared and unshared fields (in this case "shared" means editable by both respondent 1 and respondent 2). In this example, I created a Q1 radio, Q2 dropdown, Q3 email, Q4 checkbox, Q5 yes/no, Q6 short answer. Respondent 1 is assigned Q1-4 and respondent 2 is assigned Q3-6, so that Q3 and Q4 are shared.

Initial submission

• Send a valid submission via cURL. This should succeed.

curl 'http://localhost:3000/api/v3/forms/65680f30df564e005e446e2d/submissions/multirespondent?captchaResponse=null' \
  -H 'Accept: application/json, text/plain, */*' \
  -H 'Accept-Language: en-US,en;q=0.9' \
  -H 'Connection: keep-alive' \
  -H 'Content-Type: multipart/form-data; boundary=----WebKitFormBoundarygrSB7FoxtaZ5Ycg7' \
  -H 'DNT: 1' \
  -H 'Origin: http://localhost:3000' \
  -H 'Referer: http://localhost:3000/65680f30df564e005e446e2d' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Site: same-origin' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36' \
  -H 'sec-ch-ua: "Not(A:Brand";v="24", "Chromium";v="122"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "macOS"' \
  --data-raw $'------WebKitFormBoundarygrSB7FoxtaZ5Ycg7\r\nContent-Disposition: form-data; name="body"\r\n\r\n{"responses":{"6571f03219eb850112ae990e":{"fieldType":"radiobutton","answer":{"value":"Radio 1"}},"65d4238b1230effbfd6f74cb":{"fieldType":"dropdown","answer":"Dropdown 1"},"65f9257217e471530f22bcd9":{"fieldType":"email","answer":{"value":"justyn@open.gov.sg"}},"65d423891230effbfd6f74b7":{"fieldType":"checkbox","answer":{"value":["Checkbox 1"]}}},"responseMetadata":{"responseTimeMs":33230,"numVisibleFields":6},"version":3}\r\n------WebKitFormBoundarygrSB7FoxtaZ5Ycg7--\r\n'
{"message":"Form submission successful.","submissionId":"65f926ca17e471530f22bea5","timestamp":1710827210246}

• Send a submission where a respondent 2-only field (e.g. Q5/6) is filled. This should be rejected by the server.

curl 'http://localhost:3000/api/v3/forms/65680f30df564e005e446e2d/submissions/multirespondent?captchaResponse=null' \
  -H 'Accept: application/json, text/plain, */*' \
  -H 'Accept-Language: en-US,en;q=0.9' \
  -H 'Connection: keep-alive' \
  -H 'Content-Type: multipart/form-data; boundary=----WebKitFormBoundarygrSB7FoxtaZ5Ycg7' \
  -H 'DNT: 1' \
  -H 'Origin: http://localhost:3000' \
  -H 'Referer: http://localhost:3000/65680f30df564e005e446e2d' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Site: same-origin' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36' \
  -H 'sec-ch-ua: "Not(A:Brand";v="24", "Chromium";v="122"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "macOS"' \
  --data-raw $'------WebKitFormBoundarygrSB7FoxtaZ5Ycg7\r\nContent-Disposition: form-data; name="body"\r\n\r\n{"responses":{"6571f03219eb850112ae990e":{"fieldType":"radiobutton","answer":{"value":"Radio 1"}},"65d4238b1230effbfd6f74cb":{"fieldType":"dropdown","answer":"Dropdown 1"},"65f9257217e471530f22bcd9":{"fieldType":"email","answer":{"value":"justyn@open.gov.sg"}},"65d423891230effbfd6f74b7":{"fieldType":"checkbox","answer":{"value":["Checkbox 1"]}},"65f9258417e471530f22bd46":{"fieldType":"yes_no","answer":"Yes"}},"responseMetadata":{"responseTimeMs":33230,"numVisibleFields":6},"version":3}\r\n------WebKitFormBoundarygrSB7FoxtaZ5Ycg7--\r\n'
{"message":"There is something wrong with your form submission. Please check your responses and try again. If the problem persists, please refresh the page."}

Respondent 2 submission
(if you're copying cURL from a browser network tab, don't forget to change the submissionId in the URL, the response itself, and the submission secret key in the data section)

• Send a valid submission via cURL. This should succeed.
• Send a submission where a shared field (e.g. Q3/4) is modified. This should succeed.

curl 'http://localhost:3000/api/v3/forms/65680f30df564e005e446e2d/submissions/65f926ca17e471530f22bea5?captchaResponse=null' \
  -X 'PUT' \
  -H 'Accept: application/json, text/plain, */*' \
  -H 'Accept-Language: en-US,en;q=0.9' \
  -H 'Connection: keep-alive' \
  -H 'Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryMAkuNxQsfBvsmprX' \
  -H 'DNT: 1' \
  -H 'Origin: http://localhost:3000' \
  -H 'Referer: http://localhost:3000/65680f30df564e005e446e2d/edit/65f9265b17e471530f22be96?key=o2QsH57uSqkqdkHelkoJt2UboIwb7alecrA7IOLlsJE%3D' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Site: same-origin' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36' \
  -H 'sec-ch-ua: "Not(A:Brand";v="24", "Chromium";v="122"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "macOS"' \
  --data-raw $'------WebKitFormBoundaryMAkuNxQsfBvsmprX\r\nContent-Disposition: form-data; name="body"\r\n\r\n{"responses":{"6571f03219eb850112ae990e":{"fieldType":"radiobutton","answer":{"value":"Radio 1"}},"65d4238b1230effbfd6f74cb":{"fieldType":"dropdown","answer":"Dropdown 1"},"65f9257217e471530f22bcd9":{"fieldType":"email","answer":{"value":"justyn@open.gov.sg"}},"65d423891230effbfd6f74b7":{"fieldType":"checkbox","answer":{"value":["Checkbox 1","Checkbox 2"]}},"65f9258417e471530f22bd46":{"fieldType":"yes_no","answer":"No"},"65d423821230effbfd6f7464":{"fieldType":"textfield","answer":"abcde"}},"responseMetadata":{"responseTimeMs":7236,"numVisibleFields":6},"submissionSecretKey":"Z2T/aqbaHECMe+JX1Wpwlz0weEA9xm/m8PoSUCvR60U=","version":3}\r\n------WebKitFormBoundaryMAkuNxQsfBvsmprX--\r\n'
{"message":"Form submission successful.","submissionId":"65f926ca17e471530f22bea5","timestamp":1710827210246}

• Send a submission where a respondent 1-only field is modified. This should be rejected by the server.

curl 'http://localhost:3000/api/v3/forms/65680f30df564e005e446e2d/submissions/65f926ca17e471530f22bea5?captchaResponse=null' \
  -X 'PUT' \
  -H 'Accept: application/json, text/plain, */*' \
  -H 'Accept-Language: en-US,en;q=0.9' \
  -H 'Connection: keep-alive' \
  -H 'Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryMAkuNxQsfBvsmprX' \
  -H 'DNT: 1' \
  -H 'Origin: http://localhost:3000' \
  -H 'Referer: http://localhost:3000/65680f30df564e005e446e2d/edit/65f9265b17e471530f22be96?key=o2QsH57uSqkqdkHelkoJt2UboIwb7alecrA7IOLlsJE%3D' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Site: same-origin' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36' \
  -H 'sec-ch-ua: "Not(A:Brand";v="24", "Chromium";v="122"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "macOS"' \
  --data-raw $'------WebKitFormBoundaryMAkuNxQsfBvsmprX\r\nContent-Disposition: form-data; name="body"\r\n\r\n{"responses":{"6571f03219eb850112ae990e":{"fieldType":"radiobutton","answer":{"value":"Radio 2"}},"65d4238b1230effbfd6f74cb":{"fieldType":"dropdown","answer":"Dropdown 1"},"65f9257217e471530f22bcd9":{"fieldType":"email","answer":{"value":"justyn@open.gov.sg"}},"65d423891230effbfd6f74b7":{"fieldType":"checkbox","answer":{"value":["Checkbox 1"]}},"65f9258417e471530f22bd46":{"fieldType":"yes_no","answer":"No"},"65d423821230effbfd6f7464":{"fieldType":"textfield","answer":"abcde"}},"responseMetadata":{"responseTimeMs":7236,"numVisibleFields":6},"submissionSecretKey":"Z2T/aqbaHECMe+JX1Wpwlz0weEA9xm/m8PoSUCvR60U=","version":3}\r\n------WebKitFormBoundaryMAkuNxQsfBvsmprX--\r\n'
{"message":"There is something wrong with your form submission. Please check your responses and try again. If the problem persists, please refresh the page."}

Now, add logic to the form and make some fields optional.

Initial submission

• With prevent submit logic, send a valid submission via cURL. This should succeed.
• With prevent submit logic, send a submission via cURL such that submission should have been prevented. This should be rejected by the server.
• With show fields logic, send a valid submission via cURL. This should succeed.
• With show fields logic, send a submission via cURL such that a value is provided for a field that should have been hidden. This should be rejected by the server.

Respondent 2 submission
(take note that form fields and logic are saved with a submission lifetime, so you if you change the form logics you will need to make a new initial submission to generate a new edit link)

• With prevent submit logic, send a valid submission via cURL. This should succeed.
• With prevent submit logic, send a submission via cURL such that submission should have been prevented. This should be rejected by the server.
• With show fields logic, send a valid submission via cURL. This should succeed.
• With show fields logic, send a submission via cURL such that a value is provided for a field that should have been hidden. This should be rejected by the server.

fix: prefill myinfo dropdown field options when faking sample submission data #7225

@KenLSM

Regression Test

• Create form with myinfo fields that has dropdown
• Ensure that /api/v3/forms/:formId/sample-submissions API doesn't crash and returns data

[kathleenkhy]

Found some issues:

• when a file is empty (does not contain text)

Server timeout and/ or latency when files are larger: Gateway time-out Error code 504