Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: Release v4.46.0 #653

Merged
merged 40 commits into from
Nov 18, 2020
Merged

build: Release v4.46.0 #653

merged 40 commits into from
Nov 18, 2020

Conversation

karrui
Copy link
Contributor

@karrui karrui commented Nov 17, 2020

v4.46.0

New

  • feat: allow .oa files to be attached #650
  • feat: Backend implementation for bulk attachment downloading #555
  • feat(monitoring): add memory monitoring to EC2 instances #620
  • style: make transfer ownership email more prominent #630

Improvements

  • chore: use travis_retry to retry flaky tests automatically #641
  • ref: migrate createPresignedPost endpoint for images to TypeScript #604
  • ref: migrate createPresignedPost endpoint for logos to TypeScript #607
  • ref: migrate fetch form submission counts flow to Typescript #592
  • ref: migrate inline queries for retrieval of submissions metadata to model static methods #601
  • ref: migrate streamEncryptedResponses flow to Typescript #615
  • refactor: convert errors.server.controller to typescript utility function #602
  • refactor: convert req.hashedFields to Set #617
  • refactor: migrate MyInfo functionality to TypeScript #560
  • refactor: migrate NRIC validator to TypeScript #639
  • feat: upgrade spcp-auth-client #649

Fixes

  • fix: add required env vars for MyInfo #612
  • fix: package.json & package-lock.json to reduce vulnerabilities #637
  • fix: resolve & in url upon redirect, shift prefill to textfield component #569
  • fix: use maildev for dev environment emails #613
  • test: enable local e2e tests #616

Dependency updates

  • chore(deps-dev): bump @types/mongoose from 5.7.36 to 5.7.37 #597
  • chore(deps-dev): bump @types/mongoose from 5.7.37 to 5.10.0 #644
  • chore(deps-dev): bump @types/node from 14.14.6 to 14.14.7 #611
  • chore(deps-dev): bump @typescript-eslint/parser from 4.5.0 to 4.7.0 #610
  • chore(deps-dev): bump babel-loader from 8.1.0 to 8.2.1 #625
  • chore(deps-dev): bump eslint from 7.12.1 to 7.13.0 #643
  • chore(deps-dev): bump eslint-plugin-simple-import-sort from 5.0.3 to 6.0.0 #646
  • chore(deps-dev): bump supertest from 5.0.0 to 6.0.1 #622
  • chore(deps-dev): bump testcafe from 1.8.6 to 1.9.4 #632
  • fix(deps): bump @sentry/browser from 5.24.2 to 5.27.3 #631
  • fix(deps): bump angular from 1.8.0 to 1.8.2 #633
  • fix(deps): bump angular-aria from 1.8.0 to 1.8.2 #619
  • fix(deps): bump helmet from 4.1.1 to 4.2.0 #624
  • fix(deps): bump jwt-decode from 2.2.0 to 3.1.1 #594
  • fix(deps): bump mongoose from 5.10.10 to 5.10.14 #645
  • fix(deps): bump uuid from 8.3.0 to 8.3.1 #626

karrui and others added 30 commits November 10, 2020 13:34
…model static methods (#601)

* feat(SubmissionModel): add findMetadataById static method

* test(EncryptSubModel): add findMetadataById tests

* ref(EncryptSubModel): rename static to findSingleMetadata

* feat(SubmissionModel): add findAllMetadataByFormId static method

* test(EncryptSubModel): add findAllMetadataByFormId tests

* feat(AdminFormsRoute): add celebrate middleware to subs metadata api

* feat(EncryptSubsCtl): replace inlined queries with model static fns

* refactor(Submission): loosen model types according to schema

* fix(AdminFormsRoutes): import missing import

* ref(SubmissionModel): move type declaration outside of function
build: merge release 4.45.0 into develop
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 14.14.6 to 14.14.7.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…tion (#602)

* chore: rename errors server controller to typescript

* chore: add @types/mongodb as devdependency

* chore: define MongoError interface

* chore: typescripify errors server controller

* chore: rename errors server controller spec to ts

* chore: clean up errors.server.controller.spec.ts

* refactor: use destructuring assignment for isEmpty

* chore: remove types from JSDocs

* chore: remove defaultErrorMessage from exports

* chore: remove mongoDuplicateKeyError as only UserSchema has unique key and that is already validated

* refactor: use arrow function for getMongoErrorMessage

* refactor: clean up getMongoErrorMessage

* chore: shift errors.server.controller to utility function

* chore: update tests

* chore: shift defaultErrorMessage to function parameter
* fix: use maildev for dev environment emails

* docs: update README with local email instructions
Bumps [angular-aria](https://github.com/angular/angular.js) from 1.8.0 to 1.8.2.
- [Release notes](https://github.com/angular/angular.js/releases)
- [Changelog](https://github.com/angular/angular.js/blob/master/CHANGELOG.md)
- [Commits](angular/angular.js@v1.8.0...v1.8.2)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* feat: Backend implementation for attachment downloading using existing CSV decryption webworkers

* support for downloading files in worker threads
Bumps [@types/mongoose](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/mongoose) from 5.7.36 to 5.7.37.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/mongoose)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix(deps): bump jwt-decode from 2.2.0 to 3.1.1

Bumps [jwt-decode](https://github.com/auth0/jwt-decode) from 2.2.0 to 3.1.1.
- [Release notes](https://github.com/auth0/jwt-decode/releases)
- [Changelog](https://github.com/auth0/jwt-decode/blob/master/CHANGELOG.md)
- [Commits](auth0/jwt-decode@v2.2.0...v3.1.1)

Signed-off-by: dependabot[bot] <support@github.com>

* fix: import jwt-decode correctly

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Antariksh <antarikshmahajan@gmail.com>
Bumps [supertest](https://github.com/visionmedia/supertest) from 5.0.0 to 6.0.1.
- [Release notes](https://github.com/visionmedia/supertest/releases)
- [Commits](ladjs/supertest@v5.0.0...v6.0.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [uuid](https://github.com/uuidjs/uuid) from 8.3.0 to 8.3.1.
- [Release notes](https://github.com/uuidjs/uuid/releases)
- [Changelog](https://github.com/uuidjs/uuid/blob/master/CHANGELOG.md)
- [Commits](uuidjs/uuid@v8.3.0...v8.3.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
)

* feat(CoreErrors): add external errors for errors arising externally

* feat(AdminFormSvc): add createPresignedPostForImages function

* test(AdminFormSvc): add test cases for createPresignedPostForImages

* feat(AdminFormCtl): add handleCreatePresignedPostForImages function

* test(AdminFormCtl): add tests for handleCreatePresignedPostForImages

* ref(AdminFormsRoutes): replace createPresignedPostForImages fn

* feat: remove unused adminForms.createPresignedPostForImages fn

* feat(AdminFormSvc): add JSDoc to createPresignedPostForImages

* ref(AdminFormSvc): extract out core presigned code

* feat(AdminFormSvc): make error message more generic on external error
Bumps [testcafe](https://github.com/DevExpress/testcafe) from 1.8.6 to 1.9.4.
- [Release notes](https://github.com/DevExpress/testcafe/releases)
- [Changelog](https://github.com/DevExpress/testcafe/blob/master/CHANGELOG.md)
- [Commits](DevExpress/testcafe@v1.8.6...v1.9.4)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [babel-loader](https://github.com/babel/babel-loader) from 8.1.0 to 8.2.1.
- [Release notes](https://github.com/babel/babel-loader/releases)
- [Changelog](https://github.com/babel/babel-loader/blob/main/CHANGELOG.md)
- [Commits](babel/babel-loader@v8.1.0...v8.2.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [angular](https://github.com/angular/angular.js) from 1.8.0 to 1.8.2.
- [Release notes](https://github.com/angular/angular.js/releases)
- [Changelog](https://github.com/angular/angular.js/blob/master/CHANGELOG.md)
- [Commits](angular/angular.js@v1.8.0...v1.8.2)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* style: warning color for email

* refactor: use class instead of id
* fix(deps): bump helmet from 4.1.1 to 4.2.0

Bumps [helmet](https://github.com/helmetjs/helmet) from 4.1.1 to 4.2.0.
- [Release notes](https://github.com/helmetjs/helmet/releases)
- [Changelog](https://github.com/helmetjs/helmet/blob/main/CHANGELOG.md)
- [Commits](helmetjs/helmet@v4.1.1...v4.2.0)

Signed-off-by: dependabot[bot] <support@github.com>

* fix: type assertion for mockHelmet.contentSecurityPolicy

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: shuli-ogp <shuli@open.gov.sg>
Bumps [@sentry/browser](https://github.com/getsentry/sentry-javascript) from 5.24.2 to 5.27.3.
- [Release notes](https://github.com/getsentry/sentry-javascript/releases)
- [Changelog](https://github.com/getsentry/sentry-javascript/blob/master/CHANGELOG.md)
- [Commits](getsentry/sentry-javascript@5.24.2...5.27.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* feat(AdminFormSvc): add createPresignedPostForLogos function

* feat(AdminFormCtl): add handler for presigning logos upload POST url

* test(AdminFormSvc): add test cases for createPresignedPostForLogos

* ref(AdminFormsRoute): use new logo presign post handler

* test(AdminFormCtl): add tests for handleCreatePresignedPostForLogos

* feat(AdminForm): replace ExternalErrors with CreatePresignedUrlError
…omponent (#569)

* chore: replace '&amp;' in ejs template

* chore: remove he decoding

* chore: shift stealth prefil to textfield component

* chore: add tests for frontend.server.controller

* chore: update package-lock

* chore: add error handling to frontend.server.controller

* chore: reorganize tests, add tests for invalid request

* chore: log errors on frontend server controller

* chore: reorganise tests

* fix: relative paths

* chore: standardise logger action to calling function

* chore: return res
* tests: move submission module tests into module directory

* feat(SubmissionSvc): add new getFormSubmissionsCount fn

* test(SubmissionSvc): add tests for getFormSubmissionsCount

* feat(adminFormUtils): add assertHasReadPermissions function

Util function to check whether given user has read permissions to given form

* ref(dateUtil): add JSDoc + stricter types for createQueryWithDateParam

* ref(SubmissionSvc): allow dateRange child keys to be undefined

* ref(User): move getUserIdFromSession to auth utils

* feat(AdminFormCtl): add handleCountFormSubmissions handler

* feat(AdminFormRoute): replace form submission count controller

* feat(AdminFormRoutes): add joi validation to form submission count api

* test(AdminFormCtl): add tests for handleCountFormSubmissions

* feat: remove submissions.server.controller#count function and tests

* test(adminFormUtils): add tests for assertHasReadPermissions

* feat(AdminFormsRoute): use auth middleware in form sub counts handler

* fix: remove unused imports

* ref(AdminFormsRoutes): extract out YYYYMMDD regex into variable

* fix(AdminFormsRoutes): use Joi.object.and instead of object.with

`with` allows for a query with only `endDate` since only `startDate` is required, which is wrong
Bumps [eslint](https://github.com/eslint/eslint) from 7.12.1 to 7.13.0.
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/master/CHANGELOG.md)
- [Commits](eslint/eslint@v7.12.1...v7.13.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [@types/mongoose](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/mongoose) from 5.7.37 to 5.10.0.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/mongoose)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…6.0.0 (#646)

* chore(deps-dev): bump eslint-plugin-simple-import-sort

Bumps [eslint-plugin-simple-import-sort](https://github.com/lydell/eslint-plugin-simple-import-sort) from 5.0.3 to 6.0.0.
- [Release notes](https://github.com/lydell/eslint-plugin-simple-import-sort/releases)
- [Changelog](https://github.com/lydell/eslint-plugin-simple-import-sort/blob/main/CHANGELOG.md)
- [Commits](lydell/eslint-plugin-simple-import-sort@v5.0.3...v6.0.0)

Signed-off-by: dependabot[bot] <support@github.com>

* fix: fix .eslintrc

* chore: run lint

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Antariksh <antarikshmahajan@gmail.com>
karrui and others added 8 commits November 17, 2020 10:05
* feat(EncryptSubModel): add getSubmissionCursorByFormId static method

* feat(EncryptSubSvc): add getSubmissionCursor service fn

* feat(EncryptSubCtl): add handler for streaming encrypted response api

* ref(AdminFormsRoutes): use migrated handler for streaming responses

* feat(AdminFormsRoutes): add celebrate validtn for stream query params

* build: update tsconfig.build to allow inconsistent casing

due to transpiling code, some packages with lowercased type files will encounter some errors like:

TS1261: Already included file name 'path/to/node_modules/@types/JSONStream/index.d.ts' differs from file name 'path/to/node_modules/@types/jsonstream/index.d.ts' only in casing.

* test(EncryptSubSvc): add tests for getSubmissionCursor

* fix: use local copy of jsonstream types

* test(EncryptSubModel): add tests for getSubmissionCursorByFormId

* ref: use StatusCodes enums instead of hardcoded numbers

* feat: remove `end` event handler in stream

the close handler should be enough to handle everything. From testing, the end event is not called anyways

* ref(AdminFormsRoutes): extract out YYYYMMDD regex into variable

* fix(AdminFormsRoutes): use Joi.object.and instead of object.with

`with` allows for a query with only `endDate` since only `startDate` is required, which is wrong

* ref(EncryptSubSvc): add transformAttachmentMetaStream function

this is a refactor of the function recently added in #555

* feat(EncryptSubCtl): insert transform stream into download pipeline

* feat: remove unused stream responses handler

* feat(EncryptSubSvc): add logging to error

also use for of loop for correct callback return

* feat(EncryptSubCtl): add error handler for attachment pipe errors

* test(EncryptSubSvc): add tests for transformAttachmentMetaStream

* chore: trigger travis rebuild
Co-authored-by: Yuanruo Liang <yuanruo@open.gov.sg>
chore: merge release 4.45.1 hotfix back to develop
Bumps [mongoose](https://github.com/Automattic/mongoose) from 5.10.10 to 5.10.14.
- [Release notes](https://github.com/Automattic/mongoose/releases)
- [Changelog](https://github.com/Automattic/mongoose/blob/master/History.md)
- [Commits](Automattic/mongoose@5.10.10...5.10.14)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@karrui
Copy link
Contributor Author

karrui commented Nov 17, 2020

Tests

@tshuli

Allow .oa files to be uploaded in attachment fields

  • In a storage mode form, create an attachment field and submit an .oa file. The submission should be successful.
    • Go to the responses of the form, should be able to download all the attachments (including the .oa) as a zip
    • Go to the responses of the form, should be able to download the attachment by itself
  • In email mode form, create an attachment field and submit an .oa file. The submission should be successful.
    • Should successfully receive the attachment in your email.

@karrui
Copy link
Contributor Author

karrui commented Nov 17, 2020

Tests

@tshuli

Typescript route migrations

  • Upload an image for image field. Upload should work successfully and the image is shown on the public form.
  • Upload a logo for any form. Upload should work successfully and the logo is shown on the public form start page component.
  • Go to the data tab of any form with submissions. The submission count should be retrieved correctly.
  • In a storage mode form, go to the response tab and view responses. The responses table should populate correctly with the correct metadata.
    • Click into any submission response. The response answers should be decrypted properly.
    • Download all encrypted responses. Should download successfully.
    • Download encrypted responses in a date range. Should download successfully.

@karrui
Copy link
Contributor Author

karrui commented Nov 17, 2020

Tests

@tshuli

  • NRIC field validation should work as per normal -- invalid NRICs should still be invalid, valid NRICs should still be valid
  • Create a form with MyInfo fields. MyInfo prefill should work properly.
  • Transfer ownership email in modal is now more prominent (sample in style: make transfer ownership email more prominent #630)

@karrui
Copy link
Contributor Author

karrui commented Nov 17, 2020

Tests

@mantariksh

Run through on IE / Intranet

General run through

  • Create email mode form with all fields. Activate and submit form. Check that you can receive responses in your GSIB inbox.
  • Collaborators can be added in email mode.
  • Create storage mode form with all fields. Activate and submit form.
  • Response CSV can be downloaded for storage mode.
  • Responses can be viewed in Data tab in storage mode.
  • Attachments can be downloaded from Data tab in storage mode.
  • Submission IDs can be filtered in storage mode.
  • Collaborators can be added and their collaborating rights edited in storage mode.
  • Create form with MyInfo fields. Submit successfully

@tshuli
Copy link
Contributor

tshuli commented Nov 17, 2020

Tests

@mantariksh

Prefill

  • Create a form with short text field. Open in public view with the query params =, where comprises alphanumeric/hyphen/underscore characters. Check that the short text field is not prefilled with value1
  • In the db, check that allowPrefill is set to false for the short text field. set allowPrefill = true. Check that prefill now works.
  • Create myinfo field. Set allowPrefill = true for the myinfo field. Access form on public view and login to singpass. Check that prefill does not work on the myinfo field.
  • Check that prefilling works even without the hashbang in the url
  • Check that for non-hashbang url with two query params, upon redirection, the redirect path uses & to separate the query params instead of &amp;
  • Add radio button to the form. Set form logic to hide the short text field conditional on the radio button. Check that the short text field is prefilled when it is unhidden
  • Prefilled field value is cleared when field is hidden by logic at the point of submission

@mantariksh
Copy link
Contributor

mantariksh commented Nov 17, 2020

@karrui

Tests

SP/CP

  • Create an email mode form. Change the auth type to SingPass. Activate the form and submit it.
  • Create an email mode form. Change the auth type to CorpPass. Activate the form and submit it.
  • Create a storage mode form. Change the auth type to SingPass. Activate the form and submit it.
  • Create a storage mode form. Change the auth type to CorpPass. Activate the form and submit it.

MyInfo

  • Create and submit a form with all MyInfo fields. Ensure that when you receive the email response, all the disabled fields (and no others) have the [MyInfo] prefix.

@karrui
Copy link
Contributor Author

karrui commented Nov 17, 2020

@karrui

Tests

Specific IE11/ Intranet tests

  • Should be able to upload .oa files in storage mode forms.
  • Should be able to upload .oa files in email mode forms.
  • Should be able to receive form submissions with .oa files in intranet email for email mode submissions
  • Should be able to download form attachment of .oa files
  • Should be able to download form attachments of entire response (including .oa file attachment)
  • Transfer form ownership modal styling is correct like in style: make transfer ownership email more prominent #630

@karrui karrui merged commit c9fbbde into release Nov 18, 2020
@liangyuanruo liangyuanruo deleted the release-v4.46.0 branch January 14, 2021 06:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants