feat: Add compatibility for MongoDB Atlas Serverless and AWS Amazon DocumentDB with collation options `enableCollationCaseComparison`, `convertEmailToLowercase`, `convertUsernameToLowercase` #8805

mattia1208 · 2023-11-10T16:01:39Z

Pull Request

Report security issues confidentially.
Any contribution is under this license.
Link this pull request to an issue.

Issue

Add options to support MongoDB Serverless and AWS DocumentDB which currently does not have collation support.

Closes: #7937

Approach

Add 3 abstract options disableCollation, transformEmailToLowerCase and transformUsernameToLowerCase to allow a developer to disable default collations of Parse Server and transformEmailToLowerCase to keep clean emails in the database and transformUsernameToLowerCase to keep clean usernames in the database.

Tasks

Add tests
Add changes to documentation (guides, repository pages, code comments)
Add security check

…ameToLowerCase

parse-github-assistant · 2023-11-10T16:01:42Z

I will reformat the title to use the proper commit message syntax.

parse-github-assistant · 2023-11-10T16:01:45Z

Thanks for opening this pull request!

mattia1208 · 2023-11-10T16:03:49Z

This PR is based on:

#8042
Original author: @Moumouls

And

#8767
Original author: @b10f

src/Options/index.js

mtrezza · 2023-11-11T13:45:24Z

@mattia1208 Thanks for picking up this PR; just a few cosmetic things and this should be good to merge.

Co-authored-by: Manuel <5673677+mtrezza@users.noreply.github.com> Signed-off-by: Mattia Faraci <82614156+mattia1208@users.noreply.github.com>

mtrezza

Could you run npm run definitions to update the definition files? There also seem to be some lint errors where the new option names are not changed in code.

mattia1208 · 2023-11-11T15:59:11Z

@mtrezza sorry, done!

mtrezza

Looks good; waiting for CI to pass...

mtrezza · 2023-11-11T19:58:42Z

CI fails with:

1) DatabaseController convertEmailToLowercase should not find a case insensitive user by email with convertEmailToLowercase
  - Error: bad or missing username

2) DatabaseController enableCollationCaseComparison should not create insensitive indexes with enableCollationCaseComparison
  - Expected object not to have properties
      case_insensitive_username: Object({ username: 1 })
      case_insensitive_email: Object({ email: 1 })

3) DatabaseController enableCollationCaseComparison should force caseInsensitive to false with enableCollationCaseComparison option
  - Expected true to equal false.

Did the tests pass locally for you?

mattia1208 · 2023-11-12T00:09:24Z

@mtrezza now all tests locally have passed

mtrezza · 2023-11-12T00:38:18Z

Restarted CI...

codecov · 2023-11-12T00:44:50Z

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (80b987d) 94.29% compared to head (71f68b6) 94.31%.

❗ Current head 71f68b6 differs from pull request most recent head c8b6129. Consider uploading reports for the commit c8b6129 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##            alpha    #8805      +/-   ##
==========================================
+ Coverage   94.29%   94.31%   +0.01%     
==========================================
  Files         186      186              
  Lines       14785    14798      +13     
==========================================
+ Hits        13941    13956      +15     
+ Misses        844      842       -2

Files	Coverage Δ
src/Options/Definitions.js	`100.00% <ø> (ø)`
src/Options/index.js	`100.00% <ø> (ø)`
src/Controllers/DatabaseController.js	`94.08% <78.94%> (+0.10%)`	⬆️

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

mtrezza · 2023-11-12T01:28:15Z

CI passed; @Moumouls, @b10f do you want to review this once more before we merge?

b10f · 2023-11-12T10:03:17Z

CI passed; @Moumouls, @b10f do you want to review this once more before we merge?

I'm good

# [6.4.0-alpha.8](6.4.0-alpha.7...6.4.0-alpha.8) (2023-11-13) ### Features * Add compatibility for MongoDB Atlas Serverless and AWS Amazon DocumentDB with collation options `enableCollationCaseComparison`, `transformEmailToLowercase`, `transformUsernameToLowercase` ([#8805](#8805)) ([09fbeeb](09fbeeb))

parseplatformorg · 2023-11-13T22:34:07Z

🎉 This change has been released in version 6.4.0-alpha.8

# [6.5.0-beta.1](6.4.0...6.5.0-beta.1) (2023-11-16) ### Bug Fixes * Context not passed to Cloud Code Trigger `beforeFind` when using `Parse.Query.include` ([#8765](#8765)) ([7d32d89](7d32d89)) * Parse Server option `fileUpload.fileExtensions` fails to determine file extension if filename contains multiple dots ([#8754](#8754)) ([3d6d50e](3d6d50e)) * Security bump @babel/traverse from 7.20.5 to 7.23.2 ([#8777](#8777)) ([2d6b3d1](2d6b3d1)) * Security upgrade graphql from 16.6.0 to 16.8.1 ([#8758](#8758)) ([71dfd8a](71dfd8a)) ### Features * Add `$setOnInsert` operator to `Parse.Server.database.update` ([#8791](#8791)) ([f630a45](f630a45)) * Add compatibility for MongoDB Atlas Serverless and AWS Amazon DocumentDB with collation options `enableCollationCaseComparison`, `transformEmailToLowercase`, `transformUsernameToLowercase` ([#8805](#8805)) ([09fbeeb](09fbeeb)) * Add context to Cloud Code Triggers `beforeLogin` and `afterLogin` ([#8724](#8724)) ([a9c34ef](a9c34ef)) * Allow setting `createdAt` and `updatedAt` during `Parse.Object` creation with maintenance key ([#8696](#8696)) ([77bbfb3](77bbfb3))

parseplatformorg · 2023-11-16T15:58:42Z

🎉 This change has been released in version 6.5.0-beta.1

# [6.5.0-alpha.1](6.4.0...6.5.0-alpha.1) (2023-11-18) ### Bug Fixes * Context not passed to Cloud Code Trigger `beforeFind` when using `Parse.Query.include` ([#8765](#8765)) ([7d32d89](7d32d89)) * Parse Server option `fileUpload.fileExtensions` fails to determine file extension if filename contains multiple dots ([#8754](#8754)) ([3d6d50e](3d6d50e)) * Security bump @babel/traverse from 7.20.5 to 7.23.2 ([#8777](#8777)) ([2d6b3d1](2d6b3d1)) * Security upgrade graphql from 16.6.0 to 16.8.1 ([#8758](#8758)) ([71dfd8a](71dfd8a)) ### Features * Add `$setOnInsert` operator to `Parse.Server.database.update` ([#8791](#8791)) ([f630a45](f630a45)) * Add compatibility for MongoDB Atlas Serverless and AWS Amazon DocumentDB with collation options `enableCollationCaseComparison`, `transformEmailToLowercase`, `transformUsernameToLowercase` ([#8805](#8805)) ([09fbeeb](09fbeeb)) * Add context to Cloud Code Triggers `beforeLogin` and `afterLogin` ([#8724](#8724)) ([a9c34ef](a9c34ef)) * Allow setting `createdAt` and `updatedAt` during `Parse.Object` creation with maintenance key ([#8696](#8696)) ([77bbfb3](77bbfb3)) * Upgrade Parse Server Push Adapter to 5.0.2 ([#8813](#8813)) ([6ef1986](6ef1986))

parseplatformorg · 2023-11-18T13:55:19Z

🎉 This change has been released in version 6.5.0-alpha.1

# [6.5.0](6.4.0...6.5.0) (2024-03-01) ### Bug Fixes * Context not passed to Cloud Code Trigger `beforeFind` when using `Parse.Query.include` ([#8765](#8765)) ([7d32d89](7d32d89)) * Improve PostgreSQL injection detection; fixes security vulnerability [GHSA-6927-3vr9-fxf2](GHSA-6927-3vr9-fxf2) which affects Parse Server deployments using a Postgres database ([#8960](#8960)) ([a6e6549](a6e6549)) * Parse Server option `fileUpload.fileExtensions` fails to determine file extension if filename contains multiple dots ([#8754](#8754)) ([3d6d50e](3d6d50e)) * Security bump @babel/traverse from 7.20.5 to 7.23.2 ([#8777](#8777)) ([2d6b3d1](2d6b3d1)) * Security upgrade graphql from 16.6.0 to 16.8.1 ([#8758](#8758)) ([71dfd8a](71dfd8a)) ### Features * Add `$setOnInsert` operator to `Parse.Server.database.update` ([#8791](#8791)) ([f630a45](f630a45)) * Add compatibility for MongoDB Atlas Serverless and AWS Amazon DocumentDB with collation options `enableCollationCaseComparison`, `transformEmailToLowercase`, `transformUsernameToLowercase` ([#8805](#8805)) ([09fbeeb](09fbeeb)) * Add context to Cloud Code Triggers `beforeLogin` and `afterLogin` ([#8724](#8724)) ([a9c34ef](a9c34ef)) * Allow setting `createdAt` and `updatedAt` during `Parse.Object` creation with maintenance key ([#8696](#8696)) ([77bbfb3](77bbfb3)) * Upgrade Parse Server Push Adapter to 5.0.2 ([#8813](#8813)) ([6ef1986](6ef1986)) ### Performance Improvements * Improved IP validation performance for `masterKeyIPs`, `maintenanceKeyIPs` ([#8510](#8510)) ([b87daba](b87daba))

parseplatformorg · 2024-03-01T16:07:07Z

🎉 This change has been released in version 6.5.0

# [7.0.0](6.4.0...7.0.0) (2024-03-19) ### Bug Fixes * CacheAdapter does not connect when using a CacheAdapter with a JSON config ([#8633](#8633)) ([720d24e](720d24e)) * Conditional email verification not working in some cases if `verifyUserEmails`, `preventLoginWithUnverifiedEmail` set to functions ([#8838](#8838)) ([8e7a6b1](8e7a6b1)) * Context not passed to Cloud Code Trigger `beforeFind` when using `Parse.Query.include` ([#8765](#8765)) ([7d32d89](7d32d89)) * Deny request if master key is not set in Parse Server option `masterKeyIps` regardless of ACL and CLP ([#8957](#8957)) ([a7b5b38](a7b5b38)) * Docker image not published to Docker Hub on new release ([#8905](#8905)) ([a2ac8d1](a2ac8d1)) * Docker version releases by removing arm/v6 and arm/v7 support ([#8976](#8976)) ([1f62dd0](1f62dd0)) * GraphQL file upload fails in case of use of pointer or relation ([#8721](#8721)) ([1aba638](1aba638)) * Improve PostgreSQL injection detection; fixes security vulnerability [GHSA-6927-3vr9-fxf2](GHSA-6927-3vr9-fxf2) which affects Parse Server deployments using a Postgres database ([#8961](#8961)) ([cbefe77](cbefe77)) * Incomplete user object in `verifyEmail` function if both username and email are changed ([#8889](#8889)) ([1eb95ae](1eb95ae)) * Parse Server option `emailVerifyTokenReuseIfValid: true` generates new token on every email verification request ([#8885](#8885)) ([0023ce4](0023ce4)) * Parse Server option `fileExtensions` default value rejects file extensions that are less than 3 or more than 4 characters long ([#8699](#8699)) ([2760381](2760381)) * Parse Server option `fileUpload.fileExtensions` fails to determine file extension if filename contains multiple dots ([#8754](#8754)) ([3d6d50e](3d6d50e)) * Security bump @babel/traverse from 7.20.5 to 7.23.2 ([#8777](#8777)) ([2d6b3d1](2d6b3d1)) * Security upgrade graphql from 16.6.0 to 16.8.1 ([#8758](#8758)) ([71dfd8a](71dfd8a)) * Server crashes on invalid Cloud Function or Cloud Job name; fixes security vulnerability [GHSA-6hh7-46r2-vf29](GHSA-6hh7-46r2-vf29) ([#9024](#9024)) ([9f6e342](9f6e342)) * Server crashes when receiving an array of `Parse.Pointer` in the request body ([#8784](#8784)) ([66e3603](66e3603)) * Username is `undefined` in email verification link on email change ([#8887](#8887)) ([e315c13](e315c13)) ### Features * Add `$setOnInsert` operator to `Parse.Server.database.update` ([#8791](#8791)) ([f630a45](f630a45)) * Add `installationId` to arguments for `verifyUserEmails`, `preventLoginWithUnverifiedEmail` ([#8836](#8836)) ([a22dbe1](a22dbe1)) * Add `installationId`, `ip`, `resendRequest` to arguments passed to `verifyUserEmails` on verification email request ([#8873](#8873)) ([8adcbee](8adcbee)) * Add `Parse.User` as function parameter to Parse Server options `verifyUserEmails`, `preventLoginWithUnverifiedEmail` on login ([#8850](#8850)) ([972f630](972f630)) * Add compatibility for MongoDB Atlas Serverless and AWS Amazon DocumentDB with collation options `enableCollationCaseComparison`, `transformEmailToLowercase`, `transformUsernameToLowercase` ([#8805](#8805)) ([09fbeeb](09fbeeb)) * Add context to Cloud Code Triggers `beforeLogin` and `afterLogin` ([#8724](#8724)) ([a9c34ef](a9c34ef)) * Add password validation via POST request for user with unverified email using master key and option `ignoreEmailVerification` ([#8895](#8895)) ([633a9d2](633a9d2)) * Add support for MongoDB 7 ([#8761](#8761)) ([3de8494](3de8494)) * Add support for MongoDB query comment ([#8928](#8928)) ([2170962](2170962)) * Add support for Node 20, drop support for Node 14, 16 ([#8907](#8907)) ([ced4872](ced4872)) * Add support for Postgres 16 ([#8898](#8898)) ([99489b2](99489b2)) * Allow `Parse.Session.current` on expired session token instead of throwing error ([#8722](#8722)) ([f9dde4a](f9dde4a)) * Allow setting `createdAt` and `updatedAt` during `Parse.Object` creation with maintenance key ([#8696](#8696)) ([77bbfb3](77bbfb3)) * Deprecation DEPPS5: Config option `allowClientClassCreation` defaults to `false` ([#8849](#8849)) ([29624e0](29624e0)) * Deprecation DEPPS6: Authentication adapters disabled by default ([#8858](#8858)) ([0cf58eb](0cf58eb)) * Deprecation DEPPS7: Remove deprecated Cloud Code file trigger syntax ([#8855](#8855)) ([4e6a375](4e6a375)) * Deprecation DEPPS8: Parse Server option `allowExpiredAuthDataToken` defaults to `false` ([#8860](#8860)) ([e29845f](e29845f)) * Deprecation DEPPS9: LiveQuery `fields` option is renamed to `keys` ([#8852](#8852)) ([38983e8](38983e8)) * Node process exits with error code 1 on uncaught exception to allow custom uncaught exception handling ([#8894](#8894)) ([70c280c](70c280c)) * Switch GraphQL server from Yoga v2 to Apollo v4 ([#8959](#8959)) ([105ae7c](105ae7c)) * Upgrade Parse Server Push Adapter to 5.0.2 ([#8813](#8813)) ([6ef1986](6ef1986)) * Upgrade to Parse JS SDK 5 ([#9022](#9022)) ([ad4aa83](ad4aa83)) ### Performance Improvements * Improved IP validation performance for `masterKeyIPs`, `maintenanceKeyIPs` ([#8510](#8510)) ([b87daba](b87daba)) ### BREAKING CHANGES * The Parse Server option `allowClientClassCreation` defaults to `false`. ([29624e0](29624e0)) * A request using the master key will now be rejected as unauthorized if the IP from which the request originates is not set in the Parse Server option `masterKeyIps`, even if the request does not require the master key permission, for example for a public object in a public class class. ([a7b5b38](a7b5b38)) * Node process now exits with code 1 on uncaught exceptions, enabling custom handlers that were blocked by Parse Server's default behavior of re-throwing errors. This change may lead to automatic process restarts by the environment, unlike before. ([70c280c](70c280c)) * Authentication adapters are disabled by default; to use an authentication adapter it needs to be explicitly enabled in the Parse Server authentication adapter option `auth.<provider>.enabled: true` ([0cf58eb](0cf58eb)) * Parse Server option `allowExpiredAuthDataToken` defaults to `false`; a 3rd party authentication token will be validated every time the user tries to log in and the login will fail if the token has expired; the effect of this change may differ for different authentication adapters, depending on the token lifetime and the token refresh logic of the adapter ([e29845f](e29845f)) * LiveQuery `fields` option is renamed to `keys` ([38983e8](38983e8)) * Cloud Code file trigger syntax has been aligned with object trigger syntax, for example `Parse.Cloud.beforeDeleteFile'` has been changed to `Parse.Cloud.beforeDelete(Parse.File, (request) => {})'` ([4e6a375](4e6a375)) * Removes support for Node 14 and 16 ([ced4872](ced4872)) * Removes support for Postgres 11 and 12 ([99489b2](99489b2)) * The `Parse.User` passed as argument if `verifyUserEmails` is set to a function is renamed from `user` to `object` for consistency with invocations of `verifyUserEmails` on signup or login; the user object is not a plain JavaScript object anymore but an instance of `Parse.User` ([8adcbee](8adcbee)) * `Parse.Session.current()` no longer throws an error if the session token is expired, but instead returns the session token with its expiration date to allow checking its validity ([f9dde4a](f9dde4a)) * `Parse.Query` no longer supports the BSON type `code`; although this feature was never officially documented, its removal is announced as a breaking change to protect deployments where it might be in use. ([3de8494](3de8494))

parseplatformorg · 2024-03-19T20:50:35Z

🎉 This change has been released in version 7.0.0

Moumouls · 2024-03-22T21:51:30Z

Ooohhh nice I didn't see this notification, thanks @mattia1208 for your work here !

Mongo serverless is really nice and can help a lot to adopt parse server, with super cheap deployment cost.

feat: disable collation, transformEmailToLowerCase and transformUsern…

1950772

…ameToLowerCase

parse-github-assistant bot changed the title ~~feat: disable collation, transformEmailToLowerCase and transformUsernameToLowerCase~~ feat: Disable collation, transformEmailToLowerCase and transformUsernameToLowerCase Nov 10, 2023

mattia1208 mentioned this pull request Nov 10, 2023

feat: Add options to disable collation and transform email and username to lower case #8767

Closed

3 tasks

mtrezza requested changes Nov 11, 2023

View reviewed changes

src/Options/index.js Outdated Show resolved Hide resolved

src/Options/index.js Outdated Show resolved Hide resolved

mattia1208 and others added 2 commits November 11, 2023 15:42

Update src/Options/index.js

f292d28

Co-authored-by: Manuel <5673677+mtrezza@users.noreply.github.com> Signed-off-by: Mattia Faraci <82614156+mattia1208@users.noreply.github.com>

Update src/Options/index.js

622dffa

Co-authored-by: Manuel <5673677+mtrezza@users.noreply.github.com> Signed-off-by: Mattia Faraci <82614156+mattia1208@users.noreply.github.com>

mattia1208 requested a review from mtrezza November 11, 2023 14:47

mtrezza requested changes Nov 11, 2023

View reviewed changes

Change options name

7bff03c

mattia1208 requested a review from mtrezza November 11, 2023 16:41

mtrezza previously approved these changes Nov 11, 2023

View reviewed changes

Fix enableCollationCaseComparison, caseInsensitive and test

c8b6129

mattia1208 dismissed mtrezza’s stale review via c8b6129 November 12, 2023 00:08

mattia1208 requested a review from mtrezza November 12, 2023 00:09

mtrezza merged commit 09fbeeb into parse-community:alpha Nov 13, 2023
25 of 26 checks passed

parseplatformorg added the state:released-alpha Released as alpha version label Nov 13, 2023

mtrezza mentioned this pull request Nov 14, 2023

feat: Add options to disable case insensitivity and force email and username to lower case #8042

Closed

6 tasks

parseplatformorg added the state:released-beta Released as beta version label Nov 16, 2023

parseplatformorg added the state:released-release-6.x.x label Mar 1, 2024

parseplatformorg added the state:released Released as stable version label Mar 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add compatibility for MongoDB Atlas Serverless and AWS Amazon DocumentDB with collation options `enableCollationCaseComparison`, `convertEmailToLowercase`, `convertUsernameToLowercase` #8805

feat: Add compatibility for MongoDB Atlas Serverless and AWS Amazon DocumentDB with collation options `enableCollationCaseComparison`, `convertEmailToLowercase`, `convertUsernameToLowercase` #8805

mattia1208 commented Nov 10, 2023 •

edited

parse-github-assistant bot commented Nov 10, 2023

parse-github-assistant bot commented Nov 10, 2023

mattia1208 commented Nov 10, 2023

mtrezza commented Nov 11, 2023

mtrezza left a comment •

edited

mattia1208 commented Nov 11, 2023

mtrezza left a comment

mtrezza commented Nov 11, 2023

mattia1208 commented Nov 12, 2023

mtrezza commented Nov 12, 2023

codecov bot commented Nov 12, 2023 •

edited

mtrezza commented Nov 12, 2023

b10f commented Nov 12, 2023

parseplatformorg commented Nov 13, 2023

parseplatformorg commented Nov 16, 2023

parseplatformorg commented Nov 18, 2023

parseplatformorg commented Mar 1, 2024

parseplatformorg commented Mar 19, 2024

Moumouls commented Mar 22, 2024

feat: Add compatibility for MongoDB Atlas Serverless and AWS Amazon DocumentDB with collation options enableCollationCaseComparison, convertEmailToLowercase, convertUsernameToLowercase #8805

feat: Add compatibility for MongoDB Atlas Serverless and AWS Amazon DocumentDB with collation options enableCollationCaseComparison, convertEmailToLowercase, convertUsernameToLowercase #8805

Conversation

mattia1208 commented Nov 10, 2023 • edited

Pull Request

Issue

Approach

Tasks

parse-github-assistant bot commented Nov 10, 2023

parse-github-assistant bot commented Nov 10, 2023

Thanks for opening this pull request!

mattia1208 commented Nov 10, 2023

mtrezza commented Nov 11, 2023

mtrezza left a comment • edited

Choose a reason for hiding this comment

mattia1208 commented Nov 11, 2023

mtrezza left a comment

Choose a reason for hiding this comment

mtrezza commented Nov 11, 2023

mattia1208 commented Nov 12, 2023

mtrezza commented Nov 12, 2023

codecov bot commented Nov 12, 2023 • edited

Codecov Report

mtrezza commented Nov 12, 2023

b10f commented Nov 12, 2023

parseplatformorg commented Nov 13, 2023

parseplatformorg commented Nov 16, 2023

parseplatformorg commented Nov 18, 2023

parseplatformorg commented Mar 1, 2024

parseplatformorg commented Mar 19, 2024

Moumouls commented Mar 22, 2024

feat: Add compatibility for MongoDB Atlas Serverless and AWS Amazon DocumentDB with collation options `enableCollationCaseComparison`, `convertEmailToLowercase`, `convertUsernameToLowercase` #8805

feat: Add compatibility for MongoDB Atlas Serverless and AWS Amazon DocumentDB with collation options `enableCollationCaseComparison`, `convertEmailToLowercase`, `convertUsernameToLowercase` #8805

mattia1208 commented Nov 10, 2023 •

edited

mtrezza left a comment •

edited

codecov bot commented Nov 12, 2023 •

edited