feat: Add option `schemaCacheTtl` for schema cache pulling as alternative to `enableSchemaHooks` #8436

dblythy · 2023-02-22T07:37:19Z

Pull Request

Report security issues confidentially.
Any contribution is under this license.
Link this pull request to an issue.

Issue

If a developer is not using enableSchemaHooks, the server will have to be restarted to fetch new schema data

Closes: #8229

Approach

Adds a conditional reload of schema if it hasn't been refreshed below databaseOptions.schemaCacheTTL

Tasks

Add tests

parse-github-assistant · 2023-02-22T07:37:21Z

I will reformat the title to use the proper commit message syntax.

parse-github-assistant · 2023-02-22T07:37:24Z

Thanks for opening this pull request!

mtrezza · 2023-02-22T07:41:36Z

I think it would be good to make this TTL refresh optional. So if the option is not set, there should be no refresh happening, which is the status quo. This way it's also less of breaking change.

codecov · 2023-02-22T23:49:50Z

Codecov Report

Base: 94.34% // Head: 94.09% // Decreases project coverage by -0.26% ⚠️

Coverage data is based on head (aa5443b) compared to base (e5d610e).
Patch coverage: 100.00% of modified lines in pull request are covered.

❗ Current head aa5443b differs from pull request most recent head 8c72deb. Consider uploading reports for the commit 8c72deb to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##            alpha    #8436      +/-   ##
==========================================
- Coverage   94.34%   94.09%   -0.26%     
==========================================
  Files         182      182              
  Lines       14405    14438      +33     
==========================================
- Hits        13590    13585       -5     
- Misses        815      853      +38

Impacted Files	Coverage Δ
src/Options/Definitions.js	`100.00% <ø> (ø)`
src/Options/index.js	`100.00% <ø> (ø)`
src/Adapters/Storage/Mongo/MongoStorageAdapter.js	`92.17% <100.00%> (+0.03%)`	⬆️
...dapters/Storage/Postgres/PostgresStorageAdapter.js	`95.73% <100.00%> (+<0.01%)`	⬆️
src/Config.js	`90.88% <100.00%> (+0.40%)`	⬆️
src/Controllers/SchemaController.js	`97.47% <100.00%> (+0.06%)`	⬆️
src/ParseServer.js	`92.20% <100.00%> (-0.41%)`	⬇️
src/Adapters/Cache/RedisCacheAdapter.js	`17.39% <0.00%> (-73.92%)`	⬇️
src/LiveQuery/ParseCloudCodePublisher.js	`84.21% <0.00%> (-15.79%)`	⬇️
src/Adapters/Files/GridFSBucketAdapter.js	`93.47% <0.00%> (-0.73%)`	⬇️
... and 4 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

mtrezza

What happens if schemaCacheTtl and enableSchemaHooks are set? Should both be enabled? There may be a use case for it, who knows. Or should it throw an error or a warning?

src/Adapters/Storage/Mongo/MongoStorageAdapter.js

dblythy · 2023-02-25T23:35:23Z

I think if they are both enabled:

Schema will update when change event is triggered
Schema will also pull every ttl

Perhaps if the network connection between the Parse Server and the database is inconsistent, change events might be missed.

into schema-pulling

mtrezza · 2023-02-26T12:30:39Z

Makes sense, so let's just allow both features to be configured independently without any warning

mtrezza

Adding doc

src/Options/index.js

src/Options/docs.js

src/Options/Definitions.js

Signed-off-by: Manuel <5673677+mtrezza@users.noreply.github.com>

mtrezza

Looks good, waiting for CI...

mtrezza

Doc rewording

src/Options/index.js

src/Options/docs.js

src/Options/Definitions.js

Signed-off-by: Manuel <5673677+mtrezza@users.noreply.github.com>

# [6.0.0-alpha.35](6.0.0-alpha.34...6.0.0-alpha.35) (2023-02-27) ### Features * Add option `schemaCacheTtl` for schema cache pulling as alternative to `enableSchemaHooks` ([#8436](#8436)) ([b3b76de](b3b76de))

parseplatformorg · 2023-02-27T00:57:07Z

🎉 This change has been released in version 6.0.0-alpha.35

# [6.1.0-beta.1](6.0.0...6.1.0-beta.1) (2023-03-02) ### Bug Fixes * Security upgrade jsonwebtoken to 9.0.0 ([#8420](#8420)) ([f5bfe45](f5bfe45)) ### Features * Add option `schemaCacheTtl` for schema cache pulling as alternative to `enableSchemaHooks` ([#8436](#8436)) ([b3b76de](b3b76de)) * Add Parse Server option `resetPasswordSuccessOnInvalidEmail` to choose success or error response on password reset with invalid email ([#7551](#7551)) ([e5d610e](e5d610e)) * Deprecate LiveQuery `fields` option in favor of `keys` for semantic consistency ([#8388](#8388)) ([a49e323](a49e323))

parseplatformorg · 2023-03-02T10:54:58Z

🎉 This change has been released in version 6.1.0-beta.1

# [6.1.0-alpha.1](6.0.0...6.1.0-alpha.1) (2023-03-03) ### Bug Fixes * Security upgrade jsonwebtoken to 9.0.0 ([#8420](#8420)) ([f5bfe45](f5bfe45)) ### Features * Add option `schemaCacheTtl` for schema cache pulling as alternative to `enableSchemaHooks` ([#8436](#8436)) ([b3b76de](b3b76de)) * Add Parse Server option `resetPasswordSuccessOnInvalidEmail` to choose success or error response on password reset with invalid email ([#7551](#7551)) ([e5d610e](e5d610e)) * Deprecate LiveQuery `fields` option in favor of `keys` for semantic consistency ([#8388](#8388)) ([a49e323](a49e323)) * Export `AuthAdapter` to make it available for extension with custom authentication adapters ([#8443](#8443)) ([40c1961](40c1961))

parseplatformorg · 2023-03-03T16:53:48Z

🎉 This change has been released in version 6.1.0-alpha.1

# [6.1.0](6.0.0...6.1.0) (2023-05-01) ### Bug Fixes * LiveQuery can return incorrectly formatted date ([#8456](#8456)) ([4ce135a](4ce135a)) * Nested date is incorrectly decoded as empty object `{}` when fetching a Parse Object ([#8446](#8446)) ([22d2446](22d2446)) * Parameters missing in `afterFind` trigger of authentication adapters ([#8458](#8458)) ([ce34747](ce34747)) * Rate limiting across multiple servers via Redis not working ([#8469](#8469)) ([d9e347d](d9e347d)) * Security upgrade jsonwebtoken to 9.0.0 ([#8420](#8420)) ([f5bfe45](f5bfe45)) ### Features * Add `afterFind` trigger to authentication adapters ([#8444](#8444)) ([c793bb8](c793bb8)) * Add option `schemaCacheTtl` for schema cache pulling as alternative to `enableSchemaHooks` ([#8436](#8436)) ([b3b76de](b3b76de)) * Add Parse Server option `resetPasswordSuccessOnInvalidEmail` to choose success or error response on password reset with invalid email ([#7551](#7551)) ([e5d610e](e5d610e)) * Add rate limiting across multiple servers via Redis ([#8394](#8394)) ([34833e4](34833e4)) * Allow multiple origins for header `Access-Control-Allow-Origin` ([#8517](#8517)) ([4f15539](4f15539)) * Deprecate LiveQuery `fields` option in favor of `keys` for semantic consistency ([#8388](#8388)) ([a49e323](a49e323)) * Export `AuthAdapter` to make it available for extension with custom authentication adapters ([#8443](#8443)) ([40c1961](40c1961))

parseplatformorg · 2023-05-01T21:52:34Z

🎉 This change has been released in version 6.1.0

commit 1506273 Author: semantic-release-bot <semantic-release-bot@martynus.net> Date: Sat May 20 23:24:03 2023 +0000 chore(release): 6.2.0 [skip ci] # [6.2.0](parse-community/parse-server@6.1.0...6.2.0) (2023-05-20) ### Features * Add new Parse Server option `fileUpload.fileExtensions` to restrict file upload by file extension; this fixes a security vulnerability in which a phishing attack could be performed using an uploaded HTML file; by default the new option only allows file extensions matching the regex pattern `^[^hH][^tT][^mM][^lL]?$`, which excludes HTML files; if your app currently depends on uploading files with HTML file extensions then this may be a breaking change and you could allow HTML file upload by setting the option to `['.*']` ([parse-community#8538](parse-community#8538)) ([a318e7b](parse-community@a318e7b)) commit a318e7b Author: Manuel <5673677+mtrezza@users.noreply.github.com> Date: Sun May 21 01:23:00 2023 +0200 feat: Add new Parse Server option `fileUpload.fileExtensions` to restrict file upload by file extension; this fixes a security vulnerability in which a phishing attack could be performed using an uploaded HTML file; by default the new option only allows file extensions matching the regex pattern `^[^hH][^tT][^mM][^lL]?$`, which excludes HTML files; if your app currently depends on uploading files with HTML file extensions then this may be a breaking change and you could allow HTML file upload by setting the option to `['.*']` (parse-community#8538) commit 832702d Author: semantic-release-bot <semantic-release-bot@martynus.net> Date: Mon May 1 21:50:23 2023 +0000 chore(release): 6.1.0 [skip ci] # [6.1.0](parse-community/parse-server@6.0.0...6.1.0) (2023-05-01) ### Bug Fixes * LiveQuery can return incorrectly formatted date ([parse-community#8456](parse-community#8456)) ([4ce135a](parse-community@4ce135a)) * Nested date is incorrectly decoded as empty object `{}` when fetching a Parse Object ([parse-community#8446](parse-community#8446)) ([22d2446](parse-community@22d2446)) * Parameters missing in `afterFind` trigger of authentication adapters ([parse-community#8458](parse-community#8458)) ([ce34747](parse-community@ce34747)) * Rate limiting across multiple servers via Redis not working ([parse-community#8469](parse-community#8469)) ([d9e347d](parse-community@d9e347d)) * Security upgrade jsonwebtoken to 9.0.0 ([parse-community#8420](parse-community#8420)) ([f5bfe45](parse-community@f5bfe45)) ### Features * Add `afterFind` trigger to authentication adapters ([parse-community#8444](parse-community#8444)) ([c793bb8](parse-community@c793bb8)) * Add option `schemaCacheTtl` for schema cache pulling as alternative to `enableSchemaHooks` ([parse-community#8436](parse-community#8436)) ([b3b76de](parse-community@b3b76de)) * Add Parse Server option `resetPasswordSuccessOnInvalidEmail` to choose success or error response on password reset with invalid email ([parse-community#7551](parse-community#7551)) ([e5d610e](parse-community@e5d610e)) * Add rate limiting across multiple servers via Redis ([parse-community#8394](parse-community#8394)) ([34833e4](parse-community@34833e4)) * Allow multiple origins for header `Access-Control-Allow-Origin` ([parse-community#8517](parse-community#8517)) ([4f15539](parse-community@4f15539)) * Deprecate LiveQuery `fields` option in favor of `keys` for semantic consistency ([parse-community#8388](parse-community#8388)) ([a49e323](parse-community@a49e323)) * Export `AuthAdapter` to make it available for extension with custom authentication adapters ([parse-community#8443](parse-community#8443)) ([40c1961](parse-community@40c1961)) commit 18b63d1 Merge: f7eee19 f59d46c Author: Manuel <5673677+mtrezza@users.noreply.github.com> Date: Mon May 1 23:49:22 2023 +0200 build: Release (parse-community#8526)

dblythy added 2 commits February 22, 2023 18:30

feat: re-add schema pulling

5052f1a

Update SchemaController.js

e90d568

parse-github-assistant bot changed the title ~~feat: re-add schema pulling~~ feat: Re-add schema pulling Feb 22, 2023

Update MongoStorageAdapter.js

b22148c

dblythy added 2 commits February 22, 2023 18:47

refactor

c4fe445

refactor

73c2e34

dblythy added 3 commits February 23, 2023 11:07

Update SchemaPerformance.spec.js

b29bcd0

postgres

dfd581b

refactor

61b64b3

mtrezza requested changes Feb 25, 2023

View reviewed changes

src/Adapters/Storage/Mongo/MongoStorageAdapter.js Outdated Show resolved Hide resolved

dblythy added 3 commits February 26, 2023 10:35

Merge branch 'alpha' into schema-pulling

8fbc92a

camel case

1a85796

Merge branch 'schema-pulling' of https://github.com/dblythy/parse-server

e3c6c2c

into schema-pulling

mtrezza requested changes Feb 26, 2023

View reviewed changes

src/Options/index.js Outdated Show resolved Hide resolved

src/Options/docs.js Outdated Show resolved Hide resolved

src/Options/Definitions.js Outdated Show resolved Hide resolved

mtrezza added 3 commits February 26, 2023 13:53

Update src/Options/docs.js

963dcdd

Signed-off-by: Manuel <5673677+mtrezza@users.noreply.github.com>

Update src/Options/index.js

9240ce5

Signed-off-by: Manuel <5673677+mtrezza@users.noreply.github.com>

Update src/Options/Definitions.js

b04f294

Signed-off-by: Manuel <5673677+mtrezza@users.noreply.github.com>

mtrezza previously approved these changes Feb 26, 2023

View reviewed changes

mtrezza reviewed Feb 26, 2023

View reviewed changes

src/Options/index.js Outdated Show resolved Hide resolved

src/Options/docs.js Outdated Show resolved Hide resolved

src/Options/Definitions.js Outdated Show resolved Hide resolved

mtrezza dismissed their stale review via bbfbe8b February 26, 2023 12:57

mtrezza added 3 commits February 26, 2023 13:57

Update src/Options/index.js

bbfbe8b

Signed-off-by: Manuel <5673677+mtrezza@users.noreply.github.com>

Update src/Options/docs.js

cd21815

Signed-off-by: Manuel <5673677+mtrezza@users.noreply.github.com>

Update src/Options/Definitions.js

8c72deb

Signed-off-by: Manuel <5673677+mtrezza@users.noreply.github.com>

mtrezza approved these changes Feb 26, 2023

View reviewed changes

mtrezza changed the title ~~feat: Re-add schema pulling~~ feat: Add schema cache pulling as alternative to schema refresh via enableSchemaHooks Feb 26, 2023

mtrezza changed the title ~~feat: Add schema cache pulling as alternative to schema refresh via enableSchemaHooks~~ feat: Add schema cache pulling as alternative to schema refresh via the option enableSchemaHooks Feb 26, 2023

mtrezza changed the title ~~feat: Add schema cache pulling as alternative to schema refresh via the option enableSchemaHooks~~ feat: Add option schemaCacheTtl for schema cache pulling as alternative to enableSchemaHooks Feb 26, 2023

mtrezza merged commit b3b76de into parse-community:alpha Feb 27, 2023

parseplatformorg added the state:released-alpha Released as alpha version label Feb 27, 2023

parseplatformorg added the state:released-beta Released as beta version label Mar 2, 2023

dblythy deleted the schema-pulling branch March 2, 2023 12:08

This was referenced Mar 8, 2023

Remove unsupported db options from mongo connection SynapLearning/parse-server#2

Merged

Database options connect fix #8470

Closed

dblythy mentioned this pull request Mar 9, 2023

GridFS fails with enableSchemaHooks is not supported #8466

Closed

4 tasks

parseplatformorg added the state:released Released as stable version label May 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add option `schemaCacheTtl` for schema cache pulling as alternative to `enableSchemaHooks` #8436

feat: Add option `schemaCacheTtl` for schema cache pulling as alternative to `enableSchemaHooks` #8436

dblythy commented Feb 22, 2023

parse-github-assistant bot commented Feb 22, 2023

parse-github-assistant bot commented Feb 22, 2023

mtrezza commented Feb 22, 2023 •

edited

codecov bot commented Feb 22, 2023 •

edited

mtrezza left a comment

dblythy commented Feb 25, 2023

mtrezza commented Feb 26, 2023 •

edited

mtrezza left a comment

mtrezza left a comment

mtrezza left a comment

parseplatformorg commented Feb 27, 2023

parseplatformorg commented Mar 2, 2023

parseplatformorg commented Mar 3, 2023

parseplatformorg commented May 1, 2023

feat: Add option schemaCacheTtl for schema cache pulling as alternative to enableSchemaHooks #8436

feat: Add option schemaCacheTtl for schema cache pulling as alternative to enableSchemaHooks #8436

Conversation

dblythy commented Feb 22, 2023

Pull Request

Issue

Approach

Tasks

parse-github-assistant bot commented Feb 22, 2023

parse-github-assistant bot commented Feb 22, 2023

Thanks for opening this pull request!

mtrezza commented Feb 22, 2023 • edited

codecov bot commented Feb 22, 2023 • edited

Codecov Report

mtrezza left a comment

Choose a reason for hiding this comment

dblythy commented Feb 25, 2023

mtrezza commented Feb 26, 2023 • edited

mtrezza left a comment

Choose a reason for hiding this comment

mtrezza left a comment

Choose a reason for hiding this comment

mtrezza left a comment

Choose a reason for hiding this comment

parseplatformorg commented Feb 27, 2023

parseplatformorg commented Mar 2, 2023

parseplatformorg commented Mar 3, 2023

parseplatformorg commented May 1, 2023

feat: Add option `schemaCacheTtl` for schema cache pulling as alternative to `enableSchemaHooks` #8436

feat: Add option `schemaCacheTtl` for schema cache pulling as alternative to `enableSchemaHooks` #8436

mtrezza commented Feb 22, 2023 •

edited

codecov bot commented Feb 22, 2023 •

edited

mtrezza commented Feb 26, 2023 •

edited