Closed
Description
New Feature / Enhancement Checklist
- I am not disclosing a vulnerability.I am not just asking a question.I have searched through existing issues.
Current Limitation
When running a Parse Job, the params is saved in the _JobStatus collection by default in MongoDB thus limiting the size of the params to the MongoDB BSON limit of 16Mb : https://www.mongodb.com/docs/manual/reference/limits/#:~:text=The%20maximum%20BSON%20document%20size,MongoDB%20provides%20the%20GridFS%20API.
Feature / Enhancement Description
We would need to modify this code :
Two ways of doing it :
- We could pass a context while launching the Parse Job telling that we should not save the params.
- We could measure the size of the params and therefore truncate the params inside the JobStatus document.
Example Use Case
For example I want to launch a job with 32Mb of data to analyse, I will launch the job but explicitly telling Parse to not save the params in database.
Alternatives / Workarounds
No easy workaround
3rd Party References
POSTing more than 16Mb of data is quite common everywhere :)
Activity
parse-github-assistant commentedon Nov 30, 2022
Thanks for opening this issue!
mtrezza commentedon Nov 30, 2022
_JobStatus
is an internal collection and therefore not supposed to be used for anything else than supporting features of Parse Server. Any change to that collection would not constitute a breaking change, so one can not rely on it, what is logged, how it's logged, or whether anything is logged at all.In that context, the params stored cannot have any operational value, they could have at most analytical or troubleshooting value. In addition, storing operational data in an internal collection is not supposed to happen and we should probably implement a guideline for reviewers about that. Operational data may be sensitive, require data protection considerations, auditing, or other things that are problematic for internal collections.
My suggestion is to remove param logging. If params are currently used by Parse Server, then we could think about finding a replacement concept. Any logging of params can be implemented fine-grained and according to use case by the developer in the job script itself.
codeKonami commentedon Dec 1, 2022
@mtrezza So if I understand you correctly, you think we should remove the
params
altogether?mtrezza commentedon Dec 2, 2022
I would say so, unless there is a good reason why they are logged. Did you look at the code to see whether they are used anywhere?
Remove the request `params` to be saved in JobStatus
_JobStatus
#8343codeKonami commentedon Dec 2, 2022
The only good reason to store the
params
in the JobStatus document would be to have a retry mechanism of some sort. But as of today these params does not seem to be used anywhere.mtrezza commentedon Dec 2, 2022
Good to know that they aren't used. So I'd suggest to remove then completely.
danielsanfr commentedon Dec 8, 2022
If I understood the discussion correctly, the solution would be to keep this file in a separate location like a ParseFile, for example. So it would only be necessary to pass an objectId as a Job parameter.
Although today the parameters are not shown on the Job Status page, I would prefer not to remove them, as simply removing them does not improve privacy or security at all.
I've also thought about the possibility of making a retry mechanism, but I never had the time.
So, if you still want to remove something... Please, just remove it from the collection, because I have several Jobs using parameters and I believe that there are many more users who also use them.
codeKonami commentedon Dec 8, 2022
The params are only removed from the collection, they are still passed through the request to the Parse.Job function so no worries about that.
mtrezza commentedon Dec 8, 2022
@danielsanfr did that answer address your concerns?
Remove the request `params` to be saved in JobStatus
parseplatformorg commentedon May 14, 2025
🎉 This change has been released in version 8.2.1-alpha.2
parseplatformorg commentedon Jun 1, 2025
🎉 This change has been released in version 8.2.1