adds support for multiple apps on a single server

[flovilmart]

• Adds support for loading configuration from JSON
• Adds support for loading multiple configurations from ENV
• Adds generator for new app keys
• Adds support for cloud code isolated processes
• Adds cloud code runner / launcher
• Full /hooks REST API (only master)
• Parse.Hooks: register hooks on nodejs/cloud code server
• Parse.Cloud with automatic hooks registration (from parse-cloud-express)
• Customizable hook registration strategy for cloud code server (always, try, never)
• All unit tests are running in a multiple apps context

[facebook-github-bot]

@flovilmart updated the pull request.

[drew-gross]

Hey, thanks for the pull request, but I don't think we will be able to accept this, as it will interfere with our implementation of Cloud Code and our (upcoming) implementation of push. We could look at how we would support this again after push lands. We would also need to test this change very thoroughly.

[flovilmart]

Cloud code would work the same as the path is provided per application.

[drew-gross]

As @gfosco explained to me, the issue is not the paths for the cloud code, but rather that the implementation of cloud code modifies global variables, so the cloud code from one app could affect the other apps. If you can write some tests that verify that this works with cloud code, and also wait until our push implementation is ready and write some tests to make sure that works as well, we could probably accept this.

[flovilmart]

In that case i would add process isolation for cloud code like i did In https://github.com/flovilmart/parse-anywhere so the environment of cloud code remain 'stable'

[flovilmart]

That seems to be a good time to also bring back the REST API hooks for cloud code. This way we ensure isolation of cloud code in a subprocess and use that API to make the calls.

[facebook-github-bot]

@flovilmart updated the pull request.

[flovilmart]

actually @drew-gross @gfosco the main problem is the Parse JS SDK here that is completely stateful, and the architecture that is semi stateful. We leverage global variables across the different parts when we should rely only on request parameters (mostly for triggers.js and functions.js).

I'd need to make a Parse SDK that could generate new instances, or factory instances per AppID, and attache the instance to the expressjs request in a middleware, this way, instead of using a global Parse object, we'd use a local Parse object per request.

[drew-gross]

Factory instances for the Parse SDK sounds like a good idea to me.

[flovilmart]

That imply a quite big refactor, just looked today, as the JS SDK is completely architectured around the CoreManage 'singleton'.

[facebook-github-bot]

@flovilmart updated the pull request.

[drew-gross]

Yeah, I suspected it would be :( and given that it's pretty easy to just run another instance of parse-server for your other apps, it's probably not worth it. We can try to keep it in mind when editing the current code though, and maybe it's something we can eventually support.

[flovilmart]

The latest version is 'stable'.
I'll add more tests for race conditions.
With the current state of the Parse JS SDK, it should not be problematic when running standalone.

When using require, the developer should make a particular effort to call Parse.initialize() outside The cloud code import to select the proper app.

I most probably should call Parse.initialize before each hook call to force the Parse singleton.

What do you think?

[facebook-github-bot]

@flovilmart updated the pull request.

[drew-gross]

I'll still have to check with our push team to see what conflict might exist with our push implementation, as I know they were worried about having multiple apps on the same server. Thanks a lot for writing tests :) Reinitializing the Parse JS SDK at the appropriate times sounds like it might be a good solution to global state problems, and we use that solution in the Parse Dashboard, but with multiple requests happening at once in Node.js, that might not end up being enough. Race condition tests sounds like a good idea.

[flovilmart]

I'll check the push implementation in that case as I rebased recently and it's there on that branch.

[flovilmart]

The solution for cloud code would be to isolate it in it's own subprocess, this way the parse would be on it's one.
The caveat would be IPC in that case, we'd have to reimplement the hooks endpoint and use HTTP as a transport.

[pivanov]

maybe this is a mistake :)

[flovilmart]

Yep... That's a rebase error... Still working on Cloud Code :)

[facebook-github-bot]

@flovilmart updated the pull request.

[flovilmart]

@drew-gross I managed to isolate the Cloud code runs in the case of multiple apps, or if specified like that. Now I need to add a persistence layer for the hooks URL's otherwise upon server restart, we loose all the mapping.
In the case of single server, we stick with the original implementation, or through configuration, it can spawn a subprocess.

The subprocess is there by default for the multiple apps case, so it is safe.

The implementation relies on parse-cloud-express (because it makes sense).

Calling any Parse.Cloud.[define, beforeSave, afterSave, beforeDelete, afterDelete] in the subprocess environment will update the hook asynchronously, and following the Parse REST API.

A developer can also generate a clean Cloud Code environment on a 3rd party server with:

var cloudCode = require('parse-server/cloud-code');
cloudCode(config);

This should be safe for cluster configuration of cloud code.

The triggers Api is pretty much unchanged, the network calls are wrapped into the trigger itself.

On that, should we load all hooks in memory upon server start or for each request?

[facebook-github-bot]

@flovilmart updated the pull request.

[flovilmart]

@gfosco I'll need a PR for that : https://github.com/ParsePlatform/parse-cloud-express/pull/17 to clean up some code

[facebook-github-bot]

@flovilmart updated the pull request.

[facebook-github-bot]

@flovilmart updated the pull request.

[flovilmart]

Needed to remove that import because of parse-cloud-express and conflicting dependencies.

[facebook-github-bot]

@flovilmart updated the pull request.

[flovilmart]

And I just had an epiphany... We can leverage heroku Procfile in order to let the cloud code process externally managed.

web: npm start // Main parse server
web: node cloud_app0.js
web: node cloud_app1.js
...
worker: node worker.js

[gfosco]

Sadly it is looking more and more like multiple-app support won't make it in to the core repo.. If you're okay with closing this for now @flovilmart, I'd like to keep things clean.

[flovilmart]

@gfosco definittely, we're 3 scripts away from supporting it as I decoupled all pieces. The 1st iteration would be experimental, without 'internal' cloud code support.

[SaifAlDilaimi]

Hey guys,

I just have read the whole conversation and I totally agree with @flovilmart reasons to support multiple apps. I would really love to see this coming to light. Just wanted to show my affection for this feature. Great work @flovilmart !

[DanielsCode]

Any news about multi-app support? Would be a great feature!

[mediaexpander]

Hi! How is multi-app support going?

[mohammed1ali]

Any updates on multiple app support?
This is an important addition.

[flovilmart]

no update and it's unlikely it will be any update coming from me

[panjiubang]

our project also conside multi-tenancy recently, Any updates on multiple app support?

[flovilmart]

Still haven't changed, and it is unlikely to.

[wujohns]

@flovilmart the node.js now support vm which provides APIs for compiling and running code within V8 Virtual Machine contexts. Could it support the multiple app feature by using those apis?

[flovilmart]

No it won't, the context need to be isolated from the main process. It's been extensively discussed before, tried and failed.

The only solid solution is to spawn subprocesses either for the full cloud code or for each function call, which is costly and inefficient.

[wujohns]

thank for your answer. I will do some try by using pm2 to manage mutiple paser-server instances. Maybe the solution is not in paser-server itself.