Allow arbitrary plugins on Cloud

[Twixes]

A while ago we released Plugins for PostHog Cloud users.
However the limitation there is that only verified plugins from the official plugins repository are allowed for most users – system safety reasons prevent us from allowing arbitrary code in the multi-tenant architecture.
This is already very useful, but it would still be fantastic and even more powerful if ANYONE could run ANY plugin code (using the source code editor for instance).

Theoretically we could allow it right now, it's practically a single switch. However there's a few issues that we need to sort out beforehand to ensure stability of the system:

• protection against plugin server being used for denial of service attacks (frontend: Make "Invite Team" link box visible #395)
• protection against plugin server out-of-memory errors, accidental or maliciously induced (Helps #155 Add use x forwarded host #159)
• protection against high sustained CPU usage
• audit package imports system for plugin server filesystem access and make sure it's impossible

In CPU and memory usage protections we're severely limited by using vm2 for running plugins, which is a wrapper over Node's vm module. That's performant and convenient – this way all of the server is Node – but it also makes management of resources difficult in a multi-tenant system. All vm VMs runs inside the same Node process as the rest of the plugin server, so they all share the same pool of process resources.
Cloudflare Workers for example use V8 Isolates more directly, running their own V8-based runtime alternative to Node. That gives them full oversight over activity of each individual worker, at the cost of lots of development effort.
A Node-based alternative to vm2 could be isolated-vm, which seems to offer better isolation characteristics. AFAIK @mariusandra may have some more context on that package.
In any case, a rework of plugin VMs could be really useful, but would also require careful consideration, so a last resort for now.

Extra reading: PostHog/plugin-server#227 on isolation between Cloud tenants.

[mariusandra]

The additional context here is that the in the first version I actually used isolated-vm instead of vm2 for exactly these reasons. However I dropped it because it made passing objects into the VM absolutely mental. At the end of the day, I would have had to implement some Proxying code similar to vm2, just to get fetch working, and decided it's not worth the effort, considering the time budget and the extremely likely case that I'll leave in some security holes.

[mariusandra]

isolated-vm also runs every vm inside it's own worker/thread. That has scalability implications. The number of VMs is virtually limitless, but the number of threads a server can run, even if massively overprovisioned, is more limited. 32k or so.

Definitely worth investigating deeper though.

[yakkomajuri]

Another issue to consider is abusing the cache and storage.

[posthog-bot]

This issue hasn't seen activity in two years! If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in two weeks.

[posthog-bot]

This issue was closed due to lack of activity. Feel free to reopen if it's still relevant.