make it possible to load multiple wasm plugin

[sanposhiho]

What would you like to be added:

Currently, the host scheduler has one wasm plugin, and it can load one wasm guest.
Meaning, basically people can have only one wasm.
It'd be great if users can use multiple wasm guest.

Why is this needed:

Completion requirements:

This enhancement requires the following artifacts:

• Design doc
• API change
• Docs update

The artifacts should be linked in subsequent comments.

[codefromthecrypt]

I have a question on this also. In the docs I see the word "plugins" used, I mean in plural. Is there existing ordering concerns with plugins in general, or any collaboration between them now?

I assume this issue is to support multiple guests in one plugin, as opposed to multiple instances of the wasm plugin which I assume would work anyway?

[kerthcet]

Is there existing ordering concerns with plugins in general

Yes, actually scheduler plugins are ordered with the configuration file.

support multiple guests in one plugin, as opposed to multiple instances of the wasm plugin which I assume would work anyway

So the essential question is can we support multi scheduling plugins which implement different extension points, then what's the architecture looks like? Based on my understanding up to now, seems we have to implement different hosts, another approach is we can support the ordering with a new args like wasm-plugin1, wasm-plugin2,... passed to the wasm plugin, and running them sequentially inside, is this possible? But the Cons is we can't order them mixed with kubernetes in-tree plugins.

[codefromthecrypt]

I agree with @kerthcet that nesting plugins inside the wasm plugin has the problem of assuming there's only wasm plugins, or there's no need for native plugins between them. I haven't seen any plugin system yet nest wasm together for reasons like this. So, one thing I think we should do is assess if the first version of this feature really needs to do this or not, as it will add complexity and we are already adding a lot of complexity without any tests.

[kerthcet]

I think we're just discussing here not target for the first release.
/priority important-longterm

[sanposhiho]

agree that it's not a priority for now.
I think there can be some options and eventually, it's needed to compare them through design doc or something.