"too many plugins" error

[markus2330]

Due to a stupid optimization the number of plugins at a single position is limited to some arbitrary (very small) number. With Elektra getting more and more modular this is getting a major problem.

Steps to Reproduce the Problem

For example, in #914 the problem was already reported quite some time ago and you easily hit the problem if

1. you have many validation plugins,
2. write your own modular extensions,
3. having modular storage plugins.

Expected Result

Mounting should work, regardless of the number of plugins.

Proposal

1. The easiest but maybe most hacky way would be to use the list plugin. Here we would only need to adapt the mount logic and no other part of Elektra. It basically boils down to extend Backend::serialize in src/libs/tools/src/backend.cpp
2. Another easy and very compatible way is to allow subpositions if the regular positions are all occupied. For example, if we need 2 postsetplugins, we would simply use setplugins/#6.0#sync#sync and setplugins/#6.1#otherplugin#otherplugin. Here we would need to adapt the mount logic and implement subpositions in elektraProcessPlugins, and fix the data structure _Backend.
3. Redesign system/elektra/mountpoints to not contain any bounded numbers. Here we would need to pump the major number except if we have keep the old way to mount plugins as legacy.

Anyone interested in helping out here?

[markus2330]

There might be a 4th option (just a short reminder for myself, on interest I can explain more detailed):

4.) We could simply extend the mount tool and the bootstrapping process that there is no maximum number, even though we keep the numbering. The trick is that we dynamically tell at which number the commit should happen. This way the bootstrap only needs a constant number of ksLookups more, and at run-time there would be no changes in performance (no list needed). Big advantage of this way: current mountpoints would work without any modification. Disadvantage: No lazy loading of plugins, we would need a "lazy" plugin that loads plugins only on demand.

[ghost]

Is there anybody here who could fix this crucial error? It is also needed for LCDproc.

I would do it myself if I had the necessary skills for it.

[markus2330]

I think the first step is to decide what to do and give a more detailed description of how to do it. Can you do such a preparation work?

[ghost]

I fear that my C/C++ skills are not enough for the preparation work unfortunately. I do not even understand any of your 1-4 possible suggested solutions 😞

[stale]

I mark this issue stale as it did not have any activity for one year. I'll close it in two weeks if no further activity occurs. If you want it to be alive again, ping the issue by writing a message here or create a new issue with the remainder of this issue.
Thank you for your contributions 💖

[stale]

I closed this issue now because it has been inactive for more than one year. If I closed it by mistake, please do not hesitate to reopen it or create a new issue with the remainder of this issue.
Thank you for your contributions 💖