Better categories for scripted operations

[javanna]

With #10116 we introduced fine-grained script settings. Each time a script is run, it is associated with a context, which holds the operation/api that is making use of the script, provided by the caller.

We currently support 4 operations: search, mapping, update and aggs. We folded suggester and percolator under search though, which might be confusing for users, hence we might want to create proper distinct categories for them.

Even more importantly, plugins may use our ScriptService to expose custom scripted operations, which may not fall into these pre-defined categories. I wonder if we should introduce a new generic category of operations for plugins, which can be enabled/disabled via settings too. This way we may also be able to remove the breaking aspect of this new feature, the problem being that each caller needs to specify which operation it's using the script for, and that is a required argument. Given that we updated all of our internal calls to provide that argument, we could now restore the original methods without the ScriptContext argument and assume that those calls come from plugins, by assigning them the default plugins category.

[javanna]

hey @uboness @clintongormley can you comment on this please?

[dadoonet]

@javanna for now, plugins I know basically use scripts to modify documents before injection. That was mostly the case for some rivers.
So update works well here. All official plugins have now been updated for 1.6.

Not sure we need something else.

[javanna]

@dadoonet yea it makes sense to use update for now but it's ambiguous because it's not really about the update api, but pre-processing of document before they get indexed. I am for making it explicit that plugins may use scripts and we just don't know exactly for what :)

[uboness]

@javanna thinking a few things:

• would be nice of the script context would be extensible... so some plugins will also be able to define their context in more fine grained fashion than a simple "capture all" plugin context. Right now the context is an enum, perhaps it should be based on opaque strings (internally we'll sill use the same constants that the enum already captures.
• we can still have a "capture all" context (say other) that can be configured like all the others and have compile/execute methods overloaded so we'll also have those without the passed in context (in which case the ctx will default to other).... we can perhaps mark it as deprecated and remove it in 2.0 (so from 2.0 ctx will be a required param)?

[javanna]

I agree with @uboness on making things more flexible, maybe we could just use the plugin name though instead of having plugins declare potentially multiple additional contexts (think of conflicting ones as well). Let's see what others think about this aspect.

That said I think we should move on with the first step of introducing an additional category that is specific for plugins, which would also have the advantage of allowing us to restore backwards compatibility on ScriptService. Will work on this.

[uboness]

maybe we could just use the plugin name though instead of having plugins declare potentially multiple additional contexts (think of conflicting ones as well). Let's see what others think about this aspect.

I can see multiple categories used within a single plugin as well... so maybe then we'll have the category be prefixed with the plugin name... as a convention at least?

That said I think we should move on with the first step of introducing an additional category that is specific for plugins, which would also have the advantage of allowing us to restore backwards compatibility on ScriptService. Will work on this.

+1

[clintongormley]

+1 to #10347 (comment)