Load script files from elasticsearch config/scripts directory

[bradvido]

From testing scripted fields in Kibana4, it seems that providing the name of a script file for the script value, does not work (it's interpreted as a literal script, not a reference to a file). It would be nice to have a tick box that toggles if the script value references a file in config/scripts.

For scripting languages that are not sandboxed and shouldn't be allowed to be executed dynamically (groovy), this would be really useful and it is the supported method for executing scripts safely, per ElasticSearch documentation:

[bradvido]

After more testing, it seems that my initial findings may not be 100% accurate. I just successfully used a script (in the discover screen) that is referencing a file name.

Either way, the Kibana Scripted field screen should have a blurb explaining that you can reference a script file (if supported)

[rashidkpc]

I'd consider this with a couple caveats:

1. We need a way to list the loadable scripts. Only scripts with a currently enabled langauge should be shown
2. We need a way to confirm those scripts are present on every elasticsearch node in the cluster.

This would need some elasticsearch APIs iirc, you might consider opening a ticket on the elasticsearch repo and linking it to here.

You can work around this using the JSON input in an agg, but we're not going to officially support it until we can do it right.

[bradvido]

I agree. to validate a script, Kibana needs to know if:

1. The script file exists on every node (for file scripts)
2. The script is compilable/runnable (for file and dynamic scripts alike)
3. if the script is allowed to execute (language and disable dynamic scripting settings in ES config may prevent it).

I believe this would require new ES APIs. I'll investigate on the ES side to see if they exist or can be created.

[JD1337]

@bradvido How were you able to use the saved script by referencing the file name?

@rashidkpc for the JSON input is there an example of referencing a script from this? I can't seem to google it anywhere?

Since #3809 is not the direction you are going in regards to the saved script files, is there an issue that isn't linked to this which is giving some progress in using the saved scripts? Or is this not happening at all in favour of something else?

I'd love to be able to get the example scripting section on this page (https://www.elastic.co/blog/kibana-4-beta-3-now-more-filtery)

[bradvido]

@rashidkpc Regarding your points 1 and 2 above, I don't think those should be blocking implementation of this feature. I'd prefer kibana to "try and execute the search/script, and if it fails, notify the user"

Does kibana really need to know beforehand that the scripts have been loaded on every ES node, or can we trust the ES administrator has done that properly in 98% of the cases? It seems easier and simpler to just try and execute the query/script and if ES throws an exception, notify the user (or use the _validate/query endpoint if you really need to verify it before actually trying). Likewise, we can trust that the user knows the script file name and has entered it properly, but show a notification if ES complains about it.

Loading and executing scripts from the config/scripts directory is more secure and more maintainable than executing dynamically entered scripts which are only known to kibana. What if I want my custom app and kibana to use the same script? I currently have to maintain two copies of it, where it would be better if they both used the same script file.

[rashidkpc]

I'm going to close this as we've decided to wait for the new Elasticsearch scripting language. We do not want to ask users to distribute scripts to every node in their cluster. Better scripting is coming, but not via filesystem scripts: elastic/elasticsearch#13084

[bradvido]

Thanks for the info. Will be watching elastic/elasticsearch#13084 closely

[feicipet]

Can I just get a confirmation that it is still possible to specify a script_field in Kibana and point it to a Groovy script on disk to execute? I do need to do some string manipulation in a field right now and this seems to be my only option. I'm running Kibana + ES using Docker containers right now so it'll take me a bit of extra work to create an image with Groovy baked in, which I'm fine to do but would like to confirm that this approach would work in the end. Thank you!