Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Search Templates: Adds API endpoint to render search templates as a response #11570

Merged
merged 1 commit into from Jun 30, 2015

Conversation

@colings86
Copy link
Member

commented Jun 10, 2015

Adds a '_render/template' endpoint which will render a search template in the response with a provided params object.

Closes #6821

@colings86 colings86 self-assigned this Jun 10, 2015
@colings86 colings86 force-pushed the colings86:feature/renderTemplate branch Jun 10, 2015
@markharwood
markharwood reviewed Jun 12, 2015
View changes
docs/reference/search/search-template.asciidoc Outdated

A template can be validated with given parameters using

[source,js]

This comment has been minimized.

Copy link
@markharwood

markharwood Jun 12, 2015

Contributor

Can we qualify what we mean by valid/invalid?
Legal JSON?
Legal search syntax?
I'm guessing the former.

@markharwood
markharwood reviewed Jun 12, 2015
View changes
...sticsearch/action/admin/indices/validate/template/TransportValidateSearchTemplateAction.java Outdated

public class TransportValidateSearchTemplateAction extends HandledTransportAction<ValidateSearchTemplateRequest, ValidateSearchTemplateResponse> {

private ScriptService scriptService;

This comment has been minimized.

Copy link
@markharwood

markharwood Jun 12, 2015

Contributor

final

@markharwood

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2015

While debugging my templates I wasn't quite sure if my saved (to file) templates had taken effect i.e. reloaded from disk. Might it makes sense to include some form of template version ID in the output - even if its based on something like System.identityHashCode(cachedScriptObject)?
That way I can tell what I am testing against (I couldn't see a way of forcing an immediate script reload)

@polyfractal
polyfractal reviewed Jun 12, 2015
View changes
rest-api-spec/test/template/30_validate_search_template.yaml Outdated
body: { "id": "1", "params": { "my_value": "bar", "my_size": 100 } }

- match: { template_output.query.match.text: "bar" }
- match: { template_output.size: "100" }

This comment has been minimized.

Copy link
@polyfractal

polyfractal Jun 12, 2015

Member

Is there a reason the template ("size": "{{my_size}}") and response (template_output.size: "100") are specified as a string, but the input is a numeric ("my_size": 100) ? I don't think this will cause problems for most clients since types are pretty loose, although .Net may be unhappy. Just curious :)

This comment has been minimized.

Copy link
@colings86

colings86 Jun 15, 2015

Author Member

In the script the size parameter is specified as a string because "size": {{my_size}} is not valid json and the parser error on it before we get a chance to run the mustache engine. This is a known bug and something we need to fix separately.

This comment has been minimized.

Copy link
@clintongormley

clintongormley Jun 15, 2015

Member

The output line check template_output.size should be numeric, otherwise .net and python (?) will probably complain. For the input, it is OK because we coerce internally.

This comment has been minimized.

Copy link
@clintongormley

clintongormley Jun 15, 2015

Member

This is a known bug and something we need to fix separately.

I don't think it is a bug, rather just a limitation. If we need to pass in a number as a mustache variable then it should be without quotes, which makes it invalid JSON, which means that the template needs to be an escaped string instead of a JSON object.

@polyfractal
polyfractal reviewed Jun 12, 2015
View changes
...sticsearch/action/admin/indices/validate/template/TransportValidateSearchTemplateAction.java Outdated

@Override
protected void doExecute(final ValidateSearchTemplateRequest request, final ActionListener<ValidateSearchTemplateResponse> listener) {
threadPool.generic().execute(new AbstractRunnable() {

This comment has been minimized.

Copy link
@polyfractal

polyfractal Jun 12, 2015

Member

Should this maybe use the search threadpool instead? Just slightly worried since generic is unbounded, and a runaway app (or malicious user) could accidentally spam a 100k validates and blow out the threadpool.

Not sure if it's kosher to add not-quite-search tasks to the search threadpool though :)

This comment has been minimized.

Copy link
@colings86

colings86 Jun 15, 2015

Author Member

I think it should be ok since only 1 thread is kicked off per validate call and if a user spams 100k validates they should be limited by the transport threadpool before this becomes a problem. Maybe @s1monw could confirm since he suggested I use the generic threadpool?

@polyfractal

This comment has been minimized.

Copy link
Member

commented Jun 12, 2015

Two comments, but LGTM (although admittedly I don't have much experience with this part of the code).

Agree with @markharwood, would be nice to see visual indication when a template changes.

On a similar note, when validating against static file templates...does this always validate using local templates? Because if it redirects to a different node, an indicator of what node executed the validation may be helpful too, in case nodes have different static files.

@clintongormley

This comment has been minimized.

Copy link
Member

commented Jun 15, 2015

Agree with @markharwood, would be nice to see visual indication when a template changes.

The same issue occurs when actually running scripts/templates. How do you know that it has been reloaded? The easiest way is just to watch the log (which you're likely to be able to see during development anyway). Also, we have plans (on linux at least) to support inotify, which will cause reloads to happen immediately. I don't think we need to add metadata here.

On a similar note, when validating against static file templates...does this always validate using local templates? Because if it redirects to a different node, an indicator of what node executed the validation may be helpful too, in case nodes have different static files.

Again, same problem when executing scripts/templates. Making sure you have the same file on all nodes is the developer's job. We could provide some API to (eg) check that a particular file exists and is the same on all nodes, but I don't think this API is the place to do that.

@colings86

This comment has been minimized.

Copy link
Member Author

commented Jun 15, 2015

@markharwood @polyfractal @clintongormley I pushed a commit with changes based on your reviews

@clintongormley

This comment has been minimized.

Copy link
Member

commented Jun 17, 2015

Hmmm it would be nice to just run a quick JSON parsing check on the output.

@colings86

This comment has been minimized.

Copy link
Member Author

commented Jun 18, 2015

I have renamed the classes and the endpoint to use render instead of validate. The original intent of this PR was to show the user what the template and params creates as a final output which would be run on a search, not to catch all the cases where the resulting request may be invalid. This can be done easily by running the template in a search request. The missing bit was something to show the user what the template becomes befor the search is run and this endpoint is intended to fill that gap.

@clintongormley

This comment has been minimized.

Copy link
Member

commented Jun 18, 2015

+1

@colings86 colings86 force-pushed the colings86:feature/renderTemplate branch Jun 22, 2015
@colings86 colings86 force-pushed the colings86:feature/renderTemplate branch Jun 30, 2015
@markharwood

This comment has been minimized.

Copy link
Contributor

commented Jun 30, 2015

Rest tests currently failing due to the way the build process works but otherwise LGTM

@colings86 colings86 force-pushed the colings86:feature/renderTemplate branch to d9ab3cb Jun 30, 2015
@colings86 colings86 merged commit d9ab3cb into elastic:master Jun 30, 2015
1 check passed
1 check passed
CLA Commit author has signed the CLA
Details
@kevinkluge kevinkluge removed the review label Jun 30, 2015
@colings86 colings86 deleted the colings86:feature/renderTemplate branch Jun 30, 2015
@javanna javanna referenced this pull request Jul 3, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.