Add transform to document before index.

[nik9000]

Closes #6566

[nik9000]

Still needs documentation.

I'm not super happy with how the transform is done - it turns the XContent into a Map them back into an XContent. Not perfect at all.

Also, this has a bunch of TODOs, some of which deserve doing before merging, I imagine.

[jpountz]

+1

[jpountz]

Thanks @nik9000 this looks very good! API-wise, I think it would be nice to be prepared for other ways to do transformations in the future? Although scripts are powerful, it think we could have dedicated transformers for common transformations like concatenating two string fields or suming up all values of a numeric field (which probably means we also need to support several transformers?).

[clintongormley]

To be consistent with other scripts, I think we should always require the script key ie:

"example": {
    "transform": {
        "script": "......"
    }
}

This also leaves the path open for built-in transforms

[nik9000]

Will do.

[clintongormley]

As per @jpountz 's comment, we could keep the path open for built-in transforms later in a format like this perhaps:

"transform": [
    { "add": { "total_price": [ "price", "tax", "markup" ]}},
    { "concat": { "full_name": [ "first", "last" ]}},
    { "script": "......", "lang": "groovy", "params": {....}}
]

[nik9000]

Would it make sense to support either the

"transform": [
    { "add": { "total_price": [ "price", "tax", "markup" ]}},
    { "concat": { "full_name": [ "first", "last" ]}},
    { "script": "......", "lang": "groovy", "params": {....}}
]

or the

"transform":     { "script": "......", "lang": "groovy", "params": {....}}

syntax? We do something like this in other places I think.

[clintongormley]

Yes, that's what I actually meant. Currently we don't have the short syntax, but as long as the way is open to support both in the future, that's all we need.

[nik9000]

Almost done with @jpountz's code comment on copy-and paste-y-ness. Going to start on api next.

It is good we're doing with with Groovy, BTW, because I tried a month ago to execute MVEL on every one of my updates and saw some real instability. Weird error message that I couldn't get in dev when doing the same things.

[nik9000]

I pushed a fix for the code duplication and rebased because it wasn't going to merge cleanly. Now to fix the syntax.

[nik9000]

Almost done with API - I've noticed the integration tests are starting much much much faster then a month ago. Neat!

[nik9000]

OK! Added @clintongormley's API reformat. I think that made it cleaner actually. I added the infrastructure for multiple transforms of different types but only implemented the script type.

[clintongormley]

w00t! I've tagged it for review. thanks @nik9000

[jpountz]

This can just be Map<String, ?> since Object is the base class.

[nik9000]

Yeah. Doing it.

[jpountz]

I left two minor comments but in general this looks very good to me! This is an exciting feature.

[nik9000]

@jpountz: updated with fixes from comments.

[jpountz]

Thanks Nik, I'll merge it soon!