Add a "start_after" parameter or something to Transforms

[benwtrent]

Description

A very common use case for transforms is pivoting historical data. But, when there is a large amount of past data, that data may not be very useful in the resulting index.

User's typically achieve this by adding a query filter to the transform:

{"range":{"timestamp": {"gte": "now-1d"}}}

But this may introduce additional issues around

• search request caching
• strange interactions with frequency, change detections && date histogram

What the user usually wants is:

• When handling the initial batch of data (all historical), only worry about the last day
• Then, take over when running continuously when data changes

It would be very useful for a new parameter to _start or something to indicate that the initial batch of the transform should be restricted to after some page of results, and then when running continuously don't worry about that filter any longer.

[elasticsearchmachine]

Pinging @elastic/ml-core (Team:ML)

[mikeh-elastic]

We are simulating this feature with starting with a query for recent data and then will update the transform to relax the time to a much longer interval or no time query at all. It would be nice to have this as a core feature of transforms.

[droberts195]

We added a from parameter in #91116.