[ES|QL] TS Command: Can't rename timestamp field

[limotova]

ts k8s 
| rename `@timestamp` AS nNVStdFbIFX 
| stats  max(rate(network.total_cost)) by bucket(nNVStdFbIFX,1hour)

same query with bucket on @timestamp also fails

            "type" : "verification_exception",
            "reason" : "Found 1 problem
line 3:14: Unknown column [@timestamp]",
            "stack_trace" : "org.elasticsearch.xpack.esql.VerificationException: Found 1 problem
line 3:14: Unknown column [@timestamp]
     org.elasticsearch.xpack.esql.analysis.Analyzer.verify(Analyzer.java:235)
     org.elasticsearch.xpack.esql.analysis.Analyzer.analyze(Analyzer.java:229)
     org.elasticsearch.xpack.esql.session.EsqlSession.analyzedPlan(EsqlSession.java:784)
     org.elasticsearch.xpack.esql.session.EsqlSession.analyzeWithRetry(EsqlSession.java:725)
     org.elasticsearch.xpack.esql.session.EsqlSession.lambda$analyzedPlan$11(EsqlSession.java:407)
     org.elasticsearch.server@9.2.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:261)
     org.elasticsearch.server@9.2.0-SNAPSHOT/org.elasticsearch.action.support.SubscribableListener$SuccessResult.complete(SubscribableListener.java:406)
     org.elasticsearch.server@9.2.0-SNAPSHOT/org.elasticsearch.action.support.SubscribableListener.tryComplete(SubscribableListener.java:326)
     org.elasticsearch.server@9.2.0-SNAPSHOT/org.elasticsearch.action.support.SubscribableListener.setResult(SubscribableListener.java:355)
     org.elasticsearch.server@9.2.0-SNAPSHOT/org.elasticsearch.action.support.SubscribableListener.onResponse(SubscribableListener.java:262)
     org.elasticsearch.xpack.esql.session.EsqlSession.preAnalyzeLookupIndices(EsqlSession.java:422)
     org.elasticsearch.xpack.esql.session.EsqlSession.lambda$analyzedPlan$10(EsqlSession.java:406)

[elasticsearchmachine]

Pinging @elastic/es-storage-engine (Team:StorageEngine)

[not-napoleon]

I have a pretty good guess as to what's going on here. I can try to look into it.

[kkrik-es]

Would that work with tbucket? Worth checking.

[not-napoleon]

It also breaks with TBucket, and for the same reason. The implicit field is not resolving correctly in the presence of the rename. If we use an over_time aggregation, which does not get an implicit timestamp field, this works fine:

TS k8s
| RENAME `@timestamp` AS newTs
| STATS maxRate = max(max_over_time(network.eth0.tx))  BY tbucket = bucket(newTs, 1hour)

I've got it reliably reproducing in tests, and I'm working on a solution.

[not-napoleon]

To be more clear, what's going on here is that rate and tbucket explicitly add in hard coded time stamp fields:

public Rate(Source source, @Param(name = "field", type = { "counter_long", "counter_integer", "counter_double" }) Expression field) {
        this(source, field, new UnresolvedAttribute(source, "@timestamp"));
}

I'm still reading through the ResolveRefs rule to see exactly what's going on here, but it seems like the rename is overriding the resolution of the hard coded field there. My instinct is that we need something in ResolveRefs that looks explicitly at TimeSeriesAggregate, finds unresolved timestamps in there and...IDK, scans the rest of the plan for a field or alias to link that to? I need to think more about that.