Skip to content

Conversation

@idegtiarenko
Copy link
Contributor

Flat index resolution for an esql query. This includes:

  • detecting the query is executed in CPS env
  • performing FC request configured with corresponding flat resolution parameters
  • initialize cross cluster state from the resolved expressions of FC response instead of original indices pattern.

@idegtiarenko idegtiarenko added >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) :Analytics/ES|QL AKA ESQL v9.3.0 labels Nov 25, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

Copy link
Contributor

@luigidellaquila luigidellaquila left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @idegtiarenko, LGTM

result.fieldNames,
// Maybe if no indices are returned, retry without index mode and provide a clearer error message.
switch (indexMode) {
case IndexMode.TIME_SERIES -> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I see this logic and below is the same as for preAnalyzeMainIndices, maybe we can extract it in a method.

)
.build();

private static final IndicesOptions FLAT_OPTIONS = IndicesOptions.builder(DEFAULT_OPTIONS)
Copy link

@quackaplop quackaplop Nov 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: can we put some comments somewhere about what "flat" means? Just a block of comment text someplace that explains what the difference between these two?

public static final String UNMAPPED = "unmapped";

public static final IndicesOptions FIELD_CAPS_INDICES_OPTIONS = IndicesOptions.builder()
public static final IndicesOptions DEFAULT_OPTIONS = IndicesOptions.builder()
Copy link

@quackaplop quackaplop Nov 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that you're trying to do here and I'm sure it works, but reads very visibly bi-modal. This is not necessarily for this PR, but maybe more for a follow up – can we split these two modalities through simple inheritance? I mean having the abstract base (IndexResolveBase) with protected final helper methods like createFieldCapsRequest, doResolveIndices and the abstract method resolveIndicesVersioned and have two classes deriving from it - DefaultIndexResolver and FlatWorldIndexResolver?
It will make for a better encapsulation and improved ability

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will try this idea, however I am afraid this will shit complexity into service injection part.
While resolution remains somewhat similar, the usage will still be different around how EsqlExecutionInfo#clusterInfo is populated (eg before FC call in regular mode and after FC in CPS).

*/
public void resolveIndicesVersioned(
String indexWildcard,
public void resolveFlatIndicesVersioned(

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: can we use "FlatWorld" as opposed to "Flat" please?

@idegtiarenko idegtiarenko merged commit d8bd6b6 into elastic:main Nov 26, 2025
34 checks passed
@idegtiarenko idegtiarenko deleted the flat_index_resolution branch November 26, 2025 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v9.3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants