Skip to content

ESQL: version-aware planning follow-ups and testing improvements #137534

@alex-spies

Description

@alex-spies
  • Make sure our internal docs/javadoc explaines the logic around version-aware planning and releasing new types sufficiently.
  • Document dangers of backporting version-aware planning changes: a fix backported to a previous minor can be breaking if it's not in all subsequent patch versions of the next minor. Example: we implement a planner change in 9.3.1 and 9.2.2 - then a 9.2.2 coordinator will assume that the minimum version is 9.2.2, and thus all nodes can handle the new planning - but in fact, in the presence of 9.3.0 nodes, we may see serialization errors (or, worse, silently wrong results).
    Consequently, backporting version-dependent planning changes more than 1 minor is generally wrong.
  • Data types need to have an additional version that they're created on, so we can switch them off in SNAPSHOT-only bwc tests against older versions. ESQL: Add createdVersion for types #137432
  • AllSupportedFieldsTestCase should also run in SNAPSHOT. That requires the point above.
  • ROW-queries with ENRICH/LOOKUP JOIN need to respect the local minimum transport version, still. ESQL: Fix enrich and lookup join resolution based on min transport version #137431
  • In case of CCS, remote EnrichPolicyResolvers need to take the overall min transport version into account, not just the local one on the remote cluster. ESQL: Fix enrich and lookup join resolution based on min transport version #137431
  • More test cases, e.g. to be added to AllSupportedFieldsTestCase:
    • Remote-only queries, like FROM remote:* | ... - without a single local index.
    • Queries with FROM and ENRICH
    • CCS queries with FROM and ENRICH
    • Queries with FROM and LOOKUP JOIN
    • Tests with CPS/remote indices aliased by a flat index name
    • Remote lookup joins: FROM remote:* | LOOKUP JOIN lu_idx ON ...
    • Tests with subqueries; esp. ones where the subqueries use different index patterns in CCS, or some use ROW and others use FROM. -> ESQL: Make local planners/mapper aware of minimum transport version so we can make version-aware changes to aggs #138805
    • Bwc tests with union types, when one (or more) of the constituent types are only supported on the newer nodes
    • Tests with _tsid
  • Find out what to do with release bwc tests: e.g. release test for 9.3 against 9.2.1 will have the problem that 9.2.1 is actually on SNAPSHOT, and thus will consider under construction data types as supported. This already applies to AllSupportedFieldsTestCase and needs to be worked around right now; it will mess with our csv tests as well when we release the next data type (date_range).
    We will either have to skip inconsistent configs (SNAPSHOT/SNAPSHOT is okay, release/release should be okay, mixed is not okay), or we have to adjust our release tests to build all clusters in release mode.
  • ESQL: Make local planners/mapper aware of minimum transport version so we can make version-aware changes to aggs #138805

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions