Skip to content

Conversation

@mark-vieira
Copy link
Contributor

Introduce a "fallback" mechanism to read query builders from percolator fields from source, rather than from the binary field value. This may be necessary in certain cases if the documents were originally ingested with a earlier version of Elasticsearch for which transport serialization compatibility has been removed. In this case the queries can still work, albeit less efficiently. When we do this we emit a warning instructing that users reindex to avoid this additional overhead.

@mark-vieira mark-vieira added >non-issue :Core/Infra/Core Core issues without another label :Search Foundations/Search Catch all for Search Foundations labels Oct 31, 2025
@mark-vieira mark-vieira requested a review from javanna October 31, 2025 15:55
@mark-vieira
Copy link
Contributor Author

@javanna I'm just looking for some validation that I'm either going in the right direction, or completely off course here :)

As we discussed, this is to address that fact that if folks have already upgraded from 8.x to 9.x, and have older indices with percolator fields, we need a way to avoid breaking those folks.

Copy link
Member

@rjernst rjernst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a unit test that trips the fallback case?

@mark-vieira
Copy link
Contributor Author

Could you add a unit test that trips the fallback case?

Yes. Wanted to get a some confirmation that this approach was reasonable.

Copy link
Contributor

@andreidan andreidan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this Mark.

This has the right sort of shape for what we need. I think it's going in the right direction.

@mark-vieira mark-vieira changed the title Initial spike of percolator field fallback compatibility Add percolator field fallback compatibility Nov 17, 2025
@mark-vieira mark-vieira marked this pull request as ready for review November 17, 2025 18:19
@elasticsearchmachine elasticsearchmachine added Team:Core/Infra Meta label for core/infra team Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch labels Nov 17, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-foundations (Team:Search Foundations)

@mark-vieira
Copy link
Contributor Author

I've updated the existing QueryBuilderBWCIT so that it's actually exercising the fallback added in this PR.

@mark-vieira mark-vieira added the test-full-bwc Trigger full BWC version matrix tests label Nov 17, 2025
Copy link
Contributor

@jdconrad jdconrad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Just left one small question.

assert valueLength > 0;

TransportVersion transportVersion;
if (indexVersion.before(IndexVersions.V_8_8_0)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume when we remove all versions prior to V_8_8_0 this if/else just becomes the else in a safe way now since it'll either reload from source or reject if there is no source?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is IndexVersion so this won't go away until ES 10.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yes. Oops! :)

@mark-vieira mark-vieira added auto-backport Automatically create backport pull requests when merged branch:9.2 branch:9.1 labels Nov 18, 2025
@mark-vieira mark-vieira merged commit 0f458df into elastic:main Nov 19, 2025
31 checks passed
@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
9.2
9.1

mark-vieira added a commit to mark-vieira/elasticsearch that referenced this pull request Nov 19, 2025
mark-vieira added a commit to mark-vieira/elasticsearch that referenced this pull request Nov 19, 2025
ncordon pushed a commit to ncordon/elasticsearch that referenced this pull request Nov 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged :Core/Infra/Core Core issues without another label >non-issue :Search Foundations/Search Catch all for Search Foundations Team:Core/Infra Meta label for core/infra team Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch test-full-bwc Trigger full BWC version matrix tests v9.1.8 v9.2.2 v9.3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants