Skip to content

Conversation

@original-brownbear
Copy link
Contributor

Delaying the creation of the next phase to only when we actually need it makes this a lot easier to reason about and should set up further simplications. Eager creation of the next phase forced a lot of needlessly complicated safety logic around resources on us. Since we never "close" the nextPhase on failure all its resources need to be tracked in via context.addReleasable. This isn't as much of an issue with some recent refactoring leaving very little resource creation in the constructors but still, delaying things saves memory and makes reasoning about failure cases far easier.

Delaying the creation of the next phase to only when we actually need it
makes this a lot easier to reason about and should set up further
simplications. Eager creation of the next phase forced a lot of
needlessly complicated safety logic around resources on us.
Since we never "close" the `nextPhase` on failure all its resources
need to be tracked in via `context.addReleasable`. This isn't as much of
an issue with some recent refactorings leaving very little resource
creation in the constructors but still, delaying things saves memory
and makes reasoning about failure cases far easier.
@original-brownbear original-brownbear added >non-issue :Search Foundations/Search Catch all for Search Foundations labels Nov 1, 2024
@elasticsearchmachine elasticsearchmachine added v9.0.0 Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch labels Nov 1, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-foundations (Team:Search Foundations)

@mosbat
Copy link

mosbat commented Nov 1, 2024

Your PR seems fine, but i hate to say it; they are not accepting such changes since they rejected immediately my PR yesterday that included code performance improvement :). You can try of course but don't get shocked if they don't like the change.

As for me, I don't like functional programming but again, your code is fine.

Copy link

@mosbat mosbat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems ok to me.

Copy link
Member

@javanna javanna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the change looks straightforward, and harmless. I would like to maybe see what simplifications this unlocks next, that you mentioned in the description of the PR. Should they be included or are they adding too much complexity? I must say that I do like how this specific change is simple and contained.

@original-brownbear
Copy link
Contributor Author

Thanks Luca, uff lets do this one by one :D I think I have a pretty short path here for getting rid of the whole tracking of releasable resources as part of the search context, that's massively inefficiency and noisy in terms of leak tracking. Also, we can probably get rid of a bunch of search phase implementations outright and just explicitly run the code, I'll open a PR shortly :)

@original-brownbear original-brownbear added auto-backport Automatically create backport pull requests when merged v8.17.0 labels Nov 1, 2024
@original-brownbear original-brownbear merged commit a222e16 into elastic:main Nov 1, 2024
16 checks passed
@original-brownbear original-brownbear deleted the delay-instantiating-next-phase branch November 1, 2024 14:45
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
8.x Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 116061

jfreden pushed a commit to jfreden/elasticsearch that referenced this pull request Nov 4, 2024
…16061)

Delaying the creation of the next phase to only when we actually need it
makes this a lot easier to reason about and should set up further
simplications. Eager creation of the next phase forced a lot of
needlessly complicated safety logic around resources on us.
Since we never "close" the `nextPhase` on failure all its resources
need to be tracked in via `context.addReleasable`. This isn't as much of
an issue with some recent refactorings leaving very little resource
creation in the constructors but still, delaying things saves memory
and makes reasoning about failure cases far easier.
original-brownbear added a commit to original-brownbear/elasticsearch that referenced this pull request Nov 22, 2024
…16061)

Delaying the creation of the next phase to only when we actually need it
makes this a lot easier to reason about and should set up further
simplications. Eager creation of the next phase forced a lot of
needlessly complicated safety logic around resources on us.
Since we never "close" the `nextPhase` on failure all its resources
need to be tracked in via `context.addReleasable`. This isn't as much of
an issue with some recent refactorings leaving very little resource
creation in the constructors but still, delaying things saves memory
and makes reasoning about failure cases far easier.
elasticsearchmachine pushed a commit that referenced this pull request Nov 22, 2024
…117369)

Delaying the creation of the next phase to only when we actually need it
makes this a lot easier to reason about and should set up further
simplications. Eager creation of the next phase forced a lot of
needlessly complicated safety logic around resources on us.
Since we never "close" the `nextPhase` on failure all its resources
need to be tracked in via `context.addReleasable`. This isn't as much of
an issue with some recent refactorings leaving very little resource
creation in the constructors but still, delaying things saves memory
and makes reasoning about failure cases far easier.
@original-brownbear original-brownbear restored the delay-instantiating-next-phase branch November 30, 2024 10:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged >non-issue :Search Foundations/Search Catch all for Search Foundations Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch v8.17.0 v9.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants