Fix unspill scheduling in join spill #9275
Conversation
|
Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please submit the signed CLA to cla@trino.io. For more information, see https://github.com/trinodb/cla. |
| public synchronized void setDisposeCompleted() | ||
| { | ||
| disposeCompleted.set(null); | ||
| setState(State.DISPOSE_COMPLETED); |
There was a problem hiding this comment.
can this be called before dispose by HashBuilderOperator?
There was a problem hiding this comment.
why do you want to call this before dispose? Then the next page could be loaded before dispose is done.
core/trino-main/src/main/java/io/trino/operator/join/SpilledLookupSourceHandle.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/operator/join/PartitionedLookupSourceFactory.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/operator/join/PartitionedConsumption.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/operator/join/PartitionedConsumption.java
Show resolved
Hide resolved
JunhyungSong
force-pushed
the
join-unspill-schedulling
branch
from
0b24925
to
99520ce
Compare
|
Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please submit the signed CLA to cla@trino.io. For more information, see https://github.com/trinodb/cla. |
|
@JunhyungSong could you sign CLA please? |
I have done that. can you check that, please? |
|
@cla-bot check |
|
The cla-bot has been summoned, and re-checked this pull request! |
|
Could you push empty commit? |
Before the fix, unspill of next partition is triggered when previous partition is released. However, it just sends release request without waiting for release to complete. As a result, multiple partitions exist simultaneously and result in memory limit error.
JunhyungSong
force-pushed
the
join-unspill-schedulling
branch
from
99520ce
to
b7ff910
Compare
Done. |
Comparable changes extracted from prestodb/presto#16339
Before the fix, unspill of next partition is
triggered when previous partition is released.
However, it just sends release request without
waiting for release to complete. As a result,
multiple partitions exist simultaneously and
result in memory limit error.