Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.
Sign upcherrypick-1.1: distsql: pre-reserve memory needed to mark rows in HashJoiner build phase #18975
Conversation
asubiotto
requested a review
from
andreimatei
Oct 3, 2017
asubiotto
requested review from
cockroachdb/distsql-prs
as
code owners
Oct 3, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
Show comment
Hide comment
asubiotto
requested a review
from cockroachdb/release
Oct 3, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
Show comment
Hide comment
andreimatei
Oct 3, 2017
Member
Review status: 0 of 3 files reviewed at latest revision, all discussions resolved, all commit checks successful.
Comments from Reviewable
|
Review status: 0 of 3 files reviewed at latest revision, all discussions resolved, all commit checks successful. Comments from Reviewable |
asubiotto
merged commit 386d96f
into
cockroachdb:release-1.1
Oct 3, 2017
asubiotto
deleted the
asubiotto:hashjoiner-1.1
branch
Oct 3, 2017
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
asubiotto commentedOct 3, 2017
A situation was uncovered by #18600, where the HashJoiner would run out
of memory in the probe phase. This was because we had made an assumption
that we wouldn't hit a memory limit if the buffer phase filled up at
most 2/3 of the limit with both streams , since the marking
infrastructure would take up only a fraction of 1/3 (the chosen stream).
This assumption failed to take into account other limits shared with
other processors. This change pre-reserves the memory needed for the
probe phase in the build phase so that we can keep a single point in the
code where we fall back to disk while not relying on any limit
assumptions.