Skip to content

Commit

Permalink
Spill batch 0 and move striping to loading (from build)
Browse files Browse the repository at this point in the history
Spilling of batch 0 for parallel hash join is now implemented.

While working on this, it occurred to us that explicit striping during
the build phase was not required. This patch splits tuples into work_mem
sized stripes while loading them.
This is done exclusively for fallback batches, so normal hashjoins will
not incur the overhead.

There are two major items (that I know about) that still need to be
dealt with:
- Pausing and resuming loading for each stripe
  For now, I added a new STS mode called Append to support the current
  "overflow" design, however, I have other design proposals
- parallel stripe instrumentation needs to change (I have design
  proposals for that).

This commit also comments out a test which inserts tuples larger than
work_mem in size (each), which no longer successfully executes.

Co-authored-by: Soumyadeep Chakraborty <soumyadeep2007@gmail.com>
  • Loading branch information
melanieplageman and soumyadeep2007 committed Aug 31, 2020
1 parent 46502bb commit c6843ef
Show file tree
Hide file tree
Showing 14 changed files with 1,694 additions and 853 deletions.
2 changes: 0 additions & 2 deletions src/backend/commands/explain.c
Expand Up @@ -2991,8 +2991,6 @@ show_hash_info(HashState *hashstate, ExplainState *es)
worker_hi->nbatch_original);
hinstrument.space_peak = Max(hinstrument.space_peak,
worker_hi->space_peak);
if (!hinstrument.fallback_batches_stats && worker_hi->fallback_batches_stats)
hinstrument.fallback_batches_stats = worker_hi->fallback_batches_stats;
}
}

Expand Down

0 comments on commit c6843ef

Please sign in to comment.