Peak memory cost calculation #11591

findepi · 2018-09-27T09:36:51Z

Draft of peak memory calculations. Does not compile yet, as this is only to share the idea of the approach.

Related to the discussion under #11495

arhimondr

Didn't see WIP. Anyhow, since i already spent time on commenting, here are some comments =)

arhimondr · 2018-09-27T14:49:41Z

presto-main/src/main/java/com/facebook/presto/cost/PlanNodeCostEstimate.java

-    public static final PlanNodeCostEstimate INFINITE_COST = new PlanNodeCostEstimate(POSITIVE_INFINITY, POSITIVE_INFINITY, POSITIVE_INFINITY);
-    public static final PlanNodeCostEstimate UNKNOWN_COST = new PlanNodeCostEstimate(NaN, NaN, NaN);
-    public static final PlanNodeCostEstimate ZERO_COST = new PlanNodeCostEstimate(0, 0, 0);
+    public static final PlanNodeCostEstimate INFINITE_COST = new PlanNodeCostEstimate(POSITIVE_INFINITY, POSITIVE_INFINITY, POSITIVE_INFINITY, POSITIVE_INFINITY);


It would be nicer if they were. You can change them in a separate commit to gain extra brownie points =)

PlanNodeCostEstimate.infinity(), PlanNodeCostEstimate.unknown(), PlanNodeCostEstimate.zero()

I will take this opportunity when i am terribly indebted 😛

arhimondr · 2018-09-27T14:51:24Z

presto-main/src/main/java/com/facebook/presto/cost/PlanNodeCostEstimate.java

    }

    private final double cpuCost;
-    private final double memoryCost;
+    private final double peakMemoryCost;
+    private final double peakMemoryWhenOutputting;


Could you please explain at least in the commit message what's the difference between peakMemoryCost and peakMemoryWhenOutputting?

Sorry for not giving this explanation upfront.

So, the basic idea is as outlined in #11495 (comment): to calculate peak memory usage of a query.
To calculate this recursively, we need to know for every node in a plan tree:

peak memory usage of the subtree

peak memory usage when the node is outputting (producing output) -- this is required to compute peak memory usage of plan tree enclosing given node

this is based on observation that until given node is outputting, plan nodes higher up need no memory (they have nothing to process)

also, some nodes will consume all input before outputting, so when they are outputting, their subtrees no longer need memory

the two properties of source nodes -- peak memory usage and peak memory usage when the node is outputting -- combined with node-specific formulas should allow to computing the properties higher up, realizing peak memory usage calculation for the whole plan

Now, the ugly part. In current cost calculation code, PlanNodeCostEstimate conveniently represents two things:

local cost of the node

total cost of the plan subtree

It's no wonder we reuse single class for this, as currently total cost of the plan subtree is just a sum of local costs of all nodes.
After this change, this design no longer fully fits:

when calculating local cost of the node, we should calculate [cpu, memory, network]

when calculating total cost of the plan subtree, we should calculate [cpu, peak memory, network] (this can still be the same type as local cost of the node), but we temporarily need to hold [cpu, peak memory, peak memory when outputting, network]

it might be that the "extended" type is just an inner thing is CostProvider implementation, so maybe the overall design still holds

... edit: oh it can't. Memo needs to keep this "extended" type as well

summing cost vectors does not always make sense

For now, I didn't address the design part at all. I am just conveniently reusing PlanNodeCostEstimate everywhere.

arhimondr · 2018-09-27T14:51:55Z

presto-main/src/main/java/com/facebook/presto/cost/PlanNodeCostEstimate.java

    }

    @Override
    public String toString()
    {
        return toStringHelper(this)
                .add("cpu", cpuCost)
-                .add("memory", memoryCost)
+                .add("memory", peakMemoryCost)


Change toString method to include both

I intentionally left out the other, as it's irrelevant from cost-comparison perspective.

presto-main/src/main/java/com/facebook/presto/cost/CostComparator.java

presto-main/src/main/java/com/facebook/presto/cost/CostCalculatorUsingExchanges.java

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

arhimondr · 2018-09-27T14:57:04Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java


        PlanNodeCostEstimate sourcesCost = sourceProvider.getSources(node).stream()
                .map(this::getCumulativeCost)
                .reduce(ZERO_COST, PlanNodeCostEstimate::add);

-        PlanNodeCostEstimate cumulativeCost = localCosts.add(sourcesCost);
+        PlanNodeCostEstimate cumulativeCost = node.accept(new CostCombiner(), new CostCombiner.Context(localCost, sourcesCost));
+        verify(cumulativeCost != null);


add a message in case it ever fails

it this unlikely case, a stacktrace should suffice

arhimondr · 2018-09-27T15:04:27Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

+        {
+            PlanNodeCostEstimate cost = context.localCost.add(context.sourcesCost);
+
+            verify(Doubles.compare(context.localCost.getPeakMemoryCost(), context.localCost.getPeakMemoryWhenOutputting()) == 0);


Doubles.compare delegates to Double.compare. But you don't need it here. It is needed if deterministic comparison with NaN is needed, or if -0 to 0 differentiation is needed. I'm not sure if that's something you expect here. I think just simple context.localCost.getPeakMemoryCost() == context.localCost.getPeakMemoryWhenOutputting() will work here, since NaN == NaN => FALSE, -0 == 0 => TRUE

public static void main(String[] args) { System.out.println("NaN == NaN: " + (NaN == NaN)); System.out.println("-0 == 0: " + (-0.0 == 0.0)); } NaN == NaN: false -0 == 0: true

Also please add a meaningful message in case it ever fails.

I needed something that will tell "OK" also for "NaN vs NaN".

rschlussel · 2018-09-27T15:58:30Z

Just want to be sure I understand the approach correctly: peakMemoryWhenOutputting is how much memory is still in use when moving to the next node. e.g. a join would have all the memory of the build side as the peakMemoryWhenOutputting and that gets included in computing the peak memory of the next node. An exchange would have peakMemoryWhenOutputting of 0, and so the next node would have its cumulative peakMemory essentially reset.

rschlussel · 2018-09-27T16:00:17Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

+
+            if (node.getStep() == AggregationNode.Step.SINGLE || node.getStep() == AggregationNode.Step.FINAL) {
+                return cost.withMemory(
+                        Math.max(context.sourcesCost.getPeakMemoryCost(), context.sourcesCost.getPeakMemoryWhenOutputting() + aggregationMemory),


Can you explain this computation? I don't understand it.

Final/Single agg lifecycle is:

idle, waiting for data

at this stage, it uses no memory, so peak memory of the whole subtree is determined by peak memory for the subplans

consuming input

at this stage, it allocates memory up to aggregationMemory; sources are outputting, so we know that subplans memory usage is no more than getPeakMemoryWhenOutputting(); thus memory usage is no greater than sourcesCost.getPeakMemoryWhenOutputting() + aggregationMemory

producing output

at this stage, it slowly deallocates memory (so memory usage is bounded by aggregationMemory); input is fully consumed so it uses no memory at all; thus memory usage is no greater than aggregationMemory (which in turn is no more than sourcesCost.getPeakMemoryWhenOutputting() + aggregationMemory)

Does this explanation help?

findepi · 2018-09-27T18:30:44Z

Didn't see WIP. Anyhow, since i already spent time on commenting, here are some comments =)

@arhimondr @rschlussel your review is appreciated! getting comments was the very purpose of this PR anyway. I will read thru the comments tomorrow.

findepi · 2018-09-27T22:03:00Z

I am copying the explanation i put in an inline comment (#11591 (comment)), so that it doesn't get hidden by GH.

Sorry for not giving this explanation upfront.

So, the basic idea is as outlined in #11495 (comment): to calculate peak memory usage of a query.
To calculate this recursively, we need to know for every node in a plan tree:

peak memory usage of the subtree
peak memory usage when the node is outputting (producing output) -- this is required to compute peak memory usage of plan tree enclosing given node
- this is based on observation that until given node is outputting, plan nodes higher up need no memory (they have nothing to process)
- also, some nodes will consume all input before outputting, so when they are outputting, their subtrees no longer need memory
the two properties of source nodes -- peak memory usage and peak memory usage when the node is outputting -- combined with node-specific formulas should allow to computing the properties higher up, realizing peak memory usage calculation for the whole plan

Now, the ugly part. In current cost calculation code, PlanNodeCostEstimate conveniently represents two things:

local cost of the node
total cost of the plan subtree

It's no wonder we reuse single class for this, as currently total cost of the plan subtree is just a sum of local costs of all nodes.
After this change, this design no longer fully fits:

when calculating local cost of the node, we should calculate [cpu, memory, network]
when calculating total cost of the plan subtree, we should calculate [cpu, peak memory, network] (this can still be the same type as local cost of the node), but we temporarily need to hold [cpu, peak memory, peak memory when outputting, network]
- it might be that the "extended" type is just an inner thing is CostProvider implementation, so maybe the overall design still holds
  - ... edit: oh it can't. Memo needs to keep this "extended" type as well
summing cost vectors does not always make sense

For now, I didn't address the design part at all. I am just conveniently reusing PlanNodeCostEstimate everywhere.

findepi · 2018-09-27T22:07:42Z

I've reorganized the code a bit. @arhimondr @rschlussel here's how it should be read:

i introduced a new class, PlanCostEstimate, the "extended" type, holding [cpu, peak memory, peak memory when outputting, network]
- this is stored in Memo and CachingCostProvider#cache; CachingCostProvider uses it to do its math
CostCalculators (they are not directly usable by rules or explainer) still calculate local [cpu, memory, network] cost, as PlanNodeCostEstimate
CachingCostProvider (directly usable by rules or explainer) still returns [cpu, memory, network] as PlanNodeCostEstimate, the only change here is that PlanNodeCostEstimate#memory is not some thoughtless sum of local PlanNodeCostEstimate#memory costs of all nodes

findepi · 2018-09-27T22:42:06Z

Note: first commit (Do not return wrong stats/cost for partial aggregations) is from #11595; should not be reviewed here.

mbasmanova · 2018-09-28T14:46:17Z

@findepi Piotr, there is a lot of discussion of the proposed design scattered through the comments. Would you consolidate these into PR description and keep it up-to-date as the discussion evolves. This will allow more robust and careful review of the final design (and assist in reviewing the implementation).

A have a few questions:

What's the difference between memory cost and peak memory of a node?
What's the most important feature this PR is trying to implement? My understanding is the following. A straightforward implementation would compute peak memory for the plan as a total sum of peak memory of each node. However, this will be an overestimation in some cases. It is not yet clear to me which cases these are. I imagine these would be the cases where the plan doesn't allow for streaming from bottom all the way up, e.g. where at different times during plan execution only a subset of nodes can be doing work. E.g. if the plan can be split into parts which execute sequentially and the next part doesn't start until the previous part is complete. Is this so? I'd like to see a specific example of a such a plan and figure out a more rigorous definition of which plans fall into this category and which don't.

Thoughts?

findepi · 2018-09-28T15:22:39Z

What's the most important feature this PR is trying to implement? My understanding is the following. A straightforward implementation would compute peak memory for the plan as a total sum of peak memory of each node.

This is exactly what we used to have.

this will be an overestimation in some cases. It is not yet clear to me which cases these are. I imagine these would be the cases where the plan doesn't allow for streaming from bottom all the way

Yes, this is the case wherever we have kind of long-lived buffering: build side of a join, final/single aggregation, sort, TopN (... ?)

Does it help?

mbasmanova · 2018-09-28T15:49:45Z

@findepi

Yes, this is the case wherever we have kind of long-lived buffering: build side of a join, final/single aggregation, sort, TopN (... ?)

Do you have a few concrete examples? I understand this in principle, but I'm having hard time constructing such a query. I'm also trying to figure out how to scan a lot of query plans and automatically detect potential beneficiaries of this change?

Also, do you happen to know what's the difference between memory cost and peak memory of a node?

haozhun · 2018-09-28T16:15:06Z

I don't have context of this work.

But here is an example. In the following example, peak memory is 2+max(1,3) instead of 1+2+3.

       J(1)
       /  \
      TS J(2)
         /  \
        TS AGG(3)
             |
            TS

rschlussel · 2018-10-03T20:37:48Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

+
+    private static class MemoryEstimate
+    {
+        private final double maxMemoryBeforeOutputting;


Add comments/javadoc about what all these things are.

I would say something like "max memory allocation before node starts outputting". Would that be more helpful?

rschlussel · 2018-10-03T20:42:03Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

+        @Override
+        protected MemoryEstimate visitPlan(PlanNode node, EstimationContext context)
+        {
+            if (context.sourcesCosts.isEmpty()) {


Is this just tablescans and values nodes? I'd rather have explicit methods for those.

Maybe i was just lazy.... OTOH even com.facebook.presto.cost.CostCalculatorUsingExchanges.CostEstimator#visitPlan has "generic" handling (or lack thereof) of many node types.
I will try to get rid of it.

I will try to get rid of it.

Just note: getting rid of this shortcut may lead to having many copy-paste-like methods.

I'm ok leaving it if leaf types would always have this kind of cost, just leave a comment explaining why.

rschlussel · 2018-10-03T20:42:30Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

+            }
+
+            // A PlanNode that is estimated to use memory should have an explicit rule here.
+            verify(isNaN(context.localMaxMemory) || context.localMaxMemory == 0, "Unexpected local memory usage estimation for %s: %s", node, context.localMaxMemory);


what sort of plan shape would get us here? Also, when would you get 0 vs. NaN?

what sort of plan shape would get us here?

I don't understand the question.

Also, when would you get 0 vs. NaN?

NaN is when CostCalculatorUsingExchanges or CostCalculatorWithEstimatedExchanges cannot calculate the local cost of the plan node.
0 is when they can calculate and consider it 0.

I guess I'm trying to understand why all things with 0/NaN cost should have their memory output treated the same, but there shouldn't be a default for things with cost estimates.

If the local cost estimate is NaN, MemoryEstimate should be NaN.

rschlussel · 2018-10-03T20:44:56Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

+            PlanCostEstimate sourcesCost = context.sourcesCosts.stream()
+                    .reduce(PlanCostEstimate.zero(), CachingCostProvider::addSiblingsCost);
+
+            return new MemoryEstimate(


why are these the costs?

Please elaborate?

I was just asking if you could explain where the numbers come from.

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

rschlussel · 2018-10-03T20:56:29Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

+                    .reduce(PlanCostEstimate.zero(), CachingCostProvider::addSiblingsCost);
+
+            return new MemoryEstimate(
+                    sourcesCost.getMaxMemory(),


can you explain this?

presto-main/src/main/java/com/facebook/presto/cost/PlanCostEstimate.java

arhimondr · 2018-10-05T20:37:18Z

@findepi Is this ready to be reviewed?

findepi · 2018-10-08T07:52:15Z

@arhimondr yes, but i am yet to apply some comments.

sopel39

I've got feeling that stas/cost calculations become too tangled in this PR. I think we need to modify CostCalculator so that it computes cumulative cost. Then CostCalculator could also derive peek memory estimation. It seems natural place for such computation instead of CachingCostProvider.

I also think it would be better if we simply added peak memory estimate (possibly package-private) to PlanNodeCostEstimate. Ideally mem and cost estimates should be evaluated by different calculators, but it's not needed now.

sopel39 · 2018-10-08T12:31:50Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

+        PlanCostEstimate cumulativeCost = cache.get(node);
+        if (cumulativeCost == null) {
+            cumulativeCost = calculateCumulativeCost(node);
+            verify(cache.put(node, cumulativeCost) == null, "Cost already set");


how can it be set when you just check cache.get is null

sopel39 · 2018-10-08T12:43:28Z

presto-main/src/main/java/com/facebook/presto/cost/PlanCostEstimate.java

+import static com.google.common.base.Preconditions.checkArgument;
+import static java.lang.Double.isNaN;
+
+public final class PlanCostEstimate


This is artificial class. I think better would be to:

rename it to something more explicit:

PlanNodeMemoryEstimate

that would store just memory estimates

rename it to PlanNodeMemoryEstimate and make it include

PlanNodeCostEstimate otherCosts

Simply store memory estimates as part of PlanNodeCostEstimate

I prefer 3)

sopel39 · 2018-10-08T12:49:17Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

@@ -36,7 +47,7 @@
    private final Session session;
    private final TypeProvider types;

-    private final Map<PlanNode, PlanNodeCostEstimate> cache = new IdentityHashMap<>();


Logically it seems we are missing one explicit calculator for the stuff here to be clean. We should have 3 calcs:

StatsCalculator

PeakMemoryCalculator

CostCalculator

depends on 1) and 2).

depends on 1)

For now I think we can make PlanNodeCostEstimate contain memory estimates also as a package private fields

sopel39 · 2018-10-08T13:07:45Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

+        return new PlanCostEstimate(
+                a.getCpuCost() + b.getCpuCost(),
+                a.getMaxMemory() + b.getMaxMemory(),
+                a.getMaxMemoryWhenOutputting() + b.getMaxMemoryWhenOutputting(),


put NaN here as this value has no meaning

sopel39 · 2018-10-08T13:10:26Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

+            extends PlanVisitor<MemoryEstimate, EstimationContext>
+    {
+        @Override
+        protected MemoryEstimate visitPlan(PlanNode node, EstimationContext context)


context suggest there is some recursion here, but there isn't. Just add fields to PeakMemoryEstimator:

double operatorMemory; List<PlanCostEstimate> sourcesCosts;

or:

PlanCostEstimate operatorCost; List<PlanCostEstimate> sourcesCosts

How would that be any better?

it doesn't suggest there is a recursion, so easier to read and understand.

sopel39 · 2018-10-08T13:15:26Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

    {
-        PlanNodeCostEstimate localCosts = costCalculator.calculateCost(node, statsProvider, session, types);
+        PlanNodeCostEstimate localCost = costCalculator.calculateCost(node, statsProvider, session, types);


To compute cumulative PlanNodeCostEstimate we need to compute PlanCostEstimate.
To compute PlanCostEstimate we need to compute PlanNodeCostEstimate (this time locally) and PlanNodeStatsEstimate.
So to compute PlanNodeCostEstimate (cumulative) we need to compute PlanNodeCostEstimate (individual).

Can we untangle it?
Let's not derive EstimationContext#localMaxMemory from PlanNodeCostEstimate, but rather compute it directly in PeakMemoryEstimator. It seems natural place for such computation

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

@sopel39

@sopel39 please re-review

sopel39

comments not addressed, see: #11591 (comment)

whatever

Non-changes in this commit: - `CostCalculator`s still calculate local `[cpu, memory, network]` cost, as `PlanNodeCostEstimate` - `CachingCostProvider` still returns cumulative `[cpu, memory, network]` cost, as `PlanNodeCostEstimate` Changes in this commit: - a new type is introduced, `PlanCostEstimate` which holds more information than `PlanNodeCostEstimate` - this new type is used by `CachingCostProvider` (the only `CostProvider` implementation) to calculate peak memory usage - this new type is kept in `Memo` as well, since `Memo` serves as storage space for `CachingCostProvider` wherever it encounters a `GroupReference` - `PlanNodeCostEstimate#memory` returned from `CostProvider` is now an estimation of peak memory usage, rather than just a sum of `PlanNodeCostEstimate#memory` cost components of all sub-nodes

sopel39 · 2018-10-19T12:15:29Z

I discussed with @findepi and decided that we could move forward with this PR (as it brings useful feature) with a possibility to extract abstractions later on.

Next step to abstract cost, stats and peak mem calculations could be #11689
Then stats, cost and peak mem can be all considered as "traits". Then trait algebra could be defined:

    Stats[parent] = FStats(Stats[children])
    PeakMem[parent] = FPeakMem(Stats[children], PeakMem[children])
    Cost[parent] = FCost(Stats[parent], Stats[children], PeakMem[parent], Cost[children])

In that regard caching won't be specific to any particular trait.

arhimondr · 2018-12-03T18:52:03Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

+
+        return new PlanCostEstimate(
+                localCost.getCpuCost() + sourcesCost.getCpuCost(),
+                Math.max(sourcesCost.getMaxMemory(), Math.max(memoryEstimate.maxMemoryBeforeOutputting, memoryEstimate.maxMemoryWhenOutputting)),


static import max

arhimondr · 2018-12-03T19:07:05Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

@@ -35,7 +77,7 @@
    private final Session session;
    private final TypeProvider types;

-    private final Map<PlanNode, PlanNodeCostEstimate> cache = new IdentityHashMap<>();
+    private final Map<PlanNode, PlanCostEstimate> cache = new IdentityHashMap<>();


It seems that what is called PlanCostEstimate must be called PlanNodeCostEstimate. And what is currently called PlanNodeCostEstimate must be called PlanCostEstimate

arhimondr · 2018-12-03T19:09:09Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

+        verify(memoryEstimate != null);
+
+        PlanCostEstimate sourcesCost = sourcesCosts.stream()
+                .reduce(PlanCostEstimate.zero(), CachingCostProvider::addSiblingsCost);


Move addSiblingsCost to PlanCostEstimate, and call it sum

arhimondr · 2018-12-03T19:11:21Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

+    private static class EstimationContext
+    {
+        private final double localMaxMemory;
+        private final List<PlanCostEstimate> sourcesCosts;


Seems that MemoryEstimate can (should?) be used here.

arhimondr · 2018-12-03T19:12:30Z

presto-main/src/main/java/com/facebook/presto/cost/CachingCostProvider.java

+        public MemoryEstimate visitTopN(TopNNode node, EstimationContext context)
+        {
+            // TODO
+            return new MemoryEstimate(NaN, NaN);


Does that mean that if there's any TopNNode in the plan - the memory estimate will be unknown?

If so - i'm not sure how useful the current implementation is. Many nodes cannot be processed based on the memory estimates. So, perhaps we need something similar as explained here: #11591 (comment). In other words it looks like this code should be inside of the CostCalculator. I would only change the trait model a bit

Stats[parent] = FStats(Stats[children]) - nothing changes, that how it is implemented right now

Cost[parent] = FCost(Stats[parent], Stats[children], Cost[children])

And that should be CostCalculators responsibility to compute everything, peak-memory/cpu/network cost. So it looks like the CostCalculator#calculateCost interface should accept the CostProvider (similarly to StatsProvider). And than based on child costs the peak memory can be estimated.

facebook-github-bot added the CLA Signed label Sep 27, 2018

arhimondr reviewed Sep 27, 2018

View reviewed changes

rschlussel reviewed Sep 27, 2018

View reviewed changes

findepi changed the title ~~Peak memory cost calculations [WIP]~~ Peak memory cost calculation Sep 27, 2018

sopel39 mentioned this pull request Oct 1, 2018

Quantify stats/cost calculator quality. Probabilistically adjust models as a longer term goal #11615

Closed

3 tasks

findepi mentioned this pull request Oct 3, 2018

Test EXPLAIN for all queries [WIP] #11596

Closed

rschlussel reviewed Oct 3, 2018

View reviewed changes

kokosing reviewed Oct 4, 2018

View reviewed changes

sopel39 previously requested changes Oct 8, 2018

View reviewed changes

mbasmanova added the cbo label Oct 9, 2018

sopel39 previously requested changes Oct 10, 2018

View reviewed changes

sopel39 mentioned this pull request Oct 10, 2018

Make CostCalculator to calculate cumulatively #11689

Closed

arhimondr reviewed Dec 3, 2018

View reviewed changes

sopel39 mentioned this pull request Jan 25, 2019

Allow connectors to participate in query optimization trinodb/trino#18

Open

26 tasks

findepi mentioned this pull request Jan 30, 2019

Calculate query peak memory usage as memory cost trinodb/trino#108

Closed

findepi closed this Jan 30, 2019

findepi deleted the findepi/peak-memory branch January 30, 2019 13:30

findepi mentioned this pull request Feb 11, 2019

Make CostCalculator calculate "cumulative cost" [WIP] trinodb/trino#210

Closed

wenleix mentioned this pull request Mar 5, 2019

[Design] Exchange Materialization #12387

Closed

Peak memory cost calculation #11591

Peak memory cost calculation #11591

Conversation

findepi commented Sep 27, 2018 • edited Loading

arhimondr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

findepi Sep 27, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rschlussel commented Sep 27, 2018

Choose a reason for hiding this comment

findepi Sep 27, 2018 • edited Loading

Choose a reason for hiding this comment

findepi commented Sep 27, 2018

findepi commented Sep 27, 2018

findepi commented Sep 27, 2018

findepi commented Sep 27, 2018

mbasmanova commented Sep 28, 2018

findepi commented Sep 28, 2018

mbasmanova commented Sep 28, 2018

haozhun commented Sep 28, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arhimondr commented Oct 5, 2018

findepi commented Oct 8, 2018

sopel39 left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sopel39 left a comment • edited Loading

Choose a reason for hiding this comment

sopel39 commented Oct 19, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

findepi commented Sep 27, 2018 •

edited

Loading

findepi Sep 27, 2018 •

edited

Loading

findepi Sep 27, 2018 •

edited

Loading

sopel39 left a comment •

edited

Loading

sopel39 left a comment •

edited

Loading

sopel39 commented Oct 19, 2018 •

edited

Loading