Skip to content

Compare performance: Detect 'shadowed' cache hits and a few other fixes #4018

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Apr 30, 2025
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Store dependencies of predicates
  • Loading branch information
asgerf committed Apr 25, 2025
commit 6329d239edf16fc9582c7dc082af03b17d386d38
2 changes: 1 addition & 1 deletion extensions/ql-vscode/src/log-insights/log-summary.ts
Original file line number Diff line number Diff line change
@@ -29,6 +29,7 @@ interface SummaryEventBase {

interface ResultEventBase extends SummaryEventBase {
resultSize: number;
dependencies?: { [key: string]: string };
}

export interface ComputeSimple extends ResultEventBase {
@@ -67,7 +68,6 @@ export interface NamedLocal extends ResultEventBase {
ra: Ra;
pipelineRuns: PipelineRun[];
queryCausingWork?: string;
dependencies: { [key: string]: string };
predicateIterationMillis: number[];
}

11 changes: 11 additions & 0 deletions extensions/ql-vscode/src/log-insights/performance-comparison.ts
Original file line number Diff line number Diff line change
@@ -56,6 +56,9 @@ export interface PerformanceComparisonDataFromLog {
* All the pipeline runs seen for the `i`th predicate from the `names` array.
*/
pipelineSummaryList: Array<Record<string, PipelineSummary>>;

/** All dependencies of the `i`th predicate from the `names` array, encoded as a list of indices in `names`. */
dependencyLists: number[][];
}

export class PerformanceOverviewScanner implements EvaluationLogScanner {
@@ -157,8 +160,10 @@ export class PerformanceOverviewScanner implements EvaluationLogScanner {
iterationCounts,
evaluationCounts,
pipelineSummaryList,
dependencyLists,
} = this.data;
const pipelineSummaries = pipelineSummaryList[index];
const dependencyList = dependencyLists[index];
for (const { counts, raReference } of event.pipelineRuns ?? []) {
// Get or create the pipeline summary for this RA
const pipelineSummary = (pipelineSummaries[raReference] ??= {
@@ -178,6 +183,12 @@ export class PerformanceOverviewScanner implements EvaluationLogScanner {
totalTuplesPerStep[i] += count;
}
}
for (const dependencyHash of Object.values(event.dependencies ?? {})) {
const dependencyIndex = this.raToIndex.get(dependencyHash);
if (dependencyIndex != null) {
dependencyList.push(dependencyIndex);
}
}
timeCosts[index] += totalTime;
tupleCosts[index] += totalTuples;
iterationCounts[index] += event.pipelineRuns?.length ?? 0;