Skip to content

perf(gfql): CPU PageRank spends ~95% of its time in to_igraph conversion and join-back, not the solver #2032

Description

@lmeyerov

On the LiveJournal graph (4.8M nodes, 69M edges, polars frames) a call('pagerank') on the CPU path takes 52.6 s of which the igraph solver (igraph.Graph.pagerank) is 2.8 s. The remaining ~50 s is to_igraph (frame to igraph edge list, node re-indexing) plus joining the scores back onto the node frame.

Measured 2026-09-04 on dgx-spark under the receipted-lane protocol (private pyg-bench GraphFrames ladder, lj-polars-pagerank; solver time from the runner's phase receipt). Numbers here are diagnostic, not the published cells.

Expected: conversion should be a small constant factor over the solver, i.e. a few seconds, not 18x the solver.

Suspects (not yet profiled at function level):

  • building the igraph graph from a Python-level edge iterable instead of a contiguous integer array pair (Graph(n, edges=...) with edges as a 2-column numpy array),
  • id remapping through pandas factorize/merge on the full node table instead of a single dense mapping,
  • the join-back of scores merging on the id column rather than positional assignment when the node order is preserved.

Acceptance: a pyg-bench receipt where to_igraph + join-back is under 2x the solver on LJ, and the CPU PageRank rung re-run under the same protocol.

Related: plans/gfql-benchmark-numbers/plan.md T2.10.6.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions