Skip to content

fix: avoid memory overflow in ensure_no_na() for large sparse matrices with mode="lower" or mode="upper"#2627

Open
Copilot wants to merge 3 commits intomainfrom
copilot/fix-memory-overflow-sparse-matrices
Open

fix: avoid memory overflow in ensure_no_na() for large sparse matrices with mode="lower" or mode="upper"#2627
Copilot wants to merge 3 commits intomainfrom
copilot/fix-memory-overflow-sparse-matrices

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 6, 2026

ensure_no_na() called upper.tri(x) / lower.tri(x) on sparse matrices, which materializes a full dense n×n logical matrix. For a ~1.2M×1.2M sparse matrix this requires ~5.4 TB of memory, crashing before any actual NA check occurs.

Changes

  • R/utils-assert-args.Rensure_no_na(): For sparse matrices, use Matrix::tril(x)@x / Matrix::triu(x)@x instead of the dense triangular extraction. This operates in O(nnz) memory by working only on explicitly stored values.

  • R/adjacency.Rgraph.adjacency.sparse(): Use Matrix::nnzero(adjmatrix, na.counted = TRUE) for the early-exit empty-graph check. Without this, a sparse matrix with any NA stored values causes nnzero() to return NA, which propagates into the if condition and throws "missing value where TRUE/FALSE needed".

  • tests/testthat/test-adjacency.R: Tests added for sparse matrices with NAs confined to one triangle — the relevant mode should error, the opposite mode should succeed.

# Before fix: crashes with "cannot allocate vector of size 5426.7 Gb"
sparse_mat <- Matrix::rsparsematrix(1206945, 1206945, density = 1e-6)
graph_from_adjacency_matrix(sparse_mat, mode = "lower", weighted = TRUE)

# After fix: works correctly
  • By submitting this pull request, I assign the copyright of my contribution to The igraph development team.

Copilot AI and others added 2 commits May 6, 2026 05:40
…with mode=lower/upper

Agent-Logs-Url: https://github.com/igraph/rigraph/sessions/a080274e-c869-44c6-b491-4748a81bfc9c

Co-authored-by: schochastics <17147355+schochastics@users.noreply.github.com>
Agent-Logs-Url: https://github.com/igraph/rigraph/sessions/a080274e-c869-44c6-b491-4748a81bfc9c

Co-authored-by: schochastics <17147355+schochastics@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix memory overflow in ensure_no_na for large sparse matrices fix: avoid memory overflow in ensure_no_na() for large sparse matrices with mode="lower" or mode="upper" May 6, 2026
Copilot AI requested a review from schochastics May 6, 2026 05:42
@schochastics schochastics marked this pull request as ready for review May 6, 2026 06:46
Copy link
Copy Markdown
Contributor

@schochastics schochastics left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ensure_no_na causes memory overflow for large sparse matrices when mode = "lower" or "upper" (igraph = 2.3.1)

2 participants