Skip to content

[integration/peerlab] Bugfix/torch array overlow#45

Merged
Tobiaspk merged 3 commits into
integration/peerlabfrom
bugfix/torch_array_overlow
May 19, 2026
Merged

[integration/peerlab] Bugfix/torch array overlow#45
Tobiaspk merged 3 commits into
integration/peerlabfrom
bugfix/torch_array_overlow

Conversation

@Tobiaspk
Copy link
Copy Markdown
Collaborator

@Tobiaspk Tobiaspk commented May 18, 2026

Closes #44 . Subsetting a graph with over 2.1B edges is not supported in torch 2.5.1 (which is often the version it's resolved to). Instead, use indices instead of boolean arrays to subset the graph. These can be obtained using chunks.

Also improve logging logic.

@Tobiaspk Tobiaspk changed the base branch from main to integration/peerlab May 18, 2026 22:12
@Tobiaspk
Copy link
Copy Markdown
Collaborator Author

Tobiaspk commented May 18, 2026

Attached is the script used to test if the new chunking logic yields identical results to the previous implementatino.

test_new_subsetting_logic.py

Outputs:

Loading HeteroData and Tiles...
#tiles: 13
#nodes: 0.3M
#edges: 1.5M
Splitting mask of size 280378 into 1402 chunks...
Checking if new and previous subsets are equal...
Node type tx matches.
Node type bd matches.

@andrewmoorman FYI, will merge tomorrow if you see no issues. :)

@Tobiaspk Tobiaspk changed the title Bugfix/torch array overlow [integration/peerlab] Bugfix/torch array overlow May 18, 2026
@Tobiaspk Tobiaspk force-pushed the bugfix/torch_array_overlow branch from 07baac1 to a134e17 Compare May 19, 2026 19:41
@Tobiaspk
Copy link
Copy Markdown
Collaborator Author

Prediction graph now works. While performance for predicting is very slow now, this fix solved the issue.

@Tobiaspk Tobiaspk merged commit c4aec63 into integration/peerlab May 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[integration/peerlab] Atera Dataset exceed torch array size

1 participant