feat: implement CSV loader and register 5 SNAP datasets

[DecisionNerd]

Summary

Implements CSV edge-list loader and registers 5 SNAP (Stanford Network Analysis Project) datasets.

Changes

Core Implementation

• CSVLoader (src/graphforge/datasets/loaders/csv.py):

  • Auto-delimiter detection (tab, comma, space)
  • Gzip compression support (.gz files)
  • Comment line handling (lines starting with #)
  • Empty line skipping
  • Weighted and unweighted edges
  • Node deduplication via cache
  • Comprehensive error handling

• SNAP Dataset Registration (src/graphforge/datasets/sources/snap.py):

  • snap-ego-facebook (4K nodes, 88K edges)
  • snap-email-enron (37K nodes, 184K edges)
  • snap-ca-astroph (19K nodes, 198K edges)
  • snap-web-google (876K nodes, 5.1M edges)
  • snap-twitter-combined (81K nodes, 1.7M edges)

• Auto-Registration: Datasets registered automatically on module import

Testing

• ✅ 14 CSV loader unit tests
• ✅ 13 SNAP dataset unit tests
• ✅ All existing tests passing (1107 passed, 14 skipped)
• ✅ 95.69% total code coverage maintained

Code Quality

• Fixed linting issues:
  • PTH123: Use Path.open() instead of open()
  • PLW2901: Avoid overwriting loop variables
  • E501: Line length compliance
• Added test fixture to handle registry isolation

Examples

from graphforge import GraphForge
from graphforge.datasets import list_datasets, load_dataset

# List available SNAP datasets
snap_datasets = list_datasets(source="snap")
for ds in snap_datasets:
    print(f"{ds.name}: {ds.nodes} nodes, {ds.edges} edges")

# Load a dataset
gf = GraphForge.from_dataset("snap-ego-facebook")

# Or load into existing instance
gf = GraphForge()
load_dataset(gf, "snap-email-enron")

# Query the graph
result = gf.execute("""
    MATCH (n)-[r:CONNECTED_TO]->(m)
    RETURN count(r) as edge_count
""")

Testing

All pre-push checks passing:

• ✅ Linting (ruff format + check)
• ✅ Type checking (mypy strict)
• ✅ 1107 tests passed, 14 skipped
• ✅ Coverage: 95.69%

Related

Part of v0.2.1 dataset infrastructure implementation.

Follows PR #68 (dataset loading infrastructure).

Summary by CodeRabbit

• New Features

  • CSV/TSV/space-delimited loader with automatic delimiter detection, weighted-edge support, node caching, and gzip handling.
  • Five SNAP datasets added: Facebook ego, Enron emails, AstroPh collaboration, Google web graph, and Twitter ego; these datasets become available automatically on initialization.

• Tests

  • Extensive tests for CSV loader behavior (formats, weights, compression, errors) and SNAP dataset metadata/registration.

[coderabbitai]

Walkthrough

Imports now auto-register SNAP datasets at package import; adds a CSVLoader for edge-list CSV/TSV/space-delimited (gzipped supported) files and unit tests for both CSV loading and SNAP dataset registration.

Changes

Cohort / File(s)	Summary
CSV Loader Implementation src/graphforge/datasets/loaders/csv.py	Adds CSVLoader: loads edge lists with auto-delimiter detection, supports comments, optional weights, gzipped files, node caching/deduplication, _load_edges, _detect_delimiter, and get_format() returning "csv".
SNAP Dataset Registration src/graphforge/datasets/sources/snap.py	Adds register_snap_datasets() which idempotently ensures the CSV loader is registered and registers five SNAP datasets with metadata (name, source, URL, counts, category, license, loader_class).
Package Initialization src/graphforge/datasets/__init__.py	Now imports and immediately calls register_snap_datasets() on package import, causing import-time registration side-effect.
Test Coverage tests/unit/datasets/test_csv_loader.py, tests/unit/datasets/test_snap_datasets.py	New unit tests: CSV loader behavior (delimiters, weights, gzip, comments, node dedup, errors, get_format) and SNAP dataset registration/metadata, URL format, loader assignment, and filtering.

Sequence Diagram

sequenceDiagram
    participant Import as "datasets/__init__.py"
    participant Reg as "register_snap_datasets()"
    participant Registry as "Global Registry (loaders & datasets)"
    participant CSV as "CSVLoader"

    Import->>Reg: call register_snap_datasets()
    Reg->>Registry: ensure loader registered ("csv")
    Registry-->>Reg: loader registered (or already present)
    Reg->>Registry: register dataset (snap-ego-facebook, metadata)
    Reg->>Registry: register dataset (snap-email-enron, metadata)
    Reg->>Registry: register dataset (snap-ca-astroph, metadata)
    Reg->>Registry: register dataset (snap-web-google, metadata)
    Reg->>Registry: register dataset (snap-twitter-combined, metadata)
    Registry-->>Import: datasets and loader available

Loading

Estimated Code Review Effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related issues

• Register additional SNAP datasets #70: Adds register_snap_datasets and related loader registration, aligning with the issue's goal to expand/refactor SNAP dataset registration.

Possibly related PRs

• feat: implement core dataset loading infrastructure #68: Introduced the dataset registry/loader abstractions and layout that this PR builds upon.

Suggested labels

v0.3

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	Title clearly summarizes the main changes: CSV loader implementation and SNAP dataset registration, which aligns with the primary objectives.
Description check	✅ Passed	Description provides comprehensive details covering implementation, testing, code quality, examples, and related context. Includes type of change (✨ New feature), test coverage, and checklist items are satisfied.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch feat/csv-loader-snap-datasets

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

Codecov Report

❌ Patch coverage is 94.02985% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.86%. Comparing base (3f39b82) to head (3111a65).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main      #69   +/-   ##
=======================================
  Coverage   93.86%   93.86%           
=======================================
  Files          17       19    +2     
  Lines        2134     2201   +67     
  Branches      526      542   +16     
=======================================
+ Hits         2003     2066   +63     
- Misses         49       52    +3     
- Partials       82       83    +1     

Flag	Coverage Δ
full-coverage	93.86% <94.02%> (+<0.01%)	⬆️
unittests	75.96% <94.02%> (+1.50%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
parser	95.31% <ø> (ø)
planner	94.20% <ø> (ø)
executor	89.87% <ø> (ø)
storage	99.50% <ø> (ø)
ast	100.00% <ø> (ø)
types	98.42% <ø> (ø)

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3f39b82...3111a65. Read the comment docs.

[coderabbitai]

Actionable comments posted: 2

🤖 Fix all issues with AI agents

In `@src/graphforge/datasets/loaders/csv.py`:
- Around line 80-95: The parser currently uses stripped_line.split(delimiter)
which, when delimiter is a single space, yields empty tokens for consecutive
spaces; update the parsing logic in the CSV loader (around
self._detect_delimiter, delimiter, and the block that assigns
source_id/target_id) so that if delimiter == " " you call stripped_line.split()
(no argument) to collapse consecutive whitespace before extracting source_id and
target_id, and keep the existing error handling; also add a unit test for the
CSV loader that parses a line with multiple consecutive spaces (e.g., "0  1 
0.5") to ensure no empty node IDs are produced.

In `@src/graphforge/datasets/sources/snap.py`:
- Around line 13-19: Wrap the call to register_loader("csv", CSVLoader) inside a
try/except in register_snap_datasets so a duplicate-registration error doesn't
abort the rest of the function; specifically, call register_loader("csv",
CSVLoader) in a try block and catch the duplicate-loader exception (e.g.,
DuplicateLoaderError or the specific exception your registry throws) and
silently ignore it (or log.debug) then continue with the dataset registration
logic so datasets still get registered; re-raise any other unexpected
exceptions.

[DecisionNerd]

Additional Improvements

CSV Loader Enhancements

• Fixed consecutive whitespace handling: Now uses split() without arguments when delimiter is a space, which properly collapses consecutive spaces
• Added test coverage: New test_load_multiple_consecutive_spaces() test verifies parsing of data with multiple consecutive spaces and weights
• Prevents empty node IDs: Ensures malformed data with excessive spacing doesn't create empty node IDs

SNAP Registration Robustness

• Idempotent registration: register_snap_datasets() now safely handles being called multiple times
• Graceful error handling: Both loader and dataset registration wrapped in try-except blocks to handle duplicate registration
• Simplified test fixtures: No longer need error handling in test setup since registration is idempotent

Testing

• ✅ 1108 tests passing (added 1 new test)
• ✅ Coverage: 95.59%
• ✅ All pre-push checks passing

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@src/graphforge/datasets/sources/snap.py`:
- Around line 100-103: The `# noqa: PERF203` on the except clause is unused and
should be removed to satisfy strict linting; edit the try/except block that
calls register_dataset(dataset) (the except ValueError as e: handler) and delete
the `# noqa: PERF203` comment so the except line reads simply `except ValueError
as e:` (no other changes needed).