Skip to content

Implement semiring algebra layer for generic path computation and graph analysis#990

Merged
SkBlaz merged 8 commits intomasterfrom
copilot/implement-semiring-functionality
Jan 6, 2026
Merged

Implement semiring algebra layer for generic path computation and graph analysis#990
SkBlaz merged 8 commits intomasterfrom
copilot/implement-semiring-functionality

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Jan 5, 2026

Adds a comprehensive semiring framework enabling generic path algorithms (shortest paths, reachability, reliability, etc.) parameterized by algebraic operations. Integrates with DSL v2, multilayer networks, and layer algebra.

Core Implementation

New package: py3plex/algebra/ (8 modules, 800+ LOC)

  • Semiring protocol with 4 built-ins: boolean (reachability), min_plus (shortest paths), max_times (reliability), max_plus (longest paths)
  • Generic path solvers: SSSP/APSP with automatic algorithm selection (Dijkstra for idempotent+monotone semirings, Bellman-Ford fallback)
  • Weight lifting: Flexible edge attribute extraction with transformations (log, custom functions)
  • Transitive closure: Floyd-Warshall and iterative methods
  • Backend dispatch: Graph backend (implemented), matrix backend (placeholder)
  • Registry: Deterministic semiring registration and discovery

DSL v2 Integration

S builder namespace:

from py3plex.dsl import S, L

# Shortest paths (min-plus semiring)
result = S.paths().from_node("Alice").to_node("Bob") \
    .semiring("min_plus").lift(attr="weight", default=1.0) \
    .from_layers(L["social"] + L["work"]) \
    .crossing_layers(mode="penalty", penalty=10.0) \
    .witness(True).execute(network)

# Reachability (boolean semiring)
result = S.paths().from_node("Alice") \
    .semiring("boolean").execute(network)

# Transitive closure
result = S.closure().semiring("boolean") \
    .from_layers(L["social"]).execute(network)

Features:

  • Layer algebra integration (from_layers(L[...]))
  • Cross-layer edge policies: allowed, forbidden, penalty
  • Path witness tracking for reconstruction
  • Per-layer semiring mapping (infrastructure ready)

Provenance & Determinism

All queries include comprehensive provenance metadata:

  • Semiring properties (idempotent, monotone, etc.)
  • Weight lifting specification
  • Algorithm selection rationale
  • Performance metrics
  • Determinism guarantees

Stable ordering: Sorted layer names, sorted node IDs, deterministic tie-breaking via semiring.better().

AST Extensions

New nodes in py3plex/dsl/ast.py:

  • SemiringSpecNode - Semiring configuration
  • WeightLiftSpecNode - Edge weight extraction
  • SemiringPathStmt, SemiringClosureStmt - Query statements
  • CrossingLayersSpec - Multilayer edge handling

Executor in py3plex/dsl/executor_semiring.py handles S builder execution and provenance generation.

Documentation

Updated AGENTS.md (+397 lines, section 28):

  • Semiring theory and applications
  • Complete API reference with 4 built-in semirings
  • Multilayer integration patterns
  • Custom semiring registration
  • Performance and algorithm selection notes

Example: examples/network_analysis/example_semiring_algebra.py demonstrates shortest paths, reachability, reliability, closure, and layer filtering.

Testing

  • 23 tests: Semiring laws (identity, idempotence, properties)
  • 17 tests: Path correctness (min_plus, boolean, max_times)
  • 13 tests: Integration with multilayer networks
  • 21 property tests: Hypothesis-based property testing for algebraic laws, weight lifting, path algorithms, closure, registry, and better() function

All core functionality passes. Minor QueryResult.to_pandas() limitations for custom targets noted but non-blocking.

Property-Based Testing

Added comprehensive property tests using Hypothesis (tests/property/test_algebra_properties.py):

  • Semiring laws: additive identity, multiplicative identity, idempotence, commutativity, zero annihilator (100+ examples per test)
  • Weight lifting: attribute extraction, defaults, log transform, custom transforms
  • Path algorithms: source distance correctness, determinism, reachability transitivity, max_hops constraint
  • Closure: reflexivity, direct edges preservation, determinism
  • Registry: sorted listing, built-in registration, consistency
  • Better function: irreflexive and asymmetric properties

Design Principles

  • Minimal API: Single S builder entry point
  • Type-safe: Full type hints throughout
  • Deterministic: Reproducible results with stable ordering
  • Performance: O(E log V) for Dijkstra, O(V·E) for Bellman-Ford, automatic selection
  • Extensible: Plugin custom semirings via registry

Backward Compatibility

No breaking changes. P.shortest() can delegate to semiring engine internally (infrastructure ready). All existing DSL v2 tests unaffected.

Original prompt

This section details on the original issue you should resolve

<issue_title>semirings</issue_title>
<issue_description>You are implementing a comprehensive semiring/algebra layer in py3plex.

CRITICAL DOC RULES

  • Do NOT add any new .md files.
  • You MUST update the existing AGENTS.md in-place to document the new semiring functionality (and keep it accurate + consistent with the code).
  • Keep documentation concise but complete; add a new section and integrate with existing DSL v2 / provenance sections.

SCOPE
Implement a semiring framework that integrates with:

  • DSL v2 builder/AST/executor
  • multilayer layer algebra (L[...]+L[...])
  • temporal windows (where applicable)
  • QueryResult + provenance
  • optional internal parallelization (do not break determinism)
  • backward compatibility: existing APIs (esp. P.shortest) keep working; semiring path engine can power them internally.

NON-GOALS

  • No user-facing tutorial overhaul. Backend first.
  • No performance regression for non-semiring code paths.

================================================================================
A) IMPLEMENTATION PLAN (CODE)

A1) Create new algebra package (no docs files)

  • Create directory: py3plex/algebra/
    • semiring.py (Semiring protocol + built-in semirings)
    • registry.py (register/get/list semirings)
    • lift.py (weight lifting spec + helpers)
    • paths.py (generic semiring path solver: SSSP/APSP/k-best optional)
    • closure.py (Kleene star / transitive closure over semiring)
    • fixed_point.py (monotone fixed-point engine for complete/idempotent semirings)
    • backend.py (backend dispatch: "graph" default, optional "matrix" later)
    • witness.py (optional witness tracking structures)

A2) Semiring core
TODO: Define Semiring interface (Protocol or ABC)

  • Required:
    • name: str
    • add(a,b) # ⊕
    • mul(a,b) # ⊗
    • zero() # additive identity
    • one() # multiplicative identity
  • Optional:
    • better(a,b)->bool for argmin/argmax tie-break + k-best
    • props metadata dict (commutative_add, idempotent_add, monotone, complete, closed, supports_k_best)
    • stable_repr / serialize() helper (for provenance)

TODO: Implement built-in semirings with correct props

  • boolean: add=or, mul=and, zero=False, one=True; idempotent_add=True, commutative_add=True
  • min_plus (tropical): add=min, mul=+, zero=+inf, one=0; idempotent_add=True, monotone=True
  • max_plus: add=max, mul=+, zero=-inf, one=0
  • max_times: add=max, mul=*, zero=0, one=1
  • logsumexp_plus (optional): add=logsumexp, mul=+ in log-domain, zero=-inf, one=0
    Add unit tests for identity laws and basic behavior.

A3) Semiring registry
TODO: Implement registry with safe overrides

  • register_semiring(name, semiring, overwrite=False)
  • get_semiring(name) -> Semiring (raise clear error if missing)
  • list_semirings() -> list[str]
    Ensure deterministic ordering in list_semirings().

A4) Weight lifting
TODO: Implement WeightLiftSpec (serializable dataclass)
Fields:

  • attr: str | None
  • transform: None | "log" | callable | str-keyed transform
  • default: Any
  • on_missing: "drop" | "fail" | "default"
  • validate: check attr exists when possible
    Provide helpers:
  • lift_edge_value(edge_attr_dict, spec) -> semiring element
  • parse shorthand: lift(weight="weight") => WeightLiftSpec(attr="weight", default=1.0, on_missing="default")

A5) Multilayer integration assumptions
TODO: Define how an “edge” is represented for algebra engine:

  • Must support retrieving:
    • source node id + source layer
    • target node id + target layer
    • edge attributes (weight, p, etc.)
  • Must support layer filtering via L expressions (existing DSL semantics)
    Implement adapter functions in algebra layer that can iterate edges efficiently given:
  • included layers set (or layer pairs)
  • crossing_layers policy (allowed / penalty / forbidden)
    Do NOT change core network representation unless necessary; prefer adapters.

A6) Generic semiring path engine (graph backend)
TODO: Implement core relax/DP engine in py3plex/algebra/paths.py
Features:

  • SSSP: from a single source to all nodes; optionally stop at target
  • hop constraint: max_hops (optional)
  • k-best (optional): only for semirings with supports_k_best OR when better() exists (define minimal contract)
    Algorithm selection based on semiring props:
  • If idempotent_add and monotone (and edge weights non-negative in min-plus analogue), use Dijkstra-like:
    • priority queue keyed by current best element using semiring.better/order
    • Must define comparator strategy:
      • For min_plus: smaller is better
      • For max_*: larger is better
      • For custom semirings: require better() or order key
  • Else fallback to Bellman-Ford style relaxation:
    • iterate |V|-1 times or until no changes
    • if complete/idempotent: can also do fixed-point with convergence criteria
      Return:
  • best distances/values per node
  • optionally p...

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 6 commits January 5, 2026 19:08
…fting

Co-authored-by: SkBlaz <10035780+SkBlaz@users.noreply.github.com>
Co-authored-by: SkBlaz <10035780+SkBlaz@users.noreply.github.com>
Co-authored-by: SkBlaz <10035780+SkBlaz@users.noreply.github.com>
Co-authored-by: SkBlaz <10035780+SkBlaz@users.noreply.github.com>
Co-authored-by: SkBlaz <10035780+SkBlaz@users.noreply.github.com>
Co-authored-by: SkBlaz <10035780+SkBlaz@users.noreply.github.com>
Copilot AI changed the title [WIP] Implement semiring framework in py3plex Implement semiring algebra layer for generic path computation and graph analysis Jan 5, 2026
Copilot AI requested a review from SkBlaz January 5, 2026 19:23
@SkBlaz
Copy link
Copy Markdown
Owner

SkBlaz commented Jan 5, 2026

@copilot missing property tests

- Add 21 property tests covering semiring laws, weight lifting, path algorithms, and closure
- Test algebraic properties: additive identity, multiplicative identity, idempotence, commutativity, zero annihilator
- Test weight lifting with various transforms and edge cases
- Test path algorithm properties: determinism, correctness, max_hops
- Test closure properties: reflexivity, transitivity, determinism
- Test semiring registry properties
- Test better() function properties: irreflexive, asymmetric
- All tests use Hypothesis for property-based testing with 100+ examples per test

Co-authored-by: SkBlaz <10035780+SkBlaz@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Jan 5, 2026

@copilot missing property tests

Added comprehensive property-based tests for semiring algebra in commit d4c0484.

The new test file tests/property/test_algebra_properties.py includes 21 property tests using Hypothesis covering:

  • Semiring laws (5 tests): additive identity, multiplicative identity, idempotence, commutativity, zero annihilator
  • Weight lifting (4 tests): attribute extraction, defaults, log transform, custom transforms
  • Path algorithms (4 tests): source distance, determinism, reachability transitivity, max_hops constraint
  • Closure (3 tests): reflexivity, direct edges preservation, determinism
  • Registry (3 tests): sorted listing, built-in registration, consistency
  • Better function (2 tests): irreflexive, asymmetric properties

All 21 tests pass with 100+ examples per test.

@SkBlaz SkBlaz marked this pull request as ready for review January 6, 2026 00:50
@SkBlaz SkBlaz merged commit 714e108 into master Jan 6, 2026
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

semirings

2 participants