Block (scc) expansion

[daemontus]

This PR builds on #118 and introduces a new "scc-like" expansion algorithm that is more efficient and (hopefully) easier to follow and prove.

Main advantages of the new approach:

1. In SCC expansion, when we "recreate" the SCC-diagram inside the "main" diagram, we are only creating expanded nodes. I.e. it no longer holds that if a node is expanded, its successor nodes are the same as in the full succession diagram. The new approach satisfies that each node is either fully expanded, or not expanded at all. (We could probably fix this in SCC expansion if we wanted to, but here it holds by design)
2. In SCC expansion, we are not guaranteed to uncover the smallest subspace for every MAA attractor. Technically, we don't miss any MAAs, but that's only because of (1) (the more specific unexpanded nodes are missing completely from the SD, hence the MAA is found in the less-specific subspace instead). This expansion should always find the minimal superspace for each MAA, if desired. However, this might have been a property in the SCC expansion before and I just broke it in Refactor SCC expansion #118?
3. In SCC expansion, we use another algorithm to expand the inner SCC-diagrams. We can recursively use the SCC expansion here to obtain smaller diagrams, but overall this is a bit "over-engineered". In the new approach, the expansion condition is local to each node, thus no need for inner diagrams or recursion.
4. SCC expansion can become inefficient if the source SCCs don't have any stable motifs (e.g. all are oscillating inputs). Here, we instead use "source blocks" which have the same effect but can incorporate source SCC w/o stable motifs.

Performance TL;DR:

• Technically, due to (1), this creates more SD nodes than the SCC expansion. However, the number of expanded nodes (i.e. those that actually matter) is in general smaller or equivalent. Specifically:
  • BBM-002, SCC expansion: 243; Block expansion: 174 expanded (599 all);
  • BBM-079, SCC expansion: 915; Block expansion: 115 expanded (655 all);
  • BBM-034, SCC expansion: 2586; Block expansion: 386 expanded (838 all);
  • BBM-143, SCC expansion: 529; Block expansion: 529 expanded (793 all);
• Overall, the method seems to be a slightly faster, but this might just be the reduced overhead due to not needing the recursive succession diagrams.

How does it work:

1. It uses the same BFS-style exploration as SCC-expansion.
2. When a node is considered, it is first fully expanded (obtaining all successors).
3. For each successor, we obtain its stable motif and compute its "block", i.e. a backward-closed superset of variables that are fixed by the stable motif. In most typical cases, a "block" is a source SCC, but it can also contain multiple SCCs if there is a stable motif outside of a source SCC.
4. Successors nodes are grouped based on the blocks in which their stable motif appears. We also disregard any inclusion non-minimal blocks, as these are not the "source" blocks (i.e. the stable motifs in those depend on other, smaller blocks).
5. If we don't care about MAAs, we simply pick the block with the least amount of motifs and continue expanding all nodes corresponding to this block. We know that the remaining blocks are "independent" (similar to how source SCCs are independent).
6. If we care about MAAs, we pick a block for which we can prove the corresponding sub-network has no MAAs in its root node. This block is safe to expand and won't have any MAAs, because every state in the "full" network can replicate a path in the sub-network that eventually leaves the sub-network's root node. This essentially "defers" the expansion of MAA trap spaces until the last possible moment. If no such block exists (all source blocks have MAAs), we expand the node completely, since the MAA(s) can be anywhere.

Other minor notes:

• The simulation limit now depends on the size of the network. Previously this was constant, hence some small networks were taking a relatively long time to finish because they were "stuck" in simulation even though the problem could have been quickly solved by reachability. Now we will only use these extreme limits for very large networks where reachability is unlikely to help.

[github-actions]

Coverage Report

File	Stmts	Miss	Cover	Missing
biobalm
_pint_reachability.py	61	50	18%	24, 40–54, 69–93, 101–146
control.py	114	14	88%	107, 119, 125, 129, 134, 143–159, 477, 480, 493
interaction_graph_utils.py	52	6	88%	11–13, 151–152, 222–223
petri_net_translation.py	149	11	93%	22–26, 79, 136, 308–309, 333–334, 343, 452
space_utils.py	132	20	85%	26–28, 104–110, 133–139, 347–350, 414, 462
succession_diagram.py	379	65	83%	6, 120, 210–215, 228, 275–282, 386–393, 410–411, 421, 427, 543, 630–636, 752, 755, 873–891, 923, 933, 936, 976, 983, 1034, 1048, 1170, 1329, 1340, 1348, 1376, 1391, 1403, 1408, 1414
symbolic_utils.py	32	5	84%	10, 39–44, 100, 128
trappist_core.py	184	27	85%	14–18, 55, 57, 92, 215, 217, 219, 247–250, 254–256, 276–282, 340, 342, 372, 420, 422
biobalm/_sd_algorithms
expand_attractor_seeds.py	60	7	88%	6, 28, 42, 109–114, 119
expand_bfs.py	28	1	96%	6
expand_dfs.py	30	1	97%	6
expand_minimal_spaces.py	37	2	95%	6, 31
expand_source_SCCs.py	111	16	86%	11–13, 50, 69, 77, 82, 103, 112, 120, 131, 140, 143, 167, 179, 242–243
expand_source_blocks.py	108	13	88%	11, 26, 41, 49, 54, 78, 128, 154, 163, 193, 203, 209, 218
expand_to_target.py	31	3	90%	6, 38, 43
biobalm/_sd_attractors
attractor_candidates.py	265	90	66%	13–15, 26–27, 93, 101, 107–108, 130, 152, 187, 193–204, 223, 239–320, 325, 329, 335, 341, 356, 383, 388, 392, 398, 400–438, 511, 582–583, 684
attractor_symbolic.py	114	16	86%	6–7, 75, 88–92, 103, 112, 144, 179, 191–193, 202, 230, 236
TOTAL	1974	347	82%

Tests	Skipped	Failures	Errors	Time
359	0 💤	0 ❌	0 🔥	1m 33s ⏱️

[jcrozum]

just a reminder that we should (eventually) either add proper docstrings to the expansion methods or else mark them as private

[jcrozum]

Could you add the parameters and returns to this docstring?

[jcrozum]

Or I suppose I could do it; I think it's straightforward here, and it looks like it's the only one that needs it.

[jcrozum]

(I just did this)

[jcrozum]

also need to update in requirements-dev.txt

[jcrozum]

(I just did this)

[jcrozum]

As discussed, I'll merge this and close #118.

[daemontus]

For completeness, here is also the performance comparison: The geomean on non-trivial benchmarks (>1s) is 26%, 20%, 5%, 17% better on BBM, NCF, NK2, NK3.

scc_vs_block.xlsx