DSL attributes (#19)

* Update changelog and version * Update DSL EBNF grammar to support edge attributes * Support constraints in parser and motif * Update executors and reorganize * Update test suite to start working with constraints * 🛠 Reformat repo [black] * Fix disagreeing-edge test * Working networkx executor with constraints * 🛠 Cypher generator for edge attributes * Working neo4j edge constraints * Add Neo4jExecutor edge constraints * Add test cases for DSL edge constraints * Add a bunch of operator tests * Add test cases for simple node constraint * Support node attributes in DSL * List node constraints in dotmotif base class * Untype strings for parsing from the grammar * Support node attributes in grammar * Add tests for utils * Add test cases for mult. edge operator constraints * Add support for multi-op edge constraint * Allow n constraints per operator in edges * Begin node constraint satisfier * Add string constraint support to edges and nodes * Add support for node attributes in networkx importer * Fix syntax error * Replace equality operator in nx * Fix one bug but find another * Complete networkx executor test suite * Add test case for node+edge attrs * Support node and edge attributes simultaneously * Add contains/in on networkx exec * Add attributes documentation * Add CircleCI * Add CircleCI nose2 test * Update documentation * Troubleshooting circleci * Make things worse for CircleCI to not complain * Undo last commit, which made things better
aplbrain · Feb 6, 2019 · 9b998be · 9b998be
1 parent edb9e61
commit 9b998be
Show file tree

Hide file tree

Showing 23 changed files with 1,545 additions and 491 deletions.
diff --git a/.circleci/config.yml b/.circleci/config.yml
@@ -0,0 +1,51 @@
+# Python CircleCI 2.0 configuration file
+#
+# Check https://circleci.com/docs/2.0/language-python/ for more details
+#
+version: 2
+jobs:
+  build:
+    docker:
+      # specify the version you desire here
+      # use `-browsers` prefix for selenium tests, e.g. `3.6.1-browsers`
+      - image: circleci/python:3.6.1
+
+      # Specify service dependencies here if necessary
+      # CircleCI maintains a library of pre-built images
+      # documented at https://circleci.com/docs/2.0/circleci-images/
+      # - image: circleci/postgres:9.4
+
+    working_directory: ~/repo
+
+    steps:
+      - checkout
+
+      # Download and cache dependencies
+      - restore_cache:
+          keys:
+          - v1-dependencies-{{ checksum "setup.py" }}
+          # fallback to using the latest cache if no exact match is found
+          - v1-dependencies-
+
+      - run:
+          name: install dependencies
+          command: |
+            python3 -m venv venv
+            . venv/bin/activate
+            pip install nose2
+            pip install -U .
+
+      - save_cache:
+          paths:
+            - ./venv
+          key: v1-dependencies-{{ checksum "setup.py" }}
+
+      - run:
+          name: run tests
+          command: |
+            . venv/bin/activate
+            nose2 -t .
+
+      - store_artifacts:
+          path: test-reports
+          destination: test-reports
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,3 +1,10 @@
+
+- **0.4.0**
+    - DSL:
+        - Support for node and edge attributes in non-template contexts
+    - Executors:
+        - `NetworkxExecutor`
+            - Support for attribute filtering
 - **0.3.0**
     - Overhaul of DotMotif Parsers
         - Implement EBNF spec of language, and complete parser (`lark`)

diff --git a/docs/Language Spec.md → docs/Language-Spec.md b/docs/Language Spec.md → docs/Language-Spec.md
diff --git a/docs/attributes.md b/docs/attributes.md
@@ -0,0 +1,98 @@
+# Node and Edge Constraints
+
+In addition to structural constraints, dotmotif supports attribute constraints. Let's look at a simple example:
+
+
+## A simple example
+
+```
+A -> B
+```
+
+This query returns _all_ edges in a graph. Not particularly useful! But look what happens when we add some qualifications:
+
+```
+A -> B [weight >= 10]
+```
+
+This motif now becomes much more powerful: Only edges with a `weight` attribute value greater than 10 are returned. What if we want a weight within a certain range?
+
+```
+A -> B [weight <= 20, weight >= 10]
+```
+
+This motif returns all edges with a weight between 10 and 20. What do you think this constraint does?
+
+```
+A -> B [weight <= 20, weight >= 10, weight != 12]
+```
+
+These examples have looked at edge constraints so far. But motif nodes can have constraints as well!
+
+## Nodes can have constraints as well
+
+Unlike edge operators, which live inside square brackets on the same line as the edge they're describing, node constraints can live anywhere in your motif:
+
+```
+A -> B
+
+A.name = "Wilbur"
+```
+
+## Some notes on constraint combinations
+
+You can reuse the same constraint operator more than once, like this:
+
+```
+A.area < 10
+A.area < 20
+```
+
+This combination is redundant, and more importantly, it can increase the runtime of your query! When you're dealing with sufficiently large graphs, be sure to design your motifs with runtime in mind.
+
+It is likewise possible (lo! even easy!) to build contradicting constraints:
+
+```
+A.name == "Fred"
+A.name != "Fred"
+```
+
+Even though this seems like a contrived example, it becomes increasingly simple to make this sort of mistake in larger motifs. Though constraint validators will often catch these sorts of mistakes, it's smart to give your motif a once-over before submitting it to run unsuccessfully.
+
+## Everybody now
+
+You can of course run node and edge attributes through the same motif:
+
+```
+A -> B [weight >= 0.6]
+A.type = "Glu"
+B.type = "ACh"
+```
+
+# Available Operators
+
+## Edge Operators
+
+| Operator | Notes |
+|----------|-------|
+| < |
+| > |
+| <= |
+| >= |
+| != | OR <> |
+| = | OR == |
+| in | Edge value is contained within the value specified. For example, `[name in "1234567890"]` will return edges with `name: 1`, `name: 23`, or `name: 6789`. |
+| contains | Edge value contains the value specified. For example, `[name contains "GABA"]` will return edges with `name: GABA1`, `name: GABAergic`, or `name: GABALLAMA`. |
+
+## Node Operators
+
+| Operator | Notes |
+|----------|-------|
+| < |
+| > |
+| <= |
+| >= |
+| != | OR <> |
+| = | OR == |
+| in | Node value is contained within the value specified. For example, `A.name in "1234567890"` will return nodes with `name: 1`, `name: 23`, or `name: 6789`. |
+| contains | Node value contains the value specified. For example, `A.name contains "GABA"` will return nodes with `name: GABA1`, `name: GABAergic`, or `name: GABALLAMA`. |
diff --git a/dotmotif/__init__.py b/dotmotif/__init__.py
@@ -20,10 +20,11 @@
 from .parsers.v2 import ParserV2
 from .validators import DisagreeingEdgesValidator
 
-__version__ = "0.3.0"
+__version__ = "0.4.0"
 
 DEFAULT_MOTIF_PARSER = ParserV2
 
+
 class MotifError(ValueError):
     pass
 
@@ -53,19 +54,20 @@ def __init__(self, **kwargs):
         self.enforce_inequality = kwargs.get("enforce_inequality", False)
         self.pretty_print = kwargs.get("pretty_print", True)
         self.parser = kwargs.get("parser", DEFAULT_MOTIF_PARSER)
-        self.validators = kwargs.get("validators", [
-            DisagreeingEdgesValidator()
-        ])
+        self.validators = kwargs.get("validators", [DisagreeingEdgesValidator()])
         self._LOOKUP = {
             "INHIBITS": "INH",
-            "EXCITES":  "EXC",
+            "EXCITES": "EXC",
             "SYNAPSES": "SYN",
-            "INH":      "INH",
-            "EXC":      "EXC",
-            "SYN":      "SYN",
+            "INH": "INH",
+            "EXC": "EXC",
+            "SYN": "SYN",
         }
         self._g = nx.MultiDiGraph()
 
+        self._edge_constraints = {}
+        self._node_constraints = {}
+
     def from_motif(self, cmd: str):
         """
         Ingest a dotmotif-format string.
@@ -78,10 +80,17 @@ def from_motif(self, cmd: str):
 
         """
         if len(cmd.split("\n")) is 1:
-            cmd = open(cmd, 'r').read()
-
-        self.cmd = cmd
-        self._g = self.parser(validators=self.validators).parse(self.cmd)
+            try:
+                cmd = open(cmd, "r").read()
+            except FileNotFoundError:
+                pass
+
+        result = self.parser(validators=self.validators).parse(cmd)
+        if isinstance(result, tuple):
+            self._g, self._edge_constraints, self._node_constraints = result
+        else:
+            # For backwards compatibility with parser v1
+            self._g = result
 
         return self
 
@@ -108,3 +117,9 @@ def to_nx(self) -> nx.DiGraph:
 
         """
         return self._g
+
+    def list_edge_constraints(self):
+        return self._edge_constraints
+
+    def list_node_constraints(self):
+        return self._node_constraints
diff --git a/dotmotif/executors/Executor.py b/dotmotif/executors/Executor.py
@@ -0,0 +1,8 @@
+from .. import dotmotif
+
+
+class Executor:
+    ...
+
+    def find(self, motif: dotmotif, limit: int = None):
+        ...