Semantic Uplift Engine & L3 Domain Ontology Mapping

[zaebee]

🎯 Feature Design: Semantic Uplift Engine & L3 Domain Ontology Mapping

1. Context & Problem Statement

Currently, our Code Graph is purely structural (L1/L2): it knows classes, methods, and calls, but it has no business context. To an AI agent, the function src.cgis.resolver.engine.ResolverEngine.resolve is just a series of syntax tokens. It does not know that this function belongs to the SemanticEngine domain or is critical for Linkage.

Following FC-4 (Semantic Uplift) in our FRD, we must build a SemanticUpliftEngine.
This engine will automatically:

1. Map node types to their formal OWL-Lite ontology_class.
2. Classify nodes into logical high-level business domains (e.g., Auth, Storage, Parser) using a declarative rules engine.
3. Propagate these domains down the structural hierarchy (Files $\to$ Classes $\to$ Methods).
4. Infer and generate high-level semantic dependencies (e.g., Domain(Parser) -[DEPENDS_ON]-> Domain(Storage)) based on the underlying call graph.

---

2. Data Contracts: Declarative Domain Specs

We will introduce a declarative configuration file, docs/ontology/domains.yaml, where the project architect defines the domains and their heuristic boundaries.

# docs/ontology/domains.yaml
version: "0.1.1"

domains:
  ParserLayer:
    description: "Source code parsing and AST extraction"
    heuristics:
      file_path_patterns: ["src/cgis/extractors/*", "src/cgis/base.py"]
      fqn_patterns: ["*.Extractor.*", "*.extractor.*"]
  
  StorageLayer:
    description: "Database storage, schemas, and persistence"
    heuristics:
      file_path_patterns: ["src/cgis/storage/*"]
      fqn_patterns: ["*.Store.*", "*.sqlite_store.*"]

  SemanticEngine:
    description: "Linkage, symbol resolution, and type propagation"
    heuristics:
      file_path_patterns: ["src/cgis/resolver/*"]
      fqn_patterns: ["*.Resolver.*", "*.engine.*"]

---

3. Algorithmic Blueprint (SemanticUpliftEngine)

We will run the SemanticUpliftEngine as a post-processing pass in our IngestionPipeline right after the ResolverEngine has finished resolving links.

Resolved SQLite Graph ──> [ 1. Ontology Class Mapping ]
                              ↓
                          [ 2. Heuristic Rule Tagging ] (1.0 Confidence)
                              ↓
                          [ 3. Structural Propagation ] (Hierarchical Downward)
                              ↓
                          [ 4. Semantic Dependency Inference ] (Domain A -> Domain B)

# Pseudo-code for src/cgis/resolver/uplift.py

class SemanticUpliftEngine:
    def __init__(self, store: SQLiteStore, domain_config_path: str = "docs/ontology/domains.yaml"):
        self.store = store
        self.domains_config = load_yaml(domain_config_path)

    def execute_uplift(self) -> None:
        """
        Runs the 4-phase Semantic Uplift pipeline over the database.
        """
        with self.store._conn:
            # Phase 1: Flat Ontology Class Mapping
            # (Easy SQL mappings: NodeType.CLASS -> 'Class', NodeType.FUNCTION -> 'Function')
            self._map_ontology_classes()

            # Phase 2: Rule-Based Tagging
            # Apply file_path_patterns and fqn_patterns from domains.yaml
            self._apply_heuristic_tagging()

            # Phase 3: Structural Propagation (Hierarchical Downward)
            # If a FILE node is in domain "ParserLayer", all CLASSes and FUNCTIONs it CONTAINS 
            # must inherit the "ParserLayer" domain with confidence = 1.0.
            # If a CLASS is in a domain, all of its DECLARED METHODs inherit it.
            self._propagate_structural_domains()

            # Phase 4: Semantic Dependency Inference (OWL Rule r2)
            # If Method A (Domain X) calls Method B (Domain Y), and X != Y, 
            # we infer that Domain X depends on Domain Y!
            # We generate and save a DOMAIN_DEPENDS_ON edge between the respective DomainConcept nodes.
            self._infer_domain_dependencies()

Phase 4 SQL Implementation (The OWL-Lite Inference Rule)

To generate high-level domain dependencies, we can run a single, highly efficient SQL join on SQLite:

-- Finds all inter-domain dependencies
SELECT DISTINCT 
    json_each.value AS source_domain, 
    json_each_target.value AS target_domain,
    e.file_path,
    e.line_number
FROM edges e
JOIN nodes n_src ON e.source = n_src.id
JOIN nodes n_tgt ON e.target = n_tgt.id,
json_each(n_src.domains) AS src_dom,
json_each(n_tgt.domains) AS tgt_dom
WHERE e.type = 'CALLS' 
  AND src_dom.value != tgt_dom.value;
-- Based on this query, we insert edges of type EdgeType.DOMAIN_DEPENDS_ON

---

4. TDD Acceptance Criteria

• Test 1 (Ontology Class Invariant): Verify that all nodes have their ontology_class correctly populated (e.g., Node(type=NodeType.CLASS) gets ontology_class="Class").
• Test 2 (Heuristic Tagging): Given the domains.yaml config, a file matching src/cgis/storage/sqlite_store.py must be tagged with the StorageLayer domain with confidence_score=1.0.
• Test 3 (Hierarchical Propagation): If sqlite_store.py is tagged as StorageLayer, verify that its declared methods (like save_graph) automatically inherit the StorageLayer domain.
• Test 4 (Domain Dependency Inference): Given IngestionPipeline.run (Domain: ParserLayer) calling ResolverEngine.resolve (Domain: SemanticEngine). Verify that an Edge of type EdgeType.DOMAIN_DEPENDS_ON is generated from ParserLayer to SemanticEngine.

---

5. Architectural High-Thinking (Watch-outs)

1. Multi-Domain Pollution: A node can theoretically belong to multiple domains (e.g., a file containing helper functions used by both Auth and Storage). While our domains column is a JSON list (which supports this), the propagation algorithm should limit transitive depth or apply a decay factor (e.g., confidence drops by 20% on every transitive step) to prevent the entire graph from dissolving into a single "universal" domain.
2. First-Class Domain Nodes: In v0.1.1.md, DomainConcept is a Class. During Phase 4, when we find domains like ParserLayer or StorageLayer, we must insert them as first-class Node objects (type=NodeType.DOMAIN_CONCEPT) into the database. This allows the React UI and Mermaid compiler to render the high-level Domain Relationship Map alongside the raw call graph!