Skip to content

feat(flow): add opt-in FLOWS_TO data-flow edges for intra-procedural taint#707

Merged
vitali87 merged 4 commits into
mainfrom
feat/flows-to
Jul 12, 2026
Merged

feat(flow): add opt-in FLOWS_TO data-flow edges for intra-procedural taint#707
vitali87 merged 4 commits into
mainfrom
feat/flows-to

Conversation

@vitali87

Copy link
Copy Markdown
Owner

What

Adds FLOWS_TO, an opt-in relationship type that records intra-procedural value flow (taint/provenance) between resources and callables. Turns questions like "does a value read here reach a write over there?" into graph reachability, which coarse CALLS co-occurrence cannot answer.

Stacked on #638 (base branch feat/relationship-capture-io): FLOWS_TO builds on that PR's Resource nodes and I/O source/sink registry, so it must land after #638. When #638 merges this PR auto-retargets to main.

Behavior

FLOWS_TO is emitted in three shapes, distinguished by a kind edge property (plus a via conduit label):

  • resource → resource (kind=resource): a value read from one resource reaches a write to another within a body, e.g. x = os.getenv("K"); print(x) yields Resource(ENV::K) -FLOWS_TO-> Resource(STDOUT).
  • caller → callee (kind=arg, via=arg:<i> / kw:<name>): a tainted local is passed as an argument to a first-party callee.
  • callee → caller (kind=return, via=return): a callee whose return value is tainted flows it back to the assignment in its caller.

Taint propagates through plain x = y assignments.

Opt-in / zero default cost

FLOWS_TO joins the existing io capture group, which is excluded from the default capture set. A default build emits zero FLOWS_TO edges and skips the body pass entirely. Enable with --capture io (or CGR_CAPTURE=io).

Design

Self-contained codebase_rag/parsers/flow_access/ submodule mirroring io_access/: one scope-pruned body DFS per caller, invoked once beside process_io_for_caller, reusing the resolver for callee resolution. Pure extractors shared with io_access were promoted to io_access/extract.py (no behavior change; the io suite is unchanged). No new node types: FLOWS_TO connects existing Function/Method/Module/Resource nodes.

Phase-1 scope is deliberately conservative and documented inline: Python-only, single function body plus one level of argument/return hand-off, direct source/sink calls, no SSA/parameter nodes. Upgrade paths are noted at each ceiling.

Tests

RED → GREEN: failing tests committed first (2c3f1fe), then the implementation (2cc78d2). Eight test_flow_edges.py cases cover each edge shape, taint propagation, the anti-false-positive negative case (co-occurrence is not flow), default-off gating, and the dangling-endpoint guard, plus capture-selection membership. Full suite green (4853 passed); ty and ruff clean; relationship properties confirmed to round-trip the protobuf path.

Docs

docs/architecture/graph-schema.md gains the Resource node row, the READS_FROM/WRITES_TO/FLOWS_TO relationship rows, and an I/O and data-flow section.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces significant improvements to the codebase, including a new dead code detection feature, enhanced C++ frontend integration with a hybrid mode, and robust handling of import edge verification and class inheritance. I have identified a critical bug in the embedding logic where global is incorrectly used instead of nonlocal for a variable defined in an enclosing scope, and an improvement opportunity for the C++ qualified name resolution to ensure consistent separator usage.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread codebase_rag/embedder.py
Comment thread codebase_rag/parsers/cpp/utils.py
@vitali87
vitali87 changed the base branch from feat/relationship-capture-io to main July 11, 2026 23:16
@vitali87

Copy link
Copy Markdown
Owner Author

@greptile review

@greptile-apps

greptile-apps Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds opt-in FLOWS_TO data-flow edges for Python I/O provenance. The main changes are:

  • New FLOWS_TO relationship type under the existing io capture group.
  • A flow_access processor that emits resource, argument, and return flow edges.
  • Shared I/O extraction helpers reused by both I/O and flow processing.
  • Schema, README, and architecture docs for Resource, READS_FROM, WRITES_TO, and FLOWS_TO.
  • Tests for capture gating, taint propagation, overwrite handling, return flow, and resource endpoint creation.

Confidence Score: 5/5

Safe to merge with minimal risk.

The change is opt-in, scoped to Python flow extraction, reuses existing I/O registries, and includes tests for the previously reported overwrite and return-origin cases.

No files require special attention.

T-Rex T-Rex Logs

What T-Rex did

  • The team reviewed the pre-run environment for the targeted pytest attempt, as shown in the python-flow-capture-01-before.log, confirming uv availability and Python 3.12 environment creation with EXIT_CODE: 1.
  • The team reviewed the post-run state after the allowed setup attempt, as shown in the python-flow-capture-02-after.log, noting missing cmake/cmake3 checks, pymgclient build failure, and EXIT_CODE: 1.

View all artifacts

T-Rex Ran code and verified through T-Rex

Important Files Changed

Filename Overview
codebase_rag/constants/graph.py Adds FLOWS_TO to RelationshipType and the opt-in io capture group.
codebase_rag/parsers/call_processor.py Initializes FlowProcessor with the shared capture selection and invokes it during callable processing.
codebase_rag/parsers/flow_access/processor.py Implements opt-in Python intra-procedural taint flow emission for resource, argument, and return edges.
codebase_rag/parsers/io_access/extract.py Extracts reusable call-name normalization and literal target helpers from the I/O processor.
codebase_rag/parsers/io_access/processor.py Refactors I/O processing to use shared extraction helpers without changing capture behavior.
codebase_rag/tests/test_flow_edges.py Adds tests for flow edge shapes, default-off behavior, stale-taint handling, and resource endpoint creation.
codebase_rag/tests/test_capture_resolver.py Adds coverage confirming FLOWS_TO is opt-in through the io capture group.
codebase_rag/types_defs.py Extends relationship schema metadata to allow FLOWS_TO across callable and resource nodes.
docs/architecture/graph-schema.md Documents Resource nodes and the I/O/data-flow edge semantics.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant CP as CallProcessor
participant FP as FlowProcessor
participant IO as io_access extract/registry
participant CR as CallResolver
participant IG as Ingestor

CP->>FP: process_flow_for_caller(caller_node, caller_spec, qn, module, language, class_context)
FP->>FP: Skip unless FLOWS_TO is enabled and language is Python
FP->>IO: Load read/write sinks and normalize call names
FP->>FP: Walk caller body in source order
alt assignment from read source
    FP->>FP: Mark local tainted with Resource binding
else tainted argument to first-party call
    FP->>CR: resolve_function_call(...)
    CR-->>FP: callee type and qualified name
    FP->>IG: "ensure FLOWS_TO caller -> callee (kind=arg, via=arg/kw)"
else tainted value reaches write sink
    FP->>IG: ensure Resource endpoints
    FP->>IG: "ensure FLOWS_TO resource -> resource (kind=resource)"
else assignment from tainted-returning callee
    FP->>CR: resolve_function_call(...)
    FP->>IG: "ensure FLOWS_TO callee -> caller (kind=return)"
    FP->>FP: Carry returned source binding into local taint
end
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant CP as CallProcessor
participant FP as FlowProcessor
participant IO as io_access extract/registry
participant CR as CallResolver
participant IG as Ingestor

CP->>FP: process_flow_for_caller(caller_node, caller_spec, qn, module, language, class_context)
FP->>FP: Skip unless FLOWS_TO is enabled and language is Python
FP->>IO: Load read/write sinks and normalize call names
FP->>FP: Walk caller body in source order
alt assignment from read source
    FP->>FP: Mark local tainted with Resource binding
else tainted argument to first-party call
    FP->>CR: resolve_function_call(...)
    CR-->>FP: callee type and qualified name
    FP->>IG: "ensure FLOWS_TO caller -> callee (kind=arg, via=arg/kw)"
else tainted value reaches write sink
    FP->>IG: ensure Resource endpoints
    FP->>IG: "ensure FLOWS_TO resource -> resource (kind=resource)"
else assignment from tainted-returning callee
    FP->>CR: resolve_function_call(...)
    FP->>IG: "ensure FLOWS_TO callee -> caller (kind=return)"
    FP->>FP: Carry returned source binding into local taint
end
Loading

Reviews (3): Last reviewed commit: "fix(flow): carry return-value taint sour..." | Re-trigger Greptile

Comment thread codebase_rag/parsers/flow_access/processor.py Outdated
@vitali87

Copy link
Copy Markdown
Owner Author

@greptile review

Comment thread codebase_rag/parsers/flow_access/processor.py
@vitali87

Copy link
Copy Markdown
Owner Author

@greptile review

@sonarqubecloud

Copy link
Copy Markdown

@vitali87
vitali87 merged commit 6dfb888 into main Jul 12, 2026
22 checks passed
@vitali87
vitali87 deleted the feat/flows-to branch July 12, 2026 00:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant