Skip to content

feat(io): handle-aware I/O for the lean non-Python walk plus Python cursor derive - #770

Merged
vitali87 merged 8 commits into
mainfrom
feat/714-handle-aware-io
Jul 16, 2026
Merged

feat(io): handle-aware I/O for the lean non-Python walk plus Python cursor derive#770
vitali87 merged 8 commits into
mainfrom
feat/714-handle-aware-io

Conversation

@vitali87

Copy link
Copy Markdown
Owner

Problem

The lean non-Python I/O walk (issue #714) only matched direct sink calls. Handle-based I/O, which is the dominant idiom in most of these languages, produced nothing:

  • Go: f, _ := os.OpenFile("data.txt", os.O_WRONLY, 0644); f.WriteString(s) — no edge (OpenFile's direction depends on flags, so it cannot be a direct sink at all).
  • JS/TS: const ws = fs.createWriteStream('out.txt'); ws.write(data) — no edge.
  • Java: new FileWriter("out.txt"), new BufferedReader(new FileReader("in.txt")), Files.newBufferedReader(Path.of("cfg.txt")) — Java had zero handle coverage since every file/DB touch goes through a handle.
  • Rust: let mut f = File::open("in.txt")?; f.read_to_string(&mut s)? — no edge.
  • C++: std::ifstream in("in.txt"); in >> word; — no edge.
  • Python: cur = conn.cursor(); cur.execute("SELECT ...") — the cursor was invisible even in the full Python handle walk.

IO_HANDLE_CONSTRUCTORS had only Python entries; the registry comment marked handles as a follow-up.

Fix

The lean walk now tracks handle bindings in source order (flat, mirroring Python's handle walk) and attributes handle-method calls to the constructor's resource:

  • Call-shaped constructors per language (os.Open/Create/OpenFile, database/sql.Open, net.Dial, fs.createReadStream/createWriteStream, Files.newBufferedReader/..., DriverManager.getConnection, std::fs::File::open/create, std::net::TcpStream::connect), resolved with the same shadow-aware, import-expanding matching as sinks.
  • new-shaped constructors (Java new FileWriter etc.) plus wrapper types (new BufferedReader(new FileReader(p)), new Scanner(new File(p))) that delegate identity to arg0; PrintWriter keeps both its writer-wrapping and filename overloads.
  • Call-shaped wrappers: Rust BufReader::new(f) / BufWriter::new(f), Go bufio.NewReader/NewWriter/NewScanner.
  • Type-declaration constructors: C++ std::ifstream in("x.txt") (init_declarator) and the most-vexing-parse form std::ifstream dyn(path) (function_declarator, <dynamic> identity). << on a bound handle writes and >> reads its resource; the existing cout/cerr stream sink behaviour is unchanged.
  • Rust Result unwrapping: bindings resolve through ? (try_expression) and .unwrap() / .expect(..).
  • Identity unwrapping: Files.newBufferedReader(Path.of("cfg.txt")) and new Scanner(new File("data.csv")) carry the literal.
  • Derive methods (IO_HANDLE_DERIVES): conn.createStatement() / conn.prepareStatement() (java.sql) and conn.cursor() (Python DB-API) yield same-resource sub-handles — this also closes the long-standing Python cursor gap.
  • Aliases track (g := f), rebinds kill, unbound receivers and arithmetic x << 2 stay silent.
  • Per-language handle-method tables (IO_LEAN_HANDLE_METHODS) map methods to direction; java.sql execute(sql) refines READ_WRITE by the SQL first keyword, like Python.

The (targets, values) extraction the flow walk already used for taint binding is factored into a shared binding_targets_values helper in io_access/extract.py, used by both walks.

Validation

RED → GREEN in commit history: the test(io) commit adds 26 failing specs first (30 after review-driven additions), covering every language plus negatives; the feature commit turns them green. Five e2e integration tests added to the per-language test_*_io_e2e.py files.

Full suite: 4986 passed, 5 skipped (pytest -n auto). No default-capture behaviour changes: everything stays behind the opt-in io capture group, and all existing direct-sink semantics (including Go os.Open/os.Create as direct sinks) are preserved.

Part of #714; the remaining lean-flow path-sensitivity depth (loops/try for non-hoisted languages, Rust if/match) follows in the next PR.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements handle-aware I/O tracking for non-Python languages (the "lean" walk, addressing issue #714). It introduces mechanisms to track resource handles (such as files, databases, and sockets) across JavaScript/TypeScript, Go, Java, Rust, and C++. This includes resolving constructors, unwrapping result/wrapper types, deriving sub-handles (e.g., database cursors), and attributing subsequent method calls or stream operations (like << and >>) to the correct underlying resource. The changes also include refactoring shared binding extraction logic and adding comprehensive integration and unit tests. There are no review comments, so I have no feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@greptile-apps

greptile-apps Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds handle-aware I/O detection across the lean non-Python walk and extends Python DB cursor tracking. The main changes are:

  • New handle constructor, wrapper, derive, and method tables for Go, JS/TS, Java, Rust, and C++.
  • Source-order handle binding and rebinding support for the lean I/O walk.
  • C++ stream extraction and insertion handling for bound file stream handles.
  • Shared binding target/value extraction used by both flow and I/O processing.
  • Unit and integration tests covering handle constructors, aliases, wrappers, rebinding, SQL direction refinement, and Python cursor derives.

Confidence Score: 5/5

Safe to merge with low risk.

No confirmed runtime, logic, or security issues were found in the changed files. The updated I/O handle paths are covered by focused unit and integration tests.

No files require special attention.

T-Rex T-Rex Logs

What T-Rex did

  • T-Rex ran the test workflow and generated a proof log of the pytest attempt.
  • The log shows a uv warning about an existing virtual environment linked to a non-existent Python interpreter, signaling a mismatch in the venv setup.
  • The virtual environment was removed and recreated, but the log then shows No module named pytest, so the focused test file did not run.

View all artifacts

T-Rex Ran code and verified through T-Rex

Important Files Changed

Filename Overview
codebase_rag/parsers/io_access/processor.py Adds lean non-Python handle binding, wrapper, derive, stream-operator, and Python cursor-derive handling.
codebase_rag/parsers/io_access/registry.py Registers per-language handle constructors, wrappers, derive methods, identity unwrappers, and handle method direction tables.
codebase_rag/parsers/io_access/extract.py Factors lean binding target/value extraction into shared helpers used by flow tainting and handle binding.
codebase_rag/parsers/flow_access/processor.py Reuses shared binding target/value extraction for lean flow taint bindings.
codebase_rag/tests/test_io_handle_edges.py Adds broad tests for handle constructors, aliases, rebinding, wrappers, SQL direction refinement, C++ streams, and Python cursor derives.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant Caller as Caller AST scope
participant Walk as IOAccessProcessor lean walk
participant Bind as Handle binding map
participant Reg as I/O registry tables
participant Graph as Graph ingestor

Caller->>Walk: Visit statements in source order
Walk->>Reg: Resolve constructor, wrapper, and derive calls
Reg-->>Walk: Handle kind, method direction, and identity rules
Walk->>Bind: Bind or rebind handle variables
Caller->>Walk: Later handle method or stream operator call
Walk->>Bind: Lookup receiver binding
Bind-->>Walk: Resource kind and identity
Walk->>Graph: Emit READS_FROM or WRITES_TO edge
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant Caller as Caller AST scope
participant Walk as IOAccessProcessor lean walk
participant Bind as Handle binding map
participant Reg as I/O registry tables
participant Graph as Graph ingestor

Caller->>Walk: Visit statements in source order
Walk->>Reg: Resolve constructor, wrapper, and derive calls
Reg-->>Walk: Handle kind, method direction, and identity rules
Walk->>Bind: Bind or rebind handle variables
Caller->>Walk: Later handle method or stream operator call
Walk->>Bind: Lookup receiver binding
Bind-->>Walk: Resource kind and identity
Walk->>Graph: Emit READS_FROM or WRITES_TO edge
Loading

Reviews (4): Last reviewed commit: "refactor(io): extract sonar-flagged dupl..." | Re-trigger Greptile

@vitali87

Copy link
Copy Markdown
Owner Author

@greptile review

Comment thread codebase_rag/parsers/io_access/processor.py
@vitali87

Copy link
Copy Markdown
Owner Author

@greptile review

@vitali87

Copy link
Copy Markdown
Owner Author

@greptile review

@sonarqubecloud

Copy link
Copy Markdown

@vitali87
vitali87 merged commit 82465b4 into main Jul 16, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant