bug(analyzer): taint is dropped on numeric values (int and boxed Integer), carried on reference types

[DavidBakerEffendi]

Hi — we maintain DataFlowBench, an open benchmark of dataflow/taint analyzers, and we recently added an OpenTaint adapter pinned to analyzer/2026.08.27.17eb0fe (adapter notes, PR). While adapting it we isolated one engine behavior that dominates its results, and it looked close enough to your recent "whole-object taint at a primitive position" work that it seemed worth reporting with a minimal repro.

Observed: for the same direct source→sink shape, varying only the value type, mode: taint rules report the String and Object variants and neither numeric one — int or boxed Integer:

static String string_src() { return "tainted"; }
static void string_sink(String v) { }
static void runString() { string_sink(string_src()); }   // reported

static int int_src() { return 1; }
static void int_sink(int v) { }
static void runInt() { int_sink(int_src()); }            // not reported

static Integer boxed_src() { return 1; }
static void boxed_sink(Integer v) { }
static void runBoxed() { boxed_sink(boxed_src()); }      // not reported

The rule-load trace shows all four rules registered, so it's not a rule-loading issue. Full retained evidence (fixture, rules, load trace, SARIF) is here, reproducible via scripts/probe-opentaint-value-kind.sh. Invocation: --project-kind=unknown, --debug-run-analysis-on-selected-entry-points=*, shipped models from opentaint-models.tar.gz.

If numeric taint is intentionally out of the threat model, a line of documentation would let us record it as a declared boundary rather than a measured miss.

Worth saying: on the reference-typed part of our corpus the engine did well — it correctly discriminates a depth-6 interprocedural relay pair that several engines miss at default call depths, plus recursion, closure capture, and two-level call context. Two smaller things we're happy to file separately if useful: a family of false positives where dynamically-selected heap locations (keyed map entries, computed members, function-valued fields) aren't kept separate, and two ergonomic asks (any version self-report in the analyzer jar, and a non-debug flag for all-methods entry points).

---

Note that I'm posting this via gh with Claude, who helped do the write-up.

[misonijnik]

@DavidBakerEffendi Hi! Thanks for the detailed feedback — this is exactly the kind of report that's useful to us.

we maintain DataFlowBench, an open benchmark of dataflow/taint analyzers

This is cool! There is a serious lack of public, independent benchmarks for taint analysis, so I'm glad to hear you're working on this.

We've also created a micro-benchmark to showcase OpenTaint's capabilities, so feel free to use it if you find the examples fair.

Observed: for the same direct source→sink shape, varying only the value type, mode: taint rules report the String and Object variants and neither numeric one — int or boxed Integer:

This looks like a rule definition issue. Primitive tracking is disabled by default; it can be enabled in the rule with:

    options:
      primitive-tracking: true

See an example of its usage here.

If numeric taint is intentionally out of the threat model, a line of documentation would let us record it as a declared boundary rather than a measured miss.

Sad but true — we lack documentation on rule writing. The good news is that it's on the way. Last weeks we worked on some nice rule extensions that will make rule pattern design more intuitive when it comes to field sensitivity, and we hope to release the docs and the extensions together soon.

Worth saying: on the reference-typed part of our corpus the engine did well — it correctly discriminates a depth-6 interprocedural relay pair that several engines miss at default call depths, plus recursion, closure capture, and two-level call context.

Great to hear!

Two smaller things we're happy to file separately if useful

That would be great, thanks!

dynamically-selected heap locations (keyed map entries, computed members, function-valued fields) aren't kept separate,

Am I right that I can find these examples in DataFlowBench?

and two ergonomic asks (any version self-report in the analyzer jar, and a non-debug flag for all-methods entry points).

Could you explain what you mean by "any version self-report in the analyzer jar"?

The --debug-run-analysis-on-selected-entry-points flag is marked as debug-only simply because we don't expect the analyzer to be used this way, given the potential for more false-positive noise and higher load. Otherwise, you're free to use it.

[DavidBakerEffendi]

We've also created a micro-benchmark to showcase OpenTaint's capabilities, so feel free to use it if you find the examples fair.

Thanks, this is useful for testing policies/rules on!

Primitive tracking is disabled by default;

That resolves the report. I re-ran the value-kind probe on the same pinned analyzer/2026.08.27.17eb0fe jar (digest-verified, invocation otherwise identical: --project-kind=unknown, --debug-run-analysis-on-selected-entry-points=*) with options: primitive-tracking: true on all four rules: the int and boxed Integer variants are now reported alongside String and Object. A Kotlin mirror of the probe (object members, Int) behaves identically. Looking good on that side now.

On your questions:

• Yes, the dynamically-selected-heap-location examples are in DataFlowBench: the four Kotlin challenge negatives (computed-property, dispatch-table, element-object, function-field), with retained SARIF under reports/raw/opentaint-kotlin-kernel/ whose codeFlows show the engine walking the wrong keyed entry (e.g. through leak after getValue("drop") in the dispatch-table case). I'll file that separately with a minimal repro.

  Whenever we do a versioned "freeze" of the latest results, it will come up on https://brokkai.github.io/dataflowbench/, OpenTaint will appear on v0.6.0 😉. Trying to find a balance of cases that show shortcomings in all candidates.

• Sorry, by "version self-report" we mean the analyzer artifact carries no build identity we could find, e.g., no --version flag, no manifest attribute, and the SARIF reports tool.driver.version: "latest". The project does publish versioned releases (v0.4.5), but the dated analyzer/* releases we consume have moved well past it (roughly ten since July 10), so a semver string alone wouldn't identify a build anyway. What would help downstream pinning is the jar reporting its own dated release tag or commit, e.g., 2026.08.27.17eb0fe, on any of those three surfaces; today we have to witness the release-asset SHA-256s instead and publish the tag only when they match.

• Understood on the entry-point flag. We keep it pinned as part of our documented invocation, and the debug labeling is now recorded as intentional rather than a stability caveat.

[DavidBakerEffendi]

(Closing this as the crux of the issue was a config fix)

[DavidBakerEffendi]

@misonijnik you guys are killing it :) https://brokkai.github.io/dataflowbench/

[misonijnik]

@misonijnik you guys are killing it :) https://brokkai.github.io/dataflowbench/

Thanks for the benchmark! We'll look into the FP/FN cases to figure out what we can do about them.