Data flow: Track precise types during field flow

[hvitved]

Overview

This PR factors type information out of the language-specific Content entities, and instead uses the actual stores to record type information. That is, when tracking flow through a field-write, x.f = y, we use the type of x during the subsequent flow calculations, rather than the result of getContainerType() on the content entity corresponding to f.

This means that we are now able to eliminate flow like in the example below:

class FieldA
{
    public object Field;

    public virtual void M() { }

    public void CallM() => this.M();

    static void M1(FieldB b, FieldC c)
    {
        b.Field = new object();
        b.CallM(); // no flow            <- NEW
        c.Field = new object();
        c.CallM(); // flow
    }
}

class FieldB : FieldA { }

class FieldC : FieldA
{
    public override void M() => Sink(this.Field);
}

While this change may not have a big impact on field flow, it is crucial in the implementation of (typed) collection flow: If we kept type information in the Content class:

newtype TContent = ... or TCollectionContent(DataFlowType elementType)

then a generic element read such as

T GetFirstElement<T>(IList<T> l) => l[0];

would have to match all TCollectionContent(t) entities, which does not scale.

Performance

I have started Difference jobs for

• C#: https://jenkins.internal.semmle.com/job/Changes/job/CSharp-Differences/165/.
• C++: https://jenkins.internal.semmle.com/job/Changes/job/CPP-Differences/1133/.
• Java: https://jenkins.internal.semmle.com/job/Changes/job/Java-Differences/746/.

I also ran @aschackmull 's tuple counting benchmark on JDK, which revealed that most relations had similar sizes; most notably, flowCandFwdConsCand increased by 56%, but from a mere 10711 tuples to 16710 tuples.

Notes for review

Commit-by-commit review is encouraged. The actual work happens in f1cd535; the other PRs deal with syncing, renaming, and follow-up changes.

[jbj]

I haven't looked at the code, but I note that the CPP-Differences job shows only one result change, which is caused by build-system wobble, and mostly unchanged performance. The slowdown on Wireshark may be significant, but it's not large enough to block this PR.

[hvitved]

I haven't looked at the code, but I note that the CPP-Differences job shows only one result change, which is caused by build-system wobble, and mostly unchanged performance. The slowdown on Wireshark may be significant, but it's not large enough to block this PR.

👍 FYI, the C++ specific parts are in 2d7470f.

[hvitved]

Ping @aschackmull.

[hvitved]

Ping.

[aschackmull]

I'm looking at this, but it's slow going, sorry. Found a bug in flowCandFwdConsCand to possibly explain the tuple explosion though, but I'll investigate further before reviewing in full.

[aschackmull]

PR against this PR: hvitved#2

[hvitved]

Updated Differences jobs:
C#: https://jenkins.internal.semmle.com/job/Changes/job/CSharp-Differences/222/
C++: https://jenkins.internal.semmle.com/job/Changes/job/CPP-Differences/1200/
Java: https://jenkins.internal.semmle.com/job/Changes/job/Java-Differences/785/

C++ and Java have completed, and they still show now change in neither performance nor results, as expected.

[hvitved]

I had to restart the C# job (https://jenkins.internal.semmle.com/job/Changes/job/CSharp-Differences/224/), which also shows no change in performance nor results.