Audit memory usage with static analysis in sfm-propagate-changes.exe

[TeamDman]

With progress in sfm-propagate-changes.exe, static analysis auditing and reporting for memory usage timing with allocations and such would be useful.

This will automate the ability of identifying problems like #602 before they are noticed by humans.

Document formats:

• Flamegraph
  • Clients include IntelliJ, FireFox, Chrome and others that reuse the file format
• Tracy
  • Clients include tracy-profiler.exe and others that reuse the file format
• Tracing rust integration
  • Clients include our own code using tracing-subscriber

This is effectively auditing the consumption of memory. It is also important to be able to predict and audit the consumption of CPU instructions, which involves awareness of multithreading.

If we think about the ticket above, the problem could be described as

every tick do
  let context be new ProgramContext(...)
  for trigger in program.triggers do -- pronunciation of this punctuation is pretty good, sometimes syntax proposals use many symbols which is uglier to pronounce than if we favoured syntaxes which use more natural language than computer language.
    if trigger.shouldTick(context) then -- ugly to pronounce parentheses here, but what would be better?
      trigger.tick(context)
    end
  end
end

then the understanding of the problem is not that context is unused, because it is used in if trigger.shouldTick(context) then.
The problem is that trigger.shouldTick isn't using all the fields of context, so we should focus on static analysis that determines when an object is being accepted as a parameter where the body of the method does not have any coverage that uses some of the fields of that object.

If we have

let context = new Context(a, b, c);
void doWork(context) {
  context.a.work();
  context.b.work();
// c is never used
}
doWork(context);

then we should raise a flag.

That was a simple case, what if we introduce more nuance?

let context = new Context(a, b, c);
context.c.work();
void doWork(context) {
  context.a.work();
  context.b.work();
// c is never used
}
doWork(context);

then we have a code flow that determines that context.c is used and should not be removed.

However, this becomes a contradiction if we don't introduce a threshold or other inflection point where it goes from being appropriate to inappropriate.

The locality of the object creation to its usage.

How would we know to advocate for

c.work();
let context = new Context(a, b); // we're assuming this doesn't have stupid side-effects
  // we just call c.work() instead of context.c.work()
void doWork(context) {
  context.a.work();
  context.b.work();
// c is never used
}
doWork(context);

The most extreme version of this is actually running a JVM and adding hooks so we can react each time the method is called so we can use a counter or something to determine usage and such.

It may make sense to introduce a new object if the fields in the object are packed in an ugly way. If you have a big class with 999 fields, will that have worse performance than one with 2? How bad is the cache locality misses versus the cost to allocate a new object to hold the parameters?

The most broad implementation of this is a coverage harness where we ask "when was the last time we asked an AI to do a performance analysis for which the results of that analysis covered the presence of this method in its current form and how can perform such an analysis?" Just throw an AI at it in a way that gives confidence each method has been reviewed.

Bonus reading

https://spark.lucko.me/

#394

https://github.com/wolfpld/tracy

https://github.com/nagisa/rust_tracy_client

[TeamDman]

Separate the requirements from the previous implementation experiment

After looking at existing standards and SFM’s previous review-session work, I think this proposal should remain agnostic about its eventual storage and UI implementation.

SFM previously experimented with sfm.review-session/1. It represented pinned before/after source snapshots, comments attached to hashed UTF-8 ranges, producer provenance, derived hashtags, styles, and approval rules. Parts of that experiment were subsequently walked back or replaced with fixture-backed explorer work, so this discussion should not imply that extending sfm.review-session/1 is the current trajectory.

The experiment is still useful prior art. It demonstrated requirements that overlap with established standards:

• SARIF represents analysis tools, runs, rules, findings, physical and logical locations, code flows, graphs, and fingerprints for correlating findings between runs.
• SWHID provides intrinsic identifiers for software artifacts, with optional context such as origin, repository snapshot, revision anchor, path, and line or byte ranges.
• Flamegraphs, Tracy captures, and tracing output represent execution evidence and timelines rather than the higher-level identity of a finding.

These appear to be complementary layers:

1. Finding interchange: What problem was detected, by which rule and producer, at which logical and physical locations, with what severity and confidence. SARIF is an existing candidate.
2. Durable source identity: Which exact source artifact or fragment the finding concerns, including enough redundant context to survive moved files, changed URLs, missing repositories, and partial bit rot. SWHID or a compatible content-addressed reference is a candidate.
3. Performance evidence: Allocation counts, estimated instruction costs, call paths, flamegraphs, Tracy captures, or tracing spans supporting the finding.
4. Presentation and review: Whatever UI eventually displays, discusses, suppresses, approves, or migrates the finding. This could be an SFM explorer/comment system, an IDE, GitHub code scanning, or something else.

Consequently, I do not think sfm.review-session/1 should be prescribed as the output of sfm-propagate-changes.exe audit. Instead, the analyzer should produce an implementation-neutral finding model with enough information to support several adapters.

A finding should be able to include:

• A stable rule identifier.
• Producer name and version.
• Severity, confidence, and assumptions.
• The estimated resource cost and how it was calculated.
• Physical source ranges and logical symbols.
• Content hashes and repository/revision/path context.
• Related locations and call/data-flow relationships.
• References to supporting dynamic evidence.
• Versioned fingerprints for determining whether a later finding is probably the same finding.
• Explicit states for exact matches, conservative relocation, ambiguity, missing scope, and changed content.

For the motivating example, the finding might say that constructing ProgramContext places fields into a frequently allocated object even though a particular call path only observes a subset of those fields. It could identify the constructor, allocation site, consumer method, fields actually observed, estimated invocation frequency, and supporting trace. That information should remain meaningful regardless of whether it is rendered through SARIF, an SFM-native comment system, Tracy, or a future review UI.

The previous sfm.review-session/1 work may still supply useful algorithms, fixtures, conformance tests, or an adapter. This research has revealed that several of its concerns resemble facilities already standardized by SARIF and SWHID. That does not mean the old experiment must be revived or become the architecture from which this proposal evolves.

The concrete direction for this discussion is therefore:

Define the information that a trustworthy performance finding must preserve, then support standard and native representations as interchangeable projections of that information.