Module graph entity–relationship modeling

[DerekNonGeneric]

For one to be able to visualize a module graph, one would need to know of parent-child relationships.

Details

This issue intends to explore how to best achieve this. More coming soon.

[ljharb]

This would also probably need to account for module graphs not being static, since dynamic import could pull in a module at any time.

[jkrems]

This feels like something that shouldn't be enabled by default. I assume this is for debugging purposes and could be an opt-in flag? That would allow us to use trace events (or something similar to it) instead of having to keep a big data structure around.

[DerekNonGeneric]

Please don't feel obliged to respond, just some notes before I forget… :)

/to @ljharb

This would also probably need to account for module graphs not being static, since dynamic import could pull in a module at any time.

Expressing dynamic imports will add complexity, though it should still be entirely possible.

"The dynamically imported module starts a new graph, which is processed separately." ^{1 2}

This seems to be akin to the top-level await subgraphs (?) mentioned in the following excerpt of the proposal repo's README (emphasis mine).

When a module includes a top-level `await` (even if that `await` is not dynamically
reached), this is not synchronous, and at the very least takes one trip through
the Promise job queue. However, module **subgraphs** which do not use top-level
await continue to run synchronously in exactly the same way as without this
proposal. And if several modules which do not use top-level `await` depend on a
module which does use it, then those modules will all run when the async module
is ready, without yielding to any other work (neither the Promise job
queue/microtask queue, nor the host's event loop, etc.).

I've already begun work on serializing relationships and rendering visual representations of the module graphs. [Hope to share my results soon.] I noticed that the specification is using this very simple directed graph notation.

Madge, which is an npm package that generates visual graphs of module dependencies, is also using a very similar notation as can be seen in this test I ran. Disappointingly, it only spits out a static image of the graph (and not during execution).

These notations are appealing due to their simplicity, though not really what I had in mind, which was something akin to Crow's Foot Notation, as seen below.

This example above is also an oversimplification, but at least more relationship information can be expressed visually.

---

/to @jkrems

[...] That would allow us to use trace events (or something similar to it) instead of having to keep a big data structure around.

There is an open proposal to implement Perfetto in Node.js @ nodejs/diagnostics#277.

# Perfetto - Performance instrumentation and tracing

Perfetto is an open-source project for performance instrumentation and tracing
of Linux/Android/Chrome platforms and user-space apps. It consists of:

A portable, high efficiency, user-space tracing library designed for tracing of
multi-process systems, based on zero-alloc zero-copy zero-syscall (on
fast-paths) writing of protobufs over shared memory.

I have yet to look too deeply into this as progress seems to have stalled (it's entirely probable that I don't fully understand how it works). Still, at first glance, it looks like this would be replacing Node's current stack tracing and diagnostics implementation, which would fuel the stack tracing side of things once ready. Anyone with further insight, feel free to chime in. :)

This feels like something that shouldn't be enabled by default.

Maybe so.

I assume this is for debugging purposes and could be an opt-in flag?

That would be cool!

---

1 https://hacks.mozilla.org/2018/03/es-modules-a-cartoon-deep-dive
2 https://tc39.es/ecma262/#sec-hostimportmoduledynamically