nim c --dump:out.json main: dump compiler output metadata (dependencies etc) to json

[timotheecour]

proposal

add a flag --dump:filename to nim cmdline which allows generating compiler output metadata to a json file, which can then be used for tooling; eg:

• set of files processed, along with their canonical module name, and whether they're import's or include's
• import graph (a imports b; a includes b)
• stats (such as the ones generated by hintSucessX msg, but can include more stats)
• inferred configuration variables such as the ones accessible via compilesettings (eg resolved lib dir, nimcache, project file etc)

design goals

• extensible: you can add fields without breaking tooling (tooling should obviously keep in mind that fields may be added)
• composable: it works with any nim command, eg: doc, c, js, r, check etc
• machine and human readable (since it uses json)
• optionally, user can select which fields to populate if some of those are expensive/large; but this isn't needed in a first version

why json?

because it's simple and universal and will not cause a performance bottleneck for the data being generated. In particular, nim genDepend main generates a custom text format deps file which can't be extended without introducing a breaking change, whereas json output can be extended by adding additional fields without a breaking change

why not nim genDepend main ?

• as mentioned above, genDepend generates a custom format text file instead of json (not extensible without breaking change)
• genDepend can't be composed with other commands (eg nim js main)
• genDepend generates potentially unwanted other files and hardcodes their location (png, dot, deps)

why not nim dump main ?

that's a separate command, it doesn't compose with other commands (eg nim c main) and can't be used for such purposes

why not nim c --genScript main ?

--genScript serves a different purpose (generating a compile script), and doesn't compose with other commands (eg nim js or nim r or nim check etc); it also modifies behavior of nim c as it doesn't actually generate object files and a binary but instead is used as precursor for another command

That said, a lot of the code can be shared and reused in the implementation of --dump since it also generates a json file in nimcache

[Varriount]

I wonder if a reevaluation of Nim's CLI interface isn't in order. Not necessarily a redesign, but just some analysis for issues and redundancies - looking at the current full help is... a lot.

Rather than dumping a selection of metadata to a file, what about:

• Allowing compiler output to be in 2 formats: human-readable and JSON.
• Allowing metadata (like dependencies) to be part of the compiler output.