proposal: cmd/compile: expose -ptabs/-exporttypes flag to force compiler to emit ptabs/type info

[eh-steve]

Currently compiler only emits ptabs for package main with -dynlink, for -buildmode=plugin. Access to type information for package exports may be useful in other contexts, and is a fairly unobtrusive change.

[ianlancetaylor]

CC @golang/runtime

[ianlancetaylor]

Is this a format that we want to commit to supporting forever?

[eh-steve]

Good question, not sure if it's aimed at me though...? Do all gcflags fall under the compatibility guarantee? This doesn't change any behaviour in the current compiler, and I don't see how it differs from other flags like DebugFlags.PCTab... Unless you meant the ptab binary format?

[ianlancetaylor]

The compiler flags don't fall under the compatibility guarantee, but still once we start supporting something it's hard to stop.

[cherrymui]

Could you explain more of your use case? I think what (and whether) to emit and the format would depend on how the data will be consumed. Thanks.

[eh-steve]

I thought I'd submitted my reply to Keith but just noticed it was still in draft.

[ianlancetaylor]

Thanks--can you copy that text to this issue?

[eh-steve]

Sure, I’ve been working on a JIT compiler (https://github.com/eh-steve/goloader/tree/master/jit#usage-and-configuration) which can build, load and unload arbitrary Go code and its dependencies from a running host binary, using the standard go toolchain (but bypassing the final link step - it uses archives directly from the compiler). It differs from std plugin because code can be unloaded/replaced, symbols are deduplicated across the host binary and the JIT code where possible (without needing the host binary to mark all methods reachable, unlike plugin), and it works with or without CGo (no dependency on libdl), and therefore on Windows.

My (unsatisfying, but working) approach so far has been to first parse the JIT package and use the *syntax.FuncDecl and *syntax.VarDecl nodes to find all exports, and generate a function for each export to retrieve its *_type via reflection, and writing this temporary generated file into the package before actually compiling. This requires the package source path to be writable, and to ensure this is possible, I introduced a bunch of unnecessary module copying.

Since this is an optional gcflag, it doesn't change the current compiler behaviour except when set, and doesn't prevent linker deadcode elimination, except for selected packages where the flag is set.

I think the previous proposal wanted to always export ptabs for all exports in the main binary's moduledata to provide a string-to-function lookup (preventing deadcode elimination) which I agree would be unnecessary.

[eh-steve]

As an alternative, I'd also be happy with a flag to tell the compiler to emit type data for global functions by skipping the ir.PFUNC check in cmd/compile/internal/gc.dumpGlobal(), if we wanted to avoid depending on the ptab binary format (i.e. emit GoAuxTypes)

[randall77]

dumpGlobal just affects DWARF. Is that all you need?