cmd/compile, cmd/link: generate dwarf type info in compiler

[zhouguangyuan0718]

Background

I have focused on dynamic link (plugin,linkshared,buildemode-shared) in golang for two years. I noticed that the dynlink library can't be debuged. And I know that the reason of it is the dwarf type info is generated in linker.
Related issues:
#38378
#44982

Later, I saw this in #47788 (comment).

@thanm said:

One thing we can do that would make the problem more tractable in general would be to move DWARF type DIE generation out of the linker and into the compiler (this is an idea that we've toyed with the in the past but haven't gotten to). Doing that would definitely help for the "link shared against libstd.so" use case.

Thanks for @thanm , I have tried to implement generating dwarf type info in compile for a long time. Now it can work, though it is a very very initial version. I think it is the correct time to submit a proposal about it, and maybe I can improve it and contribute it to go community.

Proposal

The dwarf type info can be generated in compiler instead of linker. My initial version implement it with this way:

1. We can collect all the types (global var, local var, const and the type which runtime type should be keeped) used in current compile unit.
2. Do "defgotype" like the linker now. And we don't need to decode the infomation from binary like linker, we only need to get the info from types.Type struct in compile.
3. Synthesize map type, chan type and soon. Because some proto type (runtime.hchan, runtime.hmap...) are not in current compile unit, we must make some "stub" type of them.
4. Before dumpobj in compiler, convert all the type dies to LSym and put them to data section.

I will send a CL to share my very very initial prototype.

If this proposal is a little likely to be accepted, I will implement it continue. I think thers is much things todo.
TODO:

1. Clarify the dwarf symbol name and type name.
2. Use aux sym for typedef sym instead of lookup it by name.
3. Investigate why fixedbugs/ssue30908.go failed. It seems like there is some different duplicate dwarf symbols.
4. Distinguish linkshared and static link. For linkshared, generate completely dwarf types for every compile unit. For static link, only generate dwarf types that defined in current compile unit.Then the user who use -linkshared, They could debug the dynlink library.

Costs

• What is the compile time cost?
  Longer. But it will not much because of parallel compile. And maybe we can get a shorter link time.
• What is the run time cost?
  No cost.
• Can you describe a possible implementation?
  Above
• Do you have a prototype? (This is not required.)
  Yes.

Update 2022.5.10:

List an intial version todo list for formal modification:

• Implement some base interface for dwarf in compiler
  • CL 399294
  • CL 399059
  • CL 399063
  • CL 399064
  • CL 399301
• generate dwarf info sym for the type (except the type need to be synthesized) defined in current compile unit in compier.
  • CL 399302
• Create some type description for synthesize type in compiler, thus we can generate and synthesize more type info in compiler.
  • CL 399877
• add some util functiong for synthesizing type.
  • CL 399879
  • CL 399879
  • CL 399880
  • CL 405455
• synthesize ptr, hchan, hmap, string and slice in compiler.
  • CL 400137
• use the dwarf sym generated in compiler in linker.
  • CL 399275
• cleanup some redundancy and duplicate code
  • CL 400138
• generate whole dwarf type info in compiler for dynlink.
  • CL 400634
  • CL 404734
• do not generate the types which must be emited by runtime
  • CL 403334
• add a new reloc type for using aux symbol for type info
• refactor and optimize the DWDIE struct to reduce the pointer
• unify the die create way in internal/dwarf
• add testcases for these cls
  • CL 404217
  • CL 404754
  • CL 404755

[gopherbot]

Change https://go.dev/cl/398735 mentions this issue: cmd/compile,link: generating dwarf type info in compiler

[ianlancetaylor]

There is no user visible change here (except that debug info is better), so this doesn't have to go through the proposal process. The linker/compiler/runtime team can decide what to do here.

CC @golang/runtime

[zhouguangyuan0718]

I will send CL tree to implement it formally.

[gopherbot]

Change https://go.dev/cl/399059 mentions this issue: cmd/link,compile: refacte dwarf generation of linker

[gopherbot]

Change https://go.dev/cl/399294 mentions this issue: cmd/internal/dwarf: define interface for dwarf type info

[gopherbot]

Change https://go.dev/cl/399062 mentions this issue: cmd/link: extract newtype1 function

[gopherbot]

Change https://go.dev/cl/399295 mentions this issue: cmd/link: complate decoupling for newtype

[gopherbot]

Change https://go.dev/cl/399063 mentions this issue: cmd/link,cmd/internal/dwarf: move newtype to dwarf package