Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile, cmd/link: generate dwarf type info in compiler #52209

Open
9 of 13 tasks
zhouguangyuan0718 opened this issue Apr 7, 2022 · 65 comments
Open
9 of 13 tasks

cmd/compile, cmd/link: generate dwarf type info in compiler #52209

zhouguangyuan0718 opened this issue Apr 7, 2022 · 65 comments
Labels
NeedsInvestigation
Milestone

Comments

@zhouguangyuan0718
Copy link
Contributor

@zhouguangyuan0718 zhouguangyuan0718 commented Apr 7, 2022

Background

I have focused on dynamic link (plugin,linkshared,buildemode-shared) in golang for two years. I noticed that the dynlink library can't be debuged. And I know that the reason of it is the dwarf type info is generated in linker.
Related issues:
#38378
#44982

Later, I saw this in #47788 (comment).

@thanm said:

One thing we can do that would make the problem more tractable in general would be to move DWARF type DIE generation out of the linker and into the compiler (this is an idea that we've toyed with the in the past but haven't gotten to). Doing that would definitely help for the "link shared against libstd.so" use case.

Thanks for @thanm , I have tried to implement generating dwarf type info in compile for a long time. Now it can work, though it is a very very initial version. I think it is the correct time to submit a proposal about it, and maybe I can improve it and contribute it to go community.

Proposal

The dwarf type info can be generated in compiler instead of linker. My initial version implement it with this way:

  1. We can collect all the types (global var, local var, const and the type which runtime type should be keeped) used in current compile unit.
  2. Do "defgotype" like the linker now. And we don't need to decode the infomation from binary like linker, we only need to get the info from types.Type struct in compile.
  3. Synthesize map type, chan type and soon. Because some proto type (runtime.hchan, runtime.hmap...) are not in current compile unit, we must make some "stub" type of them.
  4. Before dumpobj in compiler, convert all the type dies to LSym and put them to data section.

I will send a CL to share my very very initial prototype.

If this proposal is a little likely to be accepted, I will implement it continue. I think thers is much things todo.
TODO:

  1. Clarify the dwarf symbol name and type name.
  2. Use aux sym for typedef sym instead of lookup it by name.
  3. Investigate why fixedbugs/ssue30908.go failed. It seems like there is some different duplicate dwarf symbols.
  4. Distinguish linkshared and static link. For linkshared, generate completely dwarf types for every compile unit. For static link, only generate dwarf types that defined in current compile unit.Then the user who use -linkshared, They could debug the dynlink library.

Costs

  • What is the compile time cost?
    Longer. But it will not much because of parallel compile. And maybe we can get a shorter link time.
  • What is the run time cost?
    No cost.
  • Can you describe a possible implementation?
    Above
  • Do you have a prototype? (This is not required.)
    Yes.

Update 2022.5.10:

List an intial version todo list for formal modification:

  • Implement some base interface for dwarf in compiler
  • generate dwarf info sym for the type (except the type need to be synthesized) defined in current compile unit in compier.
  • Create some type description for synthesize type in compiler, thus we can generate and synthesize more type info in compiler.
  • add some util functiong for synthesizing type.
  • synthesize ptr, hchan, hmap, string and slice in compiler.
  • use the dwarf sym generated in compiler in linker.
  • cleanup some redundancy and duplicate code
  • generate whole dwarf type info in compiler for dynlink.
  • do not generate the types which must be emited by runtime
  • add a new reloc type for using aux symbol for type info
  • refactor and optimize the DWDIE struct to reduce the pointer
  • unify the die create way in internal/dwarf
  • add testcases for these cls
@gopherbot gopherbot added this to the Proposal milestone Apr 7, 2022
@gopherbot
Copy link

@gopherbot gopherbot commented Apr 7, 2022

Change https://go.dev/cl/398735 mentions this issue: cmd/compile,link: generating dwarf type info in compiler

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Apr 7, 2022

There is no user visible change here (except that debug info is better), so this doesn't have to go through the proposal process. The linker/compiler/runtime team can decide what to do here.

CC @golang/runtime

@ianlancetaylor ianlancetaylor changed the title proposal: cmd/compile,link: generating dwarf type info in compiler cmd/compile, cmd/link: generate dwarf type info in compiler Apr 7, 2022
@ianlancetaylor ianlancetaylor added NeedsInvestigation and removed Proposal labels Apr 7, 2022
@ianlancetaylor ianlancetaylor removed this from the Proposal milestone Apr 7, 2022
@ianlancetaylor ianlancetaylor added this to the Backlog milestone Apr 7, 2022
@zhouguangyuan0718
Copy link
Contributor Author

@zhouguangyuan0718 zhouguangyuan0718 commented Apr 9, 2022

I will send CL tree to implement it formally.

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 9, 2022

Change https://go.dev/cl/399059 mentions this issue: cmd/link,compile: refacte dwarf generation of linker

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 9, 2022

Change https://go.dev/cl/399294 mentions this issue: cmd/internal/dwarf: define interface for dwarf type info

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 9, 2022

Change https://go.dev/cl/399062 mentions this issue: cmd/link: extract newtype1 function

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 9, 2022

Change https://go.dev/cl/399295 mentions this issue: cmd/link: complate decoupling for newtype

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 9, 2022

Change https://go.dev/cl/399063 mentions this issue: cmd/link,cmd/internal/dwarf: move newtype to dwarf package

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 9, 2022

Change https://go.dev/cl/399064 mentions this issue: cmd/compile: implement dwarf.Type interface for types.Type

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 9, 2022

Change https://go.dev/cl/399296 mentions this issue: cmd/internal/dwarf: clarify symbol name and dwarfname

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 10, 2022

Change https://go.dev/cl/399297 mentions this issue: cmd/internal/dwarf: clarify Type and Sym

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 10, 2022

Change https://go.dev/cl/399298 mentions this issue: cmd/link,obj/dwarf: unify DefGoType and DefPtrTo

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 10, 2022

Change https://go.dev/cl/399301 mentions this issue: cmd/internal/obj/dwarf: implement some base method for dwCtxt

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 10, 2022

Change https://go.dev/cl/399302 mentions this issue: cmd/compile: generate dwarf type info as aux sym of type

@thanm
Copy link
Contributor

@thanm thanm commented Apr 11, 2022

@zhouguangyuan0718 thanks for writing this up and sending a first set of patches. Let me know when they are ready for review.

Moving type generation from linker to compiler is definitely a worthy project. If I were doing this work myself, there are some other cleanups I would also consider folding in as well. Specifically:

  • the way the linker currently does type DIE generation is more "C-like" than it should be (basically since the code was originally written in C and then converted/translated to Go). The DWDie/DWAttr setup is also more memory-intensive than it needs to be (these structs also have a lot of pointers, making extra work for the GC).

  • it is also confusing to have two entirely separate ways of creating DWARF DIEs in the the toolchain (e.g. the code that emits subprogram and related DIEs has one way of creating the DWARF, and the linker code has an entirely different way via DWDie/DWAttr). Ideally we would want to unify these two.

With that said, these cleanups are independent and can be tackled separately at some future time.

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 11, 2022

Change https://go.dev/cl/399275 mentions this issue: cmd/link: use the dwarf type info generated by compiler

@zhouguangyuan0718
Copy link
Contributor Author

@zhouguangyuan0718 zhouguangyuan0718 commented Apr 11, 2022

@thanm thanks for your reply. I'm glad to know it is a worthy project.
Now it can simply work with CL 399275 (with some bugs I’m investigating) and the relation chain of it. At this time, the dwarf type info is generated in both compiler and linker. In compiler, generate the type difined in current compile unit except the synthesize type. In linker, generate all the others info. I will move the remaining step by step.
To avoid my idea is on the wrong track, Could you take a little look for the current patchsets ?It begin with CL 399059. Thanks very much.

I want to cleanups something, too. I didn't add duplicate code in compiler, I moved some code from linker to cmd/internal/dwarf. We can use them in both compiler and linker. In the future, it can be unified more easy.

@thanm
Copy link
Contributor

@thanm thanm commented Apr 12, 2022

To avoid my idea is on the wrong track, Could you take a little look for the current patchsets ?It begin with CL 399059. Thanks very much.

I took a very quick skim. In general looks to be moving in the right direction.

I am not sure about https://go-review.googlesource.com/c/go/+/399302 however-- what is the purpose of making DWARF type die symbols into aux symbols attached to type symbols? If the type DIE symbol is an aux, it means you can't look it up by name. This seems to me that it will force a lot of rewriting/post-processing in the linker.

@zhouguangyuan0718
Copy link
Contributor Author

@zhouguangyuan0718 zhouguangyuan0718 commented Apr 12, 2022

it means you can't look it up by name.

Sorry, maybe I use a incorrect combine of aux and pkgDef for dwarf sym? It seems I can still look it up by name now. And I can also get them by aux of a type symbol.

what is the purpose of making DWARF type die symbols into aux symbols attached to type symbols?

According to the understanding I described above, I hope that all the dwarf type can be collected by using aux sym of reachable gotype and traverse the relocs of them at last. May be it is more fast than name lookup.

If I missed something which can prevent doing as this, I will put them in data.

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 12, 2022

Change https://go.dev/cl/399877 mentions this issue: cmd/compile: add a generator for synthesize dwarf type

@thanm
Copy link
Contributor

@thanm thanm commented Apr 12, 2022

It doesn't really make sense to have a symbol that is both aux and pkgdef -- the primary reason we make the other DWARF sub-symbols (ex: DWARFLINES, DWARFFCN) is that we never have to look them up by name. Better to have the DWARF type DIE symbols just be regular named symbols.

@zhouguangyuan0718
Copy link
Contributor Author

@zhouguangyuan0718 zhouguangyuan0718 commented Apr 12, 2022

Thank you for explanation. Maybe dealing with the reloction of them correctly is a little dificult. I met many problem about relocations at the beginning. Particularly, some dwarf type symbol are nameless and duplicate. However, I will try to make them as regular symbols continue.

The other thing I can't do with certainty is CL 399877. Should I do it like this? Or put them into cmd/compile/internal/typecheck/builtin, then fix mkbuiltin.go and deal with them by a common way? Could you give me some advice?

@zhouguangyuan0718
Copy link
Contributor Author

@zhouguangyuan0718 zhouguangyuan0718 commented Apr 18, 2022

I use bent to test the build time for master and my patchset, seems a little slower now. I will investigate this.

bent -v -N=0 -a=5 -L=bentjobs.log -C=configurations-cmpjob.toml benchstat old-740a490f71 new-2e81957480
benchstat old-740a490f71 new-2e81957480
name                                         old build-real-time/op  new build-real-time/op  delta
Gonum_lapack_native                                      2.58s ± 3%              2.58s ± 1%     ~     (p=1.000 n=5+5)
Nelsam_gxui_interval                                     184ms ± 5%              176ms ± 1%     ~     (p=0.111 n=5+4)
Gonum_blas_native                                        1.42s ± 3%              1.52s ± 4%   +7.24%  (p=0.008 n=5+5)
Ajstarks_deck_generate                                   139ms ± 4%              138ms ± 4%     ~     (p=1.000 n=5+5)
Cespare_mph                                              103ms ±21%               97ms ±14%     ~     (p=0.548 n=5+5)
Ethereum_ethash                                          7.24s ± 2%              7.71s ±10%     ~     (p=0.310 n=5+5)
Uber_zap                                                 1.34s ± 4%              1.53s ±18%  +14.22%  (p=0.032 n=5+5)
Benhoyt_goawk                                            442ms ± 3%              459ms ± 3%   +4.02%  (p=0.032 n=5+5)
K8s_cache                                                6.25s ±17%              6.33s ± 9%     ~     (p=0.548 n=5+5)
Gonum_topo                                               2.74s ± 8%              2.87s ±28%     ~     (p=0.841 n=5+5)
Hugo_helpers                                             8.39s ± 5%              8.38s ± 4%     ~     (p=0.841 n=5+5)
Ericlagergren_decimal                                    370ms ±18%              368ms ± 3%     ~     (p=0.310 n=5+5)
Semver                                                   169ms ±13%              171ms ± 7%     ~     (p=0.841 n=5+5)
Capnproto2                                               4.25s ± 4%              4.52s ± 3%   +6.39%  (p=0.008 n=5+5)
Dustin_broadcast                                        97.5ms ±17%             94.4ms ± 7%     ~     (p=0.690 n=5+5)
Aws_jsonrpc                                              2.28s ± 5%              2.36s ± 8%     ~     (p=0.421 n=5+5)
Klauspost                                               91.9ms ± 6%             96.2ms ±14%     ~     (p=0.548 n=5+5)
Ethereum_ecies                                           5.84s ± 1%              5.83s ± 3%     ~     (p=0.548 n=5+5)
Aws_restjson                                             2.59s ±16%              2.75s ±21%     ~     (p=0.151 n=5+5)
Ethereum_trie                                            6.96s ± 5%              7.15s ± 4%     ~     (p=0.222 n=5+5)
Spexs2                                                   286ms ± 5%              295ms ±20%     ~     (p=1.000 n=5+5)
Gonum_mat                                                3.03s ± 1%              3.05s ± 1%     ~     (p=0.556 n=5+4)
Ethereum_bitutil                                         156ms ± 5%              156ms ± 4%     ~     (p=0.690 n=5+5)
Gonum_path                                               2.64s ± 6%              2.74s ± 9%     ~     (p=0.095 n=5+5)
Bindata                                                  668ms ± 0%              701ms ± 7%   +4.91%  (p=0.016 n=4+5)
Dustin_humanize                                          180ms ± 9%              186ms ±17%     ~     (p=0.690 n=5+5)
K8s_workqueue                                            528ms ±22%              524ms ± 5%     ~     (p=0.421 n=5+5)
Cespare_xxhash                                           133ms ±16%              139ms ±19%     ~     (p=0.548 n=5+5)
Kanzi                                                    522ms ± 4%              518ms ± 2%     ~     (p=0.690 n=5+5)
Ethereum_core                                            8.24s ± 1%              8.64s ± 7%   +4.77%  (p=0.008 n=5+5)
Gtank_blake2s                                            136ms ±16%              126ms ± 7%     ~     (p=0.841 n=5+5)
Gonum_community                                          2.79s ± 1%              2.82s ± 3%     ~     (p=0.310 n=5+5)
Aws_restxml                                              2.52s ± 6%              2.86s ±17%  +13.61%  (p=0.032 n=5+5)
Hugo_hugolib                                             13.1s ± 2%              13.3s ± 5%     ~     (p=0.548 n=5+5)
Aws_jsonutil                                             1.65s ±15%              1.61s ± 6%     ~     (p=1.000 n=5+5)
Commonmark_markdown                                      3.59s ± 3%              4.54s ± 5%  +26.56%  (p=0.008 n=5+5)
Ethereum_corevm                                          7.60s ± 6%              7.58s ± 9%     ~     (p=0.548 n=5+5)
Shopify_sarama                                           2.88s ±10%              2.98s ± 3%     ~     (p=0.222 n=5+5)
Bloom_bloom                                              182ms ±25%              177ms ± 6%     ~     (p=0.421 n=5+5)
Gonum_traverse                                           2.49s ±12%              2.48s ± 4%     ~     (p=0.690 n=5+5)

name old build-user-time/op new build-user-time/op delta
Gonum_lapack_native 6.86s ± 4% 6.82s ± 3% ~ (p=0.310 n=5+5)
Nelsam_gxui_interval 346ms ± 6% 353ms ±22% ~ (p=0.690 n=5+5)
Gonum_blas_native 4.05s ± 2% 4.23s ± 5% ~ (p=0.222 n=5+5)
Ajstarks_deck_generate 266ms ± 5% 260ms ± 7% ~ (p=0.548 n=5+5)
Cespare_mph 187ms ±28% 179ms ±11% ~ (p=0.841 n=5+5)
Ethereum_ethash 14.8s ± 2% 16.6s ±13% ~ (p=0.095 n=5+5)
Uber_zap 4.47s ± 4% 5.07s ±16% +13.38% (p=0.032 n=5+5)
Benhoyt_goawk 927ms ± 5% 936ms ± 5% ~ (p=0.548 n=5+5)
K8s_cache 24.5s ±18% 24.3s ± 9% ~ (p=0.548 n=5+5)
Gonum_topo 8.54s ±11% 8.09s ± 2% ~ (p=0.286 n=5+4)
Hugo_helpers 44.8s ± 3% 44.4s ± 2% ~ (p=0.690 n=5+5)
Ericlagergren_decimal 1.37s ±21% 1.32s ± 4% ~ (p=1.000 n=5+5)
Semver 352ms ±13% 350ms ± 8% ~ (p=1.000 n=5+5)
Capnproto2 11.2s ± 4% 11.6s ± 2% ~ (p=0.056 n=5+5)
Dustin_broadcast 193ms ±22% 177ms ± 7% ~ (p=0.310 n=5+5)
Aws_jsonrpc 5.82s ± 4% 5.96s ± 3% ~ (p=0.310 n=5+5)
Klauspost 167ms ±11% 171ms ±17% ~ (p=1.000 n=5+5)
Ethereum_ecies 6.31s ± 1% 6.33s ± 3% ~ (p=0.841 n=5+5)
Aws_restjson 6.67s ±21% 7.04s ±20% ~ (p=0.095 n=5+5)
Ethereum_trie 13.2s ± 8% 13.6s ±11% ~ (p=0.548 n=5+5)
Spexs2 818ms ± 2% 842ms ±19% ~ (p=0.548 n=5+5)
Gonum_mat 9.51s ± 3% 9.55s ± 0% ~ (p=0.556 n=5+4)
Ethereum_bitutil 321ms ± 5% 312ms ± 6% ~ (p=0.222 n=5+5)
Gonum_path 8.00s ± 3% 7.96s ± 1% ~ (p=0.556 n=5+4)
Bindata 907ms ± 1% 957ms ±10% +5.57% (p=0.016 n=4+5)
Dustin_humanize 388ms ± 9% 410ms ±20% ~ (p=0.421 n=5+5)
K8s_workqueue 1.69s ±20% 1.66s ± 4% ~ (p=0.548 n=5+5)
Cespare_xxhash 296ms ±17% 302ms ±18% ~ (p=0.690 n=5+5)
Kanzi 1.34s ± 5% 1.31s ± 3% ~ (p=0.310 n=5+5)
Ethereum_core 19.0s ± 1% 19.6s ± 1% +3.10% (p=0.016 n=5+4)
Gtank_blake2s 248ms ±13% 228ms ± 2% ~ (p=0.222 n=5+5)
Gonum_community 8.39s ± 2% 8.36s ± 3% ~ (p=0.690 n=5+5)
Aws_restxml 6.46s ± 5% 7.22s ±18% ~ (p=0.222 n=5+5)
Hugo_hugolib 74.0s ± 1% 75.3s ± 3% ~ (p=0.056 n=5+5)
Aws_jsonutil 3.29s ±17% 3.24s ± 4% ~ (p=0.421 n=5+5)
Commonmark_markdown 5.23s ± 7% 6.18s ± 8% +18.11% (p=0.008 n=5+5)
Ethereum_corevm 15.6s ±11% 15.4s ± 9% ~ (p=0.841 n=5+5)
Shopify_sarama 9.46s ±14% 9.43s ± 3% ~ (p=0.421 n=5+5)
Bloom_bloom 420ms ±23% 401ms ± 5% ~ (p=1.000 n=5+5)
Gonum_traverse 7.47s ±13% 7.31s ± 3% ~ (p=1.000 n=5+5)

name old build-sys-time/op new build-sys-time/op delta
Gonum_lapack_native 376ms ± 7% 380ms ±10% ~ (p=0.841 n=5+5)
Nelsam_gxui_interval 93.8ms ± 9% 102.5ms ±12% ~ (p=0.151 n=5+5)
Gonum_blas_native 272ms ± 6% 293ms ± 4% +7.61% (p=0.016 n=5+5)
Ajstarks_deck_generate 94.4ms ±16% 93.7ms ±17% ~ (p=1.000 n=5+5)
Cespare_mph 87.0ms ±10% 83.5ms ±14% ~ (p=0.690 n=5+5)
Ethereum_ethash 1.23s ± 3% 1.34s ± 9% ~ (p=0.095 n=5+5)
Uber_zap 404ms ± 5% 430ms ±15% ~ (p=0.421 n=5+5)
Benhoyt_goawk 148ms ± 8% 159ms ± 2% +7.56% (p=0.016 n=5+5)
K8s_cache 1.48s ±14% 1.54s ± 4% ~ (p=0.222 n=5+5)
Gonum_topo 683ms ± 6% 683ms ±21% ~ (p=0.548 n=5+5)
Hugo_helpers 2.42s ± 8% 2.35s ± 3% ~ (p=0.310 n=5+5)
Ericlagergren_decimal 194ms ±12% 204ms ± 9% ~ (p=0.421 n=5+5)
Semver 92.8ms ±13% 103.5ms ± 9% ~ (p=0.095 n=5+5)
Capnproto2 580ms ± 6% 647ms ±11% +11.50% (p=0.008 n=5+5)
Dustin_broadcast 83.2ms ±23% 86.2ms ±10% ~ (p=0.548 n=5+5)
Aws_jsonrpc 647ms ± 4% 642ms ±11% ~ (p=0.690 n=5+5)
Klauspost 84.9ms ± 7% 90.0ms ±15% ~ (p=0.421 n=5+5)
Ethereum_ecies 496ms ± 5% 484ms ± 6% ~ (p=0.310 n=5+5)
Aws_restjson 675ms ±10% 703ms ±12% ~ (p=0.548 n=5+5)
Ethereum_trie 1.02s ±11% 1.05s ±14% ~ (p=0.421 n=5+5)
Spexs2 201ms ±11% 207ms ±20% ~ (p=1.000 n=5+5)
Gonum_mat 498ms ± 6% 523ms ±13% ~ (p=0.421 n=5+5)
Ethereum_bitutil 106ms ± 7% 105ms ±16% ~ (p=1.000 n=5+5)
Gonum_path 596ms ± 5% 644ms ± 8% +8.10% (p=0.016 n=5+5)
Bindata 158ms ±17% 155ms ±12% ~ (p=0.841 n=5+5)
Dustin_humanize 94.7ms ±16% 94.2ms ±10% ~ (p=1.000 n=5+5)
K8s_workqueue 266ms ±20% 250ms ± 9% ~ (p=0.421 n=5+5)
Cespare_xxhash 113ms ±19% 111ms ±16% ~ (p=0.841 n=5+5)
Kanzi 171ms ±11% 172ms ± 7% ~ (p=0.841 n=5+5)
Ethereum_core 1.44s ± 2% 1.52s ±11% ~ (p=0.056 n=5+5)
Gtank_blake2s 91.0ms ±18% 92.4ms ±14% ~ (p=0.548 n=5+5)
Gonum_community 620ms ± 6% 631ms ± 7% ~ (p=0.548 n=5+5)
Aws_restxml 682ms ± 2% 737ms ±23% ~ (p=0.690 n=5+5)
Hugo_hugolib 4.42s ± 3% 4.46s ± 2% ~ (p=0.548 n=5+5)
Aws_jsonutil 333ms ±10% 319ms ±11% ~ (p=0.421 n=5+5)
Commonmark_markdown 318ms ± 7% 323ms ±11% ~ (p=0.841 n=5+5)
Ethereum_corevm 1.23s ± 7% 1.20s ± 5% ~ (p=0.548 n=5+5)
Shopify_sarama 773ms ± 8% 774ms ± 1% ~ (p=1.000 n=5+5)
Bloom_bloom 109ms ±31% 104ms ± 7% ~ (p=0.690 n=5+5)
Gonum_traverse 579ms ±15% 569ms ± 4% ~ (p=1.000 n=5+5)

name old zdebug-bytes new zdebug-bytes delta
gonum_lapack_native_zdebug_total 0.00B 0.00B ~ (all equal)
nelsam_gxui_interval_zdebug_total 0.00B 0.00B ~ (all equal)
gonum_blas_native_zdebug_total 0.00B 0.00B ~ (all equal)
ajstarks_deck_generate_zdebug_total 0.00B 0.00B ~ (all equal)
cespare_mph_zdebug_total 0.00B 0.00B ~ (all equal)
ethereum_ethash_zdebug_total 0.00B 0.00B ~ (all equal)
uber_zap_zdebug_total 0.00B 0.00B ~ (all equal)
benhoyt_goawk_zdebug_total 0.00B 0.00B ~ (all equal)
k8s_cache_zdebug_total 0.00B 0.00B ~ (all equal)
gonum_topo_zdebug_total 0.00B 0.00B ~ (all equal)
hugo_helpers_zdebug_total 0.00B 0.00B ~ (all equal)
ericlagergren_decimal_zdebug_total 0.00B 0.00B ~ (all equal)
semver_zdebug_total 0.00B 0.00B ~ (all equal)
capnproto2_zdebug_total 0.00B 0.00B ~ (all equal)
dustin_broadcast_zdebug_total 0.00B 0.00B ~ (all equal)
aws_jsonrpc_zdebug_total 0.00B 0.00B ~ (all equal)
klauspost_zdebug_total 0.00B 0.00B ~ (all equal)
ethereum_ecies_zdebug_total 0.00B 0.00B ~ (all equal)
aws_restjson_zdebug_total 0.00B 0.00B ~ (all equal)
ethereum_trie_zdebug_total 0.00B 0.00B ~ (all equal)
spexs2_zdebug_total 0.00B 0.00B ~ (all equal)
gonum_mat_zdebug_total 0.00B 0.00B ~ (all equal)
ethereum_bitutil_zdebug_total 0.00B 0.00B ~ (all equal)
gonum_path_zdebug_total 0.00B 0.00B ~ (all equal)
bindata_zdebug_total 0.00B 0.00B ~ (all equal)
dustin_humanize_zdebug_total 0.00B 0.00B ~ (all equal)
k8s_workqueue_zdebug_total 0.00B 0.00B ~ (all equal)
cespare_xxhash_zdebug_total 0.00B 0.00B ~ (all equal)
kanzi_zdebug_total 0.00B 0.00B ~ (all equal)
ethereum_core_zdebug_total 0.00B 0.00B ~ (all equal)
gtank_blake2s_zdebug_total 0.00B 0.00B ~ (all equal)
gonum_community_zdebug_total 0.00B 0.00B ~ (all equal)
aws_restxml_zdebug_total 0.00B 0.00B ~ (all equal)
hugo_hugolib_zdebug_total 0.00B 0.00B ~ (all equal)
aws_jsonutil_zdebug_total 0.00B 0.00B ~ (all equal)
commonmark_markdown_zdebug_total 0.00B 0.00B ~ (all equal)
ethereum_corevm_zdebug_total 0.00B 0.00B ~ (all equal)
shopify_sarama_zdebug_total 0.00B 0.00B ~ (all equal)
bloom_bloom_zdebug_total 0.00B 0.00B ~ (all equal)
gonum_traverse_zdebug_total 0.00B 0.00B ~ (all equal)

name old inputs-quality new inputs-quality delta
gonum_lapack_native_dwarf_input_goodness 0.50 ± 0% 0.50 ± 0% ~ (all equal)
nelsam_gxui_interval_dwarf_input_goodness 0.52 ± 0% 0.53 ± 0% ~ (p=1.000 n=1+1)
gonum_blas_native_dwarf_input_goodness 0.49 ± 0% 0.50 ± 0% ~ (p=1.000 n=1+1)
ajstarks_deck_generate_dwarf_input_goodness 0.52 ± 0% 0.52 ± 0% ~ (all equal)
cespare_mph_dwarf_input_goodness 0.52 ± 0% 0.52 ± 0% ~ (all equal)
ethereum_ethash_dwarf_input_goodness 0.52 ± 0% 0.52 ± 0% ~ (all equal)
uber_zap_dwarf_input_goodness 0.54 ± 0% 0.54 ± 0% ~ (all equal)
benhoyt_goawk_dwarf_input_goodness 0.53 ± 0% 0.53 ± 0% ~ (all equal)
k8s_cache_dwarf_input_goodness 0.54 ± 0% 0.54 ± 0% ~ (all equal)
gonum_topo_dwarf_input_goodness 0.51 ± 0% 0.51 ± 0% ~ (all equal)
hugo_helpers_dwarf_input_goodness 0.55 ± 0% 0.55 ± 0% ~ (all equal)
ericlagergren_decimal_dwarf_input_goodness 0.53 ± 0% 0.53 ± 0% ~ (all equal)
semver_dwarf_input_goodness 0.53 ± 0% 0.53 ± 0% ~ (all equal)
capnproto2_dwarf_input_goodness 0.49 ± 0% 0.49 ± 0% ~ (all equal)
dustin_broadcast_dwarf_input_goodness 0.52 ± 0% 0.52 ± 0% ~ (all equal)
aws_jsonrpc_dwarf_input_goodness 0.52 ± 0% 0.52 ± 0% ~ (all equal)
klauspost_dwarf_input_goodness 0.52 ± 0% 0.52 ± 0% ~ (all equal)
ethereum_ecies_dwarf_input_goodness 0.52 ± 0% 0.53 ± 0% ~ (p=1.000 n=1+1)
aws_restjson_dwarf_input_goodness 0.52 ± 0% 0.52 ± 0% ~ (all equal)
ethereum_trie_dwarf_input_goodness 0.54 ± 0% 0.54 ± 0% ~ (all equal)
spexs2_dwarf_input_goodness 0.53 ± 0% 0.53 ± 0% ~ (all equal)
gonum_mat_dwarf_input_goodness 0.49 ± 0% 0.49 ± 0% ~ (all equal)
ethereum_bitutil_dwarf_input_goodness 0.52 ± 0% 0.52 ± 0% ~ (all equal)
gonum_path_dwarf_input_goodness 0.51 ± 0% 0.51 ± 0% ~ (all equal)
bindata_dwarf_input_goodness 0.54 ± 0% 0.54 ± 0% ~ (all equal)
dustin_humanize_dwarf_input_goodness 0.53 ± 0% 0.54 ± 0% ~ (p=1.000 n=1+1)
k8s_workqueue_dwarf_input_goodness 0.54 ± 0% 0.55 ± 0% ~ (p=1.000 n=1+1)
cespare_xxhash_dwarf_input_goodness 0.52 ± 0% 0.52 ± 0% ~ (all equal)
kanzi_dwarf_input_goodness 0.52 ± 0% 0.52 ± 0% ~ (all equal)
ethereum_core_dwarf_input_goodness 0.52 ± 0% 0.53 ± 0% ~ (p=1.000 n=1+1)
gtank_blake2s_dwarf_input_goodness 0.52 ± 0% 0.53 ± 0% ~ (p=1.000 n=1+1)
gonum_community_dwarf_input_goodness 0.48 ± 0% 0.49 ± 0% ~ (p=1.000 n=1+1)
aws_restxml_dwarf_input_goodness 0.52 ± 0% 0.52 ± 0% ~ (all equal)
hugo_hugolib_dwarf_input_goodness 0.52 ± 0% 0.53 ± 0% ~ (p=1.000 n=1+1)
aws_jsonutil_dwarf_input_goodness 0.54 ± 0% 0.54 ± 0% ~ (all equal)
commonmark_markdown_dwarf_input_goodness 0.53 ± 0% 0.54 ± 0% ~ (p=1.000 n=1+1)
ethereum_corevm_dwarf_input_goodness 0.54 ± 0% 0.55 ± 0% ~ (p=1.000 n=1+1)
shopify_sarama_dwarf_input_goodness 0.53 ± 0% 0.53 ± 0% ~ (all equal)
bloom_bloom_dwarf_input_goodness 0.53 ± 0% 0.53 ± 0% ~ (all equal)
gonum_traverse_dwarf_input_goodness 0.52 ± 0% 0.52 ± 0% ~ (all equal)

name old args-quality new args-quality delta
gonum_lapack_native_dwarf_args_goodness 0.86 ± 0% 0.86 ± 0% ~ (all equal)
nelsam_gxui_interval_dwarf_args_goodness 0.91 ± 0% 0.91 ± 0% ~ (all equal)
gonum_blas_native_dwarf_args_goodness 0.90 ± 0% 0.90 ± 0% ~ (all equal)
ajstarks_deck_generate_dwarf_args_goodness 0.92 ± 0% 0.92 ± 0% ~ (all equal)
cespare_mph_dwarf_args_goodness 0.92 ± 0% 0.92 ± 0% ~ (all equal)
uber_zap_dwarf_args_goodness 0.87 ± 0% 0.87 ± 0% ~ (all equal)
benhoyt_goawk_dwarf_args_goodness 0.92 ± 0% 0.92 ± 0% ~ (all equal)
k8s_cache_dwarf_args_goodness 0.85 ± 0% 0.85 ± 0% ~ (p=1.000 n=1+1)
gonum_topo_dwarf_args_goodness 0.91 ± 0% 0.91 ± 0% ~ (all equal)
hugo_helpers_dwarf_args_goodness 0.84 ± 0% 0.84 ± 0% ~ (p=1.000 n=1+1)
ericlagergren_decimal_dwarf_args_goodness 0.90 ± 0% 0.90 ± 0% ~ (all equal)
semver_dwarf_args_goodness 0.92 ± 0% 0.92 ± 0% ~ (all equal)
capnproto2_dwarf_args_goodness 0.92 ± 0% 0.92 ± 0% ~ (all equal)
dustin_broadcast_dwarf_args_goodness 0.92 ± 0% 0.92 ± 0% ~ (all equal)
aws_jsonrpc_dwarf_args_goodness 0.87 ± 0% 0.87 ± 0% ~ (p=1.000 n=1+1)
klauspost_dwarf_args_goodness 0.92 ± 0% 0.92 ± 0% ~ (all equal)
aws_restjson_dwarf_args_goodness 0.87 ± 0% 0.87 ± 0% ~ (p=1.000 n=1+1)
spexs2_dwarf_args_goodness 0.91 ± 0% 0.91 ± 0% ~ (all equal)
gonum_mat_dwarf_args_goodness 0.84 ± 0% 0.84 ± 0% ~ (all equal)
ethereum_bitutil_dwarf_args_goodness 0.91 ± 0% 0.91 ± 0% ~ (all equal)
gonum_path_dwarf_args_goodness 0.89 ± 0% 0.89 ± 0% ~ (all equal)
bindata_dwarf_args_goodness 0.91 ± 0% 0.91 ± 0% ~ (all equal)
dustin_humanize_dwarf_args_goodness 0.90 ± 0% 0.90 ± 0% ~ (all equal)
k8s_workqueue_dwarf_args_goodness 0.86 ± 0% 0.86 ± 0% ~ (p=1.000 n=1+1)
cespare_xxhash_dwarf_args_goodness 0.92 ± 0% 0.92 ± 0% ~ (all equal)
kanzi_dwarf_args_goodness 0.91 ± 0% 0.91 ± 0% ~ (all equal)
gtank_blake2s_dwarf_args_goodness 0.92 ± 0% 0.92 ± 0% ~ (all equal)
gonum_community_dwarf_args_goodness 0.87 ± 0% 0.87 ± 0% ~ (all equal)
aws_restxml_dwarf_args_goodness 0.86 ± 0% 0.87 ± 0% ~ (p=1.000 n=1+1)
hugo_hugolib_dwarf_args_goodness 0.80 ± 0% 0.80 ± 0% ~ (p=1.000 n=1+1)
aws_jsonutil_dwarf_args_goodness 0.90 ± 0% 0.90 ± 0% ~ (all equal)
commonmark_markdown_dwarf_args_goodness 0.92 ± 0% 0.92 ± 0% ~ (all equal)
shopify_sarama_dwarf_args_goodness 0.85 ± 0% 0.85 ± 0% ~ (all equal)
bloom_bloom_dwarf_args_goodness 0.92 ± 0% 0.92 ± 0% ~ (all equal)
gonum_traverse_dwarf_args_goodness 0.91 ± 0% 0.91 ± 0% ~ (all equal)

name old stmts-quality new stmts-quality delta
gonum_lapack_native_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (all equal)
nelsam_gxui_interval_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (p=1.000 n=1+1)
gonum_blas_native_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (p=1.000 n=1+1)
ajstarks_deck_generate_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (all equal)
cespare_mph_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (p=1.000 n=1+1)
ethereum_ethash_dwarf_stmt_goodness 0.97 ± 0% 0.97 ± 0% ~ (p=1.000 n=1+1)
uber_zap_dwarf_stmt_goodness 0.95 ± 0% 0.95 ± 0% ~ (p=1.000 n=1+1)
benhoyt_goawk_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (all equal)
k8s_cache_dwarf_stmt_goodness 0.98 ± 0% 0.98 ± 0% ~ (p=1.000 n=1+1)
gonum_topo_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (p=1.000 n=1+1)
hugo_helpers_dwarf_stmt_goodness 0.98 ± 0% 0.98 ± 0% ~ (p=1.000 n=1+1)
ericlagergren_decimal_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (p=1.000 n=1+1)
semver_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (p=1.000 n=1+1)
capnproto2_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (all equal)
dustin_broadcast_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (all equal)
aws_jsonrpc_dwarf_stmt_goodness 0.97 ± 0% 0.97 ± 0% ~ (p=1.000 n=1+1)
klauspost_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (all equal)
ethereum_ecies_dwarf_stmt_goodness 0.92 ± 0% 0.92 ± 0% ~ (p=1.000 n=1+1)
aws_restjson_dwarf_stmt_goodness 0.97 ± 0% 0.97 ± 0% ~ (p=1.000 n=1+1)
ethereum_trie_dwarf_stmt_goodness 0.95 ± 0% 0.95 ± 0% ~ (p=1.000 n=1+1)
spexs2_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (all equal)
gonum_mat_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (p=1.000 n=1+1)
ethereum_bitutil_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (p=1.000 n=1+1)
gonum_path_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (all equal)
bindata_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (all equal)
dustin_humanize_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (all equal)
k8s_workqueue_dwarf_stmt_goodness 0.95 ± 0% 0.95 ± 0% ~ (p=1.000 n=1+1)
cespare_xxhash_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (all equal)
kanzi_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (all equal)
ethereum_core_dwarf_stmt_goodness 0.97 ± 0% 0.97 ± 0% ~ (p=1.000 n=1+1)
gtank_blake2s_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (p=1.000 n=1+1)
gonum_community_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (p=1.000 n=1+1)
aws_restxml_dwarf_stmt_goodness 0.97 ± 0% 0.97 ± 0% ~ (p=1.000 n=1+1)
hugo_hugolib_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (p=1.000 n=1+1)
aws_jsonutil_dwarf_stmt_goodness 0.94 ± 0% 0.94 ± 0% ~ (p=1.000 n=1+1)
commonmark_markdown_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (p=1.000 n=1+1)
ethereum_corevm_dwarf_stmt_goodness 0.95 ± 0% 0.95 ± 0% ~ (p=1.000 n=1+1)
shopify_sarama_dwarf_stmt_goodness 0.97 ± 0% 0.97 ± 0% ~ (p=1.000 n=1+1)
bloom_bloom_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (all equal)
gonum_traverse_dwarf_stmt_goodness 0.99 ± 0% 0.99 ± 0% ~ (p=1.000 n=1+1)

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 19, 2022

Change https://go.dev/cl/400874 mentions this issue: cmd/compile: do not dump incorrect prototype when using -linkshared

myy2829 pushed a commit to myy2829/go that referenced this issue Apr 19, 2022
… code with specific implement

To generate dwarf type info in compiler, I move some functions in cmd/link/internal/ld/dwarf.go to cmd/internal/dwarf, because they are independent of linker. No functional change now. They will be used in compiler later.

For golang#52209

Change-Id: I10c9bbaf530f84f3fe4d94687a9325b665a7a9ce

cmd/internal/dwarf: define interface for dwarf type info

Dwarf type info can be generated based on the abstract type. We can refact it to be independent of linker with "abstract type".

For golang#52209

Change-Id: I4d8fd3c19bcb9a22cdb037fb5f2b05e1a5907e23

[release-branch.go1.17] cmd/link: extract newtype1 function

No functional change, the logic of generating dwarf type info will be moved to cmd/internal/dwarf. So decouple it with the specific implement with linker. Not complete now.

Note: the dwsym will implement dwarf.Type interface temporarily, it will be removed at last.

For golang#52209

Change-Id: I2de0d9e47c78529e6021b5f74375f21bdb4ea070

[release-branch.go1.17] cmd/link: complete decoupling for newtype

Implement the interface dwarf.Type for dwsym, and use it in newtype1. Now, newtype1 is no related to any specific code with linker, it will moved to cmd/internal/dwarf next cl.

note:
Using dwsym as interface will cost a lot of time because of duplicate
function call. But in the future, all the dwarf type info will be generated in compiler, these will be removed.

For golang#52209

Change-Id: I30165d0bb7722924ed1f306a2807fabf3eb4c22f

cmd/link,cmd/internal/dwarf: move newtype to dwarf package

cmd/link/internal/ld/dwarf.go:/^func.newtype1 --> cmd/internal/dwarf/dwarf.go:/^func.NewType

For golang#52209

Change-Id: Ie95eff712418ee454a9c3887dcd7e7488a42b011

cmd/compile: implement dwarf.Type interface for types.Type

Implement cmd/internal/dwarf.Type interface fpr cmd/compile/internal/types.Type with a wrapper.

For golang#52209

Change-Id: I46ce4735bfcf900d9e17c14b6f69ea542eac0a8a

cmd/internal/dwarf: clarify symbol name and dwarfname

In compiler, the name for die must be exactly name, not the name with "". So distinguish them in NewDie

For golang#52209

Change-Id: I51320fd7a3f2fdba768535d514e00f3a707aaac9

cmd/internal/obj/dwarf: implement some base method for dwCtxt

Only generate type info for types defined in current compile unit. So only emit a reloc for subtype when DefGoType.
And add a stub type for eface and iface type, only need their name to be referenced.

For golang#52209

Change-Id: If723e0d9588c1e82cf1a129ac4c8ec7eb3b1d98a

cmd/compile: generate dwarf type info in compiler

Generate dwarf type info and dump it to objfile. It is not be used in linker now.
Some types need to synthesize and the prototypes are still generated in linker. I will resolve it in the future.

For golang#52209

Change-Id: Ie7c67197f930462ae4474f291e02d3009b6651bc

cmd/link: use the dwarf type info generated by compiler

It's time to begin using the dwarf type info generated by compiler in linker. Though dwinfo of many types is generated by compiler, but It is still necessary to check these types with fuction subtype, because the subtype of them is not defined.

For golang#52209

Change-Id: Ic12f4ed1f510454880d6ef9c8608b405c5404a92

cmd/link: refactor mkinternaltype and cleanup DefPtrTo

move mkinternaltype to internal/dwarf, then we can decouple synthesizeXXXtypes continue.

For golang#52209

Change-Id: I5d038c1f8f97fae36fb18d474ec78fb8269db584

cmd/link: decouple synthesizeXXXtypes with specific code in linker

They will be moved to internal/dwarf/dwarf.go in next CL, this CL is for we can review it more easily.

For golang#52209

Change-Id: I27831f65f998f5bc100182f598be25870ac916da

cmd/link,internal/dwarf: move synthesizeXXXtypes to internal/dwarf

Only a code move.

For golang#52209

Change-Id: I58f6c401898c2d3350fdf51d17ca059ec829dfa5

cmd/link: unify the key of prototypedies in linker and compiler

Remove the "type." prefix of the keys.The keys in compiler didn't prefix with "type."

For golang#52209

Change-Id: Ibd0b10f8a2b6ded1a157b87cea0ca397be6cf848

[release-branch.go1.17] cmd/compile: add ptototype of the types need synthesized to builtin

To synthesize dwarf type in compiler, the prototype of string, map, slice, channel is needed. It can't be easily got when compile the code out of runtime. So use add them to builtin so we can get the prototype information.

More detailed explanation:
When a map(eg: map[int]int) is used, we need more dwarf type info than we can see in language level. The pseudo type hash<int,int>, bucket<int,int>, []val<int> and []key<int> will be generated for debug more easy. They are "template" type, we need to fill them when a map is instantiated. And we can't define these type directly by exist go runtime type. So they should be added to buitin for filling the "template" when generating the dwarf type info of them.
So does others.

See golang#52209 (comment)

For golang#52209

Change-Id: Ifcb1e15d2300323980b320673adb67df2ee07956

[release-branch.go1.17] cmd/compile: synthesize types in compiler

Now, we can synthesize types in compiler. And all the dwarf type sym can be kept by relocs. subtype in linker can be removed now. There is some undifined relocs to runtime type in dwarf type.
These relocs are added by compiler, it don't know the corresponding runtime type will be defined or not. So skip them in relocsym.

For golang#52209

Change-Id: Ief7683f8d9033ebf0f682f582e2fffde63587bf0

[release-branch.go1.17] cmd/link: remove the dwarf type info generation code in linker

Now, all the dwarf type info is generated in compiler, it is not necessary to keep these code in linker, remove it.

For golang#52209

Change-Id: I9cfe253bd4b4ea7d93d0c4cfbf6d309f566f4552

cmd/compile,internal/dwarf: cleanup some code

Expect no functional change.
1. Make some function in cmd/internal/dwarf unexport. No one call them out of the package.
2. Store some data and status on the dwctxt in internal/obj, instead of many global variables.
3. Remove the flag dwarfType I added before, use unified dwarf flag.

For golang#52209

Change-Id: I7683e53b6d7c7d015fc8b8bc778b72fe495ae2e4

cmd/compile: lazy load prototype die when synthesizing types

Do not need to prepare prototype die when compiling any compile unit.
Generate them only when we need.

For golang#52209

Change-Id: I4f8191ed7c3f9866580f574bfce6a939157d15f2

[release-branch.go1.17] cmd/compile,link: support generating dwarf info when -linkshared

Now, all the dwarf type can be generated by compiler, when using static link, every compiler unit generate its own dwarf type die. when using -linkshared, all the types current compile unit need can be collected. So we can generate corresponding dwarf type info for it and subtypes of it.
For synthesizetype, we also need to keep the type info of them when linkshared

For golang#52209

Change-Id: Ia70ba39708ce710c45d923d5b86dac91d7e9d616

cmd/compile: do not dump incorrect prototype when using -linkshared

the prototype must be dumped when runtime is compiled. when we compiling a non runtime package, it should keep minimal type info for prototypes, so, do not dump the incorrect prototype, such as runtime.g, runtime._type and soon. Replace them as a uintptr.

For golang#52209

Change-Id: Ib33ff92b5be50bf376bdf1931b726929a6186a71
myy2829 pushed a commit to myy2829/go that referenced this issue Apr 19, 2022
… code with specific implement

To generate dwarf type info in compiler, I move some functions in cmd/link/internal/ld/dwarf.go to cmd/internal/dwarf, because they are independent of linker. No functional change now. They will be used in compiler later.

For golang#52209

Change-Id: I10c9bbaf530f84f3fe4d94687a9325b665a7a9ce

cmd/internal/dwarf: define interface for dwarf type info

Dwarf type info can be generated based on the abstract type. We can refact it to be independent of linker with "abstract type".

For golang#52209

Change-Id: I4d8fd3c19bcb9a22cdb037fb5f2b05e1a5907e23

[release-branch.go1.17] cmd/link: extract newtype1 function

No functional change, the logic of generating dwarf type info will be moved to cmd/internal/dwarf. So decouple it with the specific implement with linker. Not complete now.

Note: the dwsym will implement dwarf.Type interface temporarily, it will be removed at last.

For golang#52209

Change-Id: I2de0d9e47c78529e6021b5f74375f21bdb4ea070

[release-branch.go1.17] cmd/link: complete decoupling for newtype

Implement the interface dwarf.Type for dwsym, and use it in newtype1. Now, newtype1 is no related to any specific code with linker, it will moved to cmd/internal/dwarf next cl.

note:
Using dwsym as interface will cost a lot of time because of duplicate
function call. But in the future, all the dwarf type info will be generated in compiler, these will be removed.

For golang#52209

Change-Id: I30165d0bb7722924ed1f306a2807fabf3eb4c22f

cmd/link,cmd/internal/dwarf: move newtype to dwarf package

cmd/link/internal/ld/dwarf.go:/^func.newtype1 --> cmd/internal/dwarf/dwarf.go:/^func.NewType

For golang#52209

Change-Id: Ie95eff712418ee454a9c3887dcd7e7488a42b011

cmd/compile: implement dwarf.Type interface for types.Type

Implement cmd/internal/dwarf.Type interface fpr cmd/compile/internal/types.Type with a wrapper.

For golang#52209

Change-Id: I46ce4735bfcf900d9e17c14b6f69ea542eac0a8a

cmd/internal/dwarf: clarify symbol name and dwarfname

In compiler, the name for die must be exactly name, not the name with "". So distinguish them in NewDie

For golang#52209

Change-Id: I51320fd7a3f2fdba768535d514e00f3a707aaac9

cmd/internal/obj/dwarf: implement some base method for dwCtxt

Only generate type info for types defined in current compile unit. So only emit a reloc for subtype when DefGoType.
And add a stub type for eface and iface type, only need their name to be referenced.

For golang#52209

Change-Id: If723e0d9588c1e82cf1a129ac4c8ec7eb3b1d98a

cmd/compile: generate dwarf type info in compiler

Generate dwarf type info and dump it to objfile. It is not be used in linker now.
Some types need to synthesize and the prototypes are still generated in linker. I will resolve it in the future.

For golang#52209

Change-Id: Ie7c67197f930462ae4474f291e02d3009b6651bc

cmd/link: use the dwarf type info generated by compiler

It's time to begin using the dwarf type info generated by compiler in linker. Though dwinfo of many types is generated by compiler, but It is still necessary to check these types with fuction subtype, because the subtype of them is not defined.

For golang#52209

Change-Id: Ic12f4ed1f510454880d6ef9c8608b405c5404a92

cmd/link: refactor mkinternaltype and cleanup DefPtrTo

move mkinternaltype to internal/dwarf, then we can decouple synthesizeXXXtypes continue.

For golang#52209

Change-Id: I5d038c1f8f97fae36fb18d474ec78fb8269db584

cmd/link: decouple synthesizeXXXtypes with specific code in linker

They will be moved to internal/dwarf/dwarf.go in next CL, this CL is for we can review it more easily.

For golang#52209

Change-Id: I27831f65f998f5bc100182f598be25870ac916da

cmd/link,internal/dwarf: move synthesizeXXXtypes to internal/dwarf

Only a code move.

For golang#52209

Change-Id: I58f6c401898c2d3350fdf51d17ca059ec829dfa5

cmd/link: unify the key of prototypedies in linker and compiler

Remove the "type." prefix of the keys.The keys in compiler didn't prefix with "type."

For golang#52209

Change-Id: Ibd0b10f8a2b6ded1a157b87cea0ca397be6cf848

[release-branch.go1.17] cmd/compile: add ptototype of the types need synthesized to builtin

To synthesize dwarf type in compiler, the prototype of string, map, slice, channel is needed. It can't be easily got when compile the code out of runtime. So use add them to builtin so we can get the prototype information.

More detailed explanation:
When a map(eg: map[int]int) is used, we need more dwarf type info than we can see in language level. The pseudo type hash<int,int>, bucket<int,int>, []val<int> and []key<int> will be generated for debug more easy. They are "template" type, we need to fill them when a map is instantiated. And we can't define these type directly by exist go runtime type. So they should be added to buitin for filling the "template" when generating the dwarf type info of them.
So does others.

See golang#52209 (comment)

For golang#52209

Change-Id: Ifcb1e15d2300323980b320673adb67df2ee07956

[release-branch.go1.17] cmd/compile: synthesize types in compiler

Now, we can synthesize types in compiler. And all the dwarf type sym can be kept by relocs. subtype in linker can be removed now. There is some undifined relocs to runtime type in dwarf type.
These relocs are added by compiler, it don't know the corresponding runtime type will be defined or not. So skip them in relocsym.

For golang#52209

Change-Id: Ief7683f8d9033ebf0f682f582e2fffde63587bf0

[release-branch.go1.17] cmd/link: remove the dwarf type info generation code in linker

Now, all the dwarf type info is generated in compiler, it is not necessary to keep these code in linker, remove it.

For golang#52209

Change-Id: I9cfe253bd4b4ea7d93d0c4cfbf6d309f566f4552

cmd/compile,internal/dwarf: cleanup some code

Expect no functional change.
1. Make some function in cmd/internal/dwarf unexport. No one call them out of the package.
2. Store some data and status on the dwctxt in internal/obj, instead of many global variables.
3. Remove the flag dwarfType I added before, use unified dwarf flag.

For golang#52209

Change-Id: I7683e53b6d7c7d015fc8b8bc778b72fe495ae2e4

cmd/compile: lazy load prototype die when synthesizing types

Do not need to prepare prototype die when compiling any compile unit.
Generate them only when we need.

For golang#52209

Change-Id: I4f8191ed7c3f9866580f574bfce6a939157d15f2

[release-branch.go1.17] cmd/compile,link: support generating dwarf info when -linkshared

Now, all the dwarf type can be generated by compiler, when using static link, every compiler unit generate its own dwarf type die. when using -linkshared, all the types current compile unit need can be collected. So we can generate corresponding dwarf type info for it and subtypes of it.
For synthesizetype, we also need to keep the type info of them when linkshared

For golang#52209

Change-Id: Ia70ba39708ce710c45d923d5b86dac91d7e9d616

cmd/compile: do not dump incorrect prototype when using -linkshared

the prototype must be dumped when runtime is compiled. when we compiling a non runtime package, it should keep minimal type info for prototypes, so, do not dump the incorrect prototype, such as runtime.g, runtime._type and soon. Replace them as a uintptr.

For golang#52209

Change-Id: Ib33ff92b5be50bf376bdf1931b726929a6186a71
@zhouguangyuan0718
Copy link
Contributor Author

@zhouguangyuan0718 zhouguangyuan0718 commented Apr 19, 2022

Hi! @thanm , I have refactored my work tree. I remove much transitional code. And make changes aggregated in every cl. I think it will be more easy to review.
Now, I think all the tasks I marked in the content of this issue are done and they are ready to be reviewed. You can review them if you are available sometimes. I will do the remain tasks continue. Thank you.

zhouguangyuan0718 added a commit to zhouguangyuan0718/go that referenced this issue Apr 20, 2022
Dwarf type info can be generated based on the abstract type. We can make it to be independent of specific implement. And define a TypeContext interface to provide some base method during dwarf type info generating.

For golang#52209

Change-Id: I4d8fd3c19bcb9a22cdb037fb5f2b05e1a5907e23
zhouguangyuan0718 added a commit to zhouguangyuan0718/go that referenced this issue Apr 20, 2022
Implement cmd/internal/dwarf.Type interface fpr cmd/compile/internal/types.Type with a wrapper.

For golang#52209

Change-Id: I46ce4735bfcf900d9e17c14b6f69ea542eac0a8a
zhouguangyuan0718 added a commit to zhouguangyuan0718/go that referenced this issue Apr 20, 2022
…r dwCtxt

Only generate type info for types defined in current compile unit. So only emit a reloc for subtype when reference it.

For golang#52209

Change-Id: If723e0d9588c1e82cf1a129ac4c8ec7eb3b1d98
zhouguangyuan0718 added a commit to zhouguangyuan0718/go that referenced this issue Apr 20, 2022
To generate dwarf type info in compiler, I copy some functions in cmd/link/internal/ld/dwarf.go to cmd/internal/dwarf, and  refactor them to be independent of linker. No functional change now. They will be used in compiler later.
And the duplicate code in linker will be removed at last.

For golang#52209

Change-Id: I10c9bbaf530f84f3fe4d94687a9325b665a7a9ce
zhouguangyuan0718 added a commit to zhouguangyuan0718/go that referenced this issue Apr 20, 2022
cmd/link/internal/ld/dwarf.go:/^func.newtype --> cmd/internal/dwarf/dwarf.go:/^func.NewType

And refactor it to adapt Type interface. newtype in cmd/link/internal/ld/dwarf.go will be removed at last.

For golang#52209

Change-Id: Ie95eff712418ee454a9c3887dcd7e7488a42b011
zhouguangyuan0718 added a commit to zhouguangyuan0718/go that referenced this issue Apr 20, 2022
Generate dwarf type info and dump it to objfile. It is not be used in linker now.
Some types need to be synthesized. I will resolve it in next CLs.

For golang#52209

Change-Id: Ie7c67197f930462ae4474f291e02d3009b6651bc
zhouguangyuan0718 added a commit to zhouguangyuan0718/go that referenced this issue Apr 20, 2022
These functions are copied from cmd\link\internal\ld\dwarf.go. Refactor them to adapt Type and TypeContext interface.

For golang#52209

Change-Id: I27831f65f998f5bc100182f598be25870ac916da
zhouguangyuan0718 added a commit to zhouguangyuan0718/go that referenced this issue Apr 20, 2022
These code is also copied and refactored from cmd/link/internal/ld/dwarf.go

For golang#52209

Change-Id: I58f6c401898c2d3350fdf51d17ca059ec829dfa5
zhouguangyuan0718 added a commit to zhouguangyuan0718/go that referenced this issue Apr 20, 2022
To synthesize dwarf type in compiler, the prototype of string, map, slice, channel is needed. It can't be easily got when compile the code out of runtime. So use add them to builtin so we can get the prototype information.

More detailed explanation:
When a map(eg: map[int]int) is used, we need more dwarf type info than we can see in language level. The pseudo type hash<int,int>, bucket<int,int>, []val<int> and []key<int> will be generated for debug more easy. They are "template" type, we need to fill them when a map is instantiated. And we can't define these type directly by exist go runtime type. So they should be added to buitin for filling the "template" when generating the dwarf type info of them.
So does others.

See golang#52209 (comment)

For golang#52209

Change-Id: Ifcb1e15d2300323980b320673adb67df2ee07956
zhouguangyuan0718 added a commit to zhouguangyuan0718/go that referenced this issue Apr 20, 2022
Now, we can synthesize types in compiler. And all the dwarf type sym can be kept by relocs. subtype in linker can be removed now. There is some undifined relocs to runtime type in dwarf type.
These relocs are added by compiler, it don't know the corresponding runtime type will be defined or not. So skip them in relocsym.

For golang#52209

Change-Id: Ief7683f8d9033ebf0f682f582e2fffde63587bf0
zhouguangyuan0718 added a commit to zhouguangyuan0718/go that referenced this issue Apr 20, 2022
It's time to begin using the dwarf type info generated by compiler in linker.

For golang#52209

Change-Id: Ic12f4ed1f510454880d6ef9c8608b405c5404a92
zhouguangyuan0718 added a commit to zhouguangyuan0718/go that referenced this issue Apr 20, 2022
Now, all the dwarf type info is generated in compiler, it is not necessary to keep these code in linker, remove it.

For golang#52209

Change-Id: I9cfe253bd4b4ea7d93d0c4cfbf6d309f566f4552
zhouguangyuan0718 added a commit to zhouguangyuan0718/go that referenced this issue Apr 20, 2022
…fo when -linkshared

Now, all the dwarf type can be generated by compiler, when using static link, every compiler unit generate its own dwarf type die. when using -linkshared, all the types current compile unit need can be collected. So we can generate corresponding dwarf type info for it and subtypes of it.
For synthesizetype, we also need to keep the type info of them when linkshared

For golang#52209

Change-Id: Ia70ba39708ce710c45d923d5b86dac91d7e9d616
@thanm
Copy link
Contributor

@thanm thanm commented Apr 21, 2022

Thanks. I'll take a look once I clear up some time.

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 26, 2022

Change https://go.dev/cl/402258 mentions this issue: internal/dwarf: create dupok sym when create internal type

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 30, 2022

Change https://go.dev/cl/403334 mentions this issue: cmd/compile: only emit basic dwarf type info for runtime package

@thanm
Copy link
Contributor

@thanm thanm commented May 4, 2022

I'll take a look once I clear up some time.

Still tied up with pre-release-freeze development work. Hope to look at this next week.

@gopherbot
Copy link

@gopherbot gopherbot commented May 5, 2022

Change https://go.dev/cl/404217 mentions this issue: cmd/compile: add some testcases for dwarf types

@zhouguangyuan0718
Copy link
Contributor Author

@zhouguangyuan0718 zhouguangyuan0718 commented May 5, 2022

I'll take a look once I clear up some time.

Still tied up with pre-release-freeze development work. Hope to look at this next week.

Thanks, and I wonder if it is possible to be merged into 1.19? If it is possible, I will do the remain things as soon as possible.

@gopherbot
Copy link

@gopherbot gopherbot commented May 7, 2022

Change https://go.dev/cl/404734 mentions this issue: cmd/compile: fix interface print in debugger for dynamic link

@gopherbot
Copy link

@gopherbot gopherbot commented May 7, 2022

Change https://go.dev/cl/404754 mentions this issue: misc/cgo/testshared: add testcases for dwarf info of buildmode shared

@gopherbot
Copy link

@gopherbot gopherbot commented May 7, 2022

Change https://go.dev/cl/404755 mentions this issue: cmd/link: add testcases for dwarf of linkshared in TestDWARF

@gopherbot
Copy link

@gopherbot gopherbot commented May 10, 2022

Change https://go.dev/cl/405455 mentions this issue: cmd/compile: move constants of type map to objabi

@zhouguangyuan0718
Copy link
Contributor Author

@zhouguangyuan0718 zhouguangyuan0718 commented May 11, 2022

$ gotip build -a -debug-trace old.out cmd/go
$ ~/01.Code/00.godev/go/bin/go build -a -debug-trace new.out cmd/go

new.txt
old.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation
Projects
None yet
Development

No branches or pull requests

6 participants