Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Go compiler and runtime meeting notes #43930

Open
jeremyfaller opened this issue Jan 26, 2021 · 14 comments
Open

Go compiler and runtime meeting notes #43930

jeremyfaller opened this issue Jan 26, 2021 · 14 comments

Comments

@jeremyfaller
Copy link
Contributor

@jeremyfaller jeremyfaller commented Jan 26, 2021

Google's Go compiler and runtime team meets periodically (roughly weekly) to discuss ongoing development of the compiler and runtime. While not open to the public, there's been desire by the community to learn what the compiler and runtime team is working on. While we learn what is a good format, and what works and doesn't, we will start publishing meeting minutes here.

These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

This meta-issue records minutes of Google's Go compiler and runtime meetings as issue comments, so that they can be cross-linked easily with the relevant issues. This meta-issue is for minutes only; comments that are not meeting minutes will be deleted.

@jeremyfaller
Copy link
Contributor Author

@jeremyfaller jeremyfaller commented Jan 26, 2021

2021-01-19

These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

  • We continued a discussion about increasing the bootstrap version of Go from 1.4 to 1.16 after the 1.17 release. We wanted to make sure it wasn't hard for people to build from scratch, but we would like to use features that have been added over time. Consensus was that this would be okay to submit as a formal proposal.
  • We discussed Austin's benchmark unit proposal. Consensus was hopeful that it would be accepted.
  • We discussed the weak maps proposal. A number of corner cases, gotchas, and subtleties were discussed. Consensus was that the discussion on the issue was up to date.
  • After falling off the prior week's agenda, we began a discussion about releasing the compiler and runtime meeting notes. We all felt communication with the community was important, and we would start with an internal publication of meeting notes. No dates were set for a final decision.
@jeremyfaller
Copy link
Contributor Author

@jeremyfaller jeremyfaller commented Jan 26, 2021

2021-01-26

These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

  • We continued discussing whether or not there were any concerns with publishing compiler and runtime meeting notes. No issues were raised, so it's likely I will go ahead and start publishing the notes.
  • We discussed the redirecting crash output proposal. No major concerns were raised, but consensus was that a call would be best approach (against environment variables, etc).
  • We discussed the pain merging the development branches (master -> dev.regabi -> dev.typeparams). The consensus was that without acceptance of the generics proposal, we need to keep living with the pain.
  • We briefly discussed using the type checker in dev.typeparams as the default type checker. Using that type checker is blocked on either acceptance of generics proposal or a separate proposal to move to a new type checker.
@ianlancetaylor ianlancetaylor changed the title proposal: Go compiler and runtime meeting notes Go compiler and runtime meeting notes Jan 26, 2021
@ianlancetaylor ianlancetaylor modified the milestones: Proposal, Unplanned Jan 26, 2021
@golang golang deleted a comment from mvdan Jan 26, 2021
@golang golang deleted a comment from mvdan Jan 26, 2021
@golang golang deleted a comment from ianlancetaylor Jan 26, 2021
@jeremyfaller
Copy link
Contributor Author

@jeremyfaller jeremyfaller commented Feb 2, 2021

2021-02-02
These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

  • We continued our discussion of moving to the new default typechecker on dev.typeparams. Engineering effort continues at a break-neck pace, and we're hopeful we'll be able to have this early.
  • We discussed the issue with spinning M's, and have seen large improvements for unique workloads within Google. Consensus was that this was an excellent find.
  • We had a brief discussion of integer constant resolution.
  • We briefly discussed merging the dev.regabi branch back to master when the tree opens. Consensus was that it would be safe to merge when the tree opened.
  • We discussed some recent additions to dev.typeparams, that added support for generic functions and types. It's possible the backend work will happen on the branch's compiler soon.
@jeremyfaller
Copy link
Contributor Author

@jeremyfaller jeremyfaller commented Feb 9, 2021

2021-02-08

These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

  • We briefly discussed the status of 1.16.
  • We discussed the tree reopening, and landing of the dev.regabi branch, and the dev.typeparms branches on master. Current plans are to merge dev.regabi during the first week. If both proposals (1, 2) are accepted, dev.typeparams will also be merged during that week.
  • We discussed the new pacer proposal, and its benefits.
  • We also discussed the status of the register abi. While functionality is gated behind a flag, we don't want to go into code-freeze, enabling the ABI that late. We discussed the amount of soak time we think we will need.
@jeremyfaller
Copy link
Contributor Author

@jeremyfaller jeremyfaller commented Feb 16, 2021

2021-02-16

These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

  • We had a small celebration about the acceptance of the Generics Proposal.
  • We discussed merging the dev.typeparams branch to master. There are public API typechecking changes on that branch that aren't stable. As a result, we will likely hide the APIs using buildtags.
  • We continued a long-running discussion on making types2 the default typechecker. We still don't need to make a decision here, so back into the oven it goes.
  • We discussed revising the syntax for typelists in generics. An internally floated proposal is similar to public proposals. More to come.
  • We discussed adding attendance to entries in this document. As no one objected to having their name added, I will start doing that. I won't start by CC-ing people, just linking to their github profile for now.

Attendees:
Carlos Amedee
David Chase
Austin Clements
Matthew Dempsky
Jeremy Faller
Robert Griesemer
Than McIntosh
Martin Möhrmann
Patrik Nyblom
Michael Pratt
Keith Randall
Dan Scales
Ian Lance Taylor
Cherry Zhang

@jeremyfaller
Copy link
Contributor Author

@jeremyfaller jeremyfaller commented Feb 23, 2021

2021-02-23

These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

  • We celebrated the reopening of tree for 1.17 development.
  • We discussed some of the features we want to land for 1.17, including the register ABI, foundational work for generics, the new gc pacer, improvements to the scheduler, escape analysis improvements, and possibly some phase changes to the compiler. We also had a more in-depth discussion of subtle points in types2.
  • We discussed the work on register ABI. A hello world is working (in review, not fully committed). More work for return values has also started, and we're having some success there.
  • A new ABI with G in R14 and a zeroed X15 has shown reasonable promise to offer good runtime performance improvements. Work is ongoing to confirm results, and clean up some poorly understood overhead in wrappers. Also, not all of the places where we use G in assembly have been converted over. More data to come.
  • We discussed an issue that turning on stack poisoning seems to bork the openbsd-amd64-68 trybot.
  • We discussed a proposal to add zeroing for cryptographic functions. Consensus was that this is non-trivial.

Attendees:
Carlos Amedee
David Chase
Austin Clements
Matthew Dempsky
Jeremy Faller
Robert Griesemer
Than McIntosh
Martin Möhrmann
Patrik Nyblom
Michael Pratt
Keith Randall
Dan Scales
Ian Lance Taylor
Cherry Zhang

@jeremyfaller
Copy link
Contributor Author

@jeremyfaller jeremyfaller commented Mar 3, 2021

2021-03-02

These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

  • We discussed the current status of our 1.17 and 1.18 goals, including the register ABI, and generics.
  • We discussed a small issue on sum types. Likely more external communication to come.
  • We discussed how best to handle the meeting notes (this issue), and questions and feedback. We decided it would be best to move queries to the mailing list(s) for now.
  • We discussed moving GOEXPERIMENT, and how important it would be to help the registerABI work to land.
  • We discussed a subtle issue in the register ABI to simplify the internal representation and requirements for go & defer. Specifically, having the compiler rewrite defer f(x) ⇒ x' := x; defer func() { f(x') }().

Attendees:
David Chase
Austin Clements
Matthew Dempsky
Jeremy Faller
Robert Griesemer
Than McIntosh
Patrik Nyblom
Michael Pratt
Keith Randall
Dan Scales
Cherry Zhang

@golang golang deleted a comment from josharian Mar 3, 2021
@jeremyfaller
Copy link
Contributor Author

@jeremyfaller jeremyfaller commented Mar 9, 2021

2021-03-09
These compiler and runtime meeting minutes are under development. We welcome feedback on content, format, level of detail, timeliness, and so on. If the minutes are helpful, please let us know. If they are less than helpful, we welcome constructive comments on how to improve them.

Notes on agenda items that address Google specific needs are elided.

  • Again, we discussed the status of our 1.17 and 1.18 goals.
  • We hope that move GOEXPERIMENT knob is accepted, as the current plan with the register ABI is to use it for turnup.
  • We discussed the progress on type lists. An internal proposal likely addresses all concerns, and an external document will be available.

Attendees:
David Chase
Austin Clements
Matthew Dempsky
Jeremy Faller
Robert Griesemer
Than McIntosh
Martin Möhrmann
Patrik Nyblom
Michael Pratt
Keith Randall
Dan Scales
Ian Lance Taylor
Cherry Zhang

@jeremyfaller
Copy link
Contributor Author

@jeremyfaller jeremyfaller commented Mar 17, 2021

2021-03-16

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

  • We discussed typelists again.
  • We discussed possibly updating MIPS to MIPS32r2. The thinking is that R2's been around for a long time (10years?), and there is significant performance to be gained in the crypto libraries if we adopt it. Previous attempts have failed because of Loongson users. Discussion will likely continue.
  • We discussed how to address amd64 per-architecture optimizations. Haswell specific optimizations have shown large improvements in certain workloads, and it might be worth providing for these wins. Discussion will likely continue.

Attendees:
The notetaker would like to apologize for leaving off Michael Knyszek from previous attendance. I don't have a good system yet, and it's quite manual.
David Chase
Austin Clements
Matthew Dempsky
Jeremy Faller
Robert Griesemer
Michael Knyszek
Than McIntosh
Martin Möhrmann
Patrik Nyblom
Michael Pratt
Keith Randall
Dan Scales
Ian Lance Taylor
Cherry Zhang

@jeremyfaller
Copy link
Contributor Author

@jeremyfaller jeremyfaller commented Mar 24, 2021

2021-03-23

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

  • Most of the meeting was a discussion about 1.18.
  • We discussed changes to the standard library for generics in 1.18
  • We discussed followon work for the register ABI (beyond amd64).
  • We discussed measuring PGO and other GC (Immix) ideas.

Attendees:
I was absent from the meeting, and unable to grab the screenshot.

@jeremyfaller
Copy link
Contributor Author

@jeremyfaller jeremyfaller commented Apr 1, 2021

2021-03-30

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

  • We discussed the remaining outstanding work, now that we're 1 month from the code freeze.
  • Windows/ARM: completed, status of cgo is unknown
  • RegisterABI: most of the rest of the changes are in review, will be able to start debugging in earnest.
  • GC Pacer: Not complicated, but currently unscheduled. Register ABI is taking precedence, and if we get a chance, it'll get in.
  • Scheduler Improvements: Most CLs are ready for review, need to do more testing for metrics.
  • Improving Escape Analysis: Most CLs are ready for review
  • Haswell Improvements: Needs a proposal first.
  • Compiler phase changes: (interleaving front-end and SSA) mostly delayed.
  • We discussed using GOEXPERIMENT as a one true feature gating system. Register ABI work has found this to be helpful

Attendees:
Lost to the sands of time. I forgot to take a screenshot.

@golang golang deleted a comment from changkun Apr 12, 2021
@jeremyfaller
Copy link
Contributor Author

@jeremyfaller jeremyfaller commented Apr 12, 2021

2021-04-06

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

  • We discussed turning up the register abi. Our current plan is to turn the five flags that control it one-by-one, and watching the builders/trybots. The date for this turn up is TBD.
  • We discussed branching again (during the freeze) for register abi and generics. Our current plan is to use a single branch for both.
  • We discussed how debug information for generics was going to work. We talked about how other compilers (gcc, swift) do it. Some issues might be brought up in a future meeting with the delve folks.

Attendees:
David Chase
Austin Clements
Matthew Dempsky
Jeremy Faller
Robert Griesemer
Michael Knyszek
Patrik Nyblom
Michael Pratt
Keith Randall
Dmitri Shuralyov
Ian Lance Taylor
Cherry Zhang

@jeremyfaller
Copy link
Contributor Author

@jeremyfaller jeremyfaller commented Apr 21, 2021

2021-04-20

These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

  • We discussed regabi which has been completely enabled in master. Please let us know if there are any instabilities. Performance (while still changing) is showing ≥ 5% wins, and smaller binaries by 2%, with 10% smaller .TEXT sections.
  • We discussed the status of types2. It's not in yet, but we intend to get it in before release.

Attendees:
David Chase
Matthew Dempsky
Jeremy Faller
Robert Griesemer
Michael Knyszek
Than McIntosh
Martin Möhrmann
Patrik Nyblom
Michael Pratt
Keith Randall
Dan Scales
Ian Lance Taylor
Cherry Zhang

@jeremyfaller
Copy link
Contributor Author

@jeremyfaller jeremyfaller commented May 17, 2021

2021-05-11
These compiler and runtime meeting minutes are a snapshot of what was discussed in an internal Google meeting. Notes on agenda items that address Google specific needs are elided. Any comments on this issue will be removed, and discussion about topics raised should be taken to the appropriate mailing list. Additionally, any feedback or suggestions for additions to the notes should be handled there as well.

I'm a little behind updating these notes. I'll consolidate the last couple of meetings into this one.

  • The freeze is on. Generics and the register ABI work will continue on a the dev.typeparams branch.
  • The register ABI is holding true on the ~5% performance improvement, and smaller binaries. There are some regressions in some benchmarks, likely related to live variables over loops with register pressure, but we might not feel comfortable fixing these for 1.17.
  • On May 11, we discussed the Uber blog post. We think it's being adequately discussed on the proposal.
  • On May 4, we discussed Ian's slices proposal.
  • We continue to track the 1.17 release milestone. So far, it looks like we're on track for the 1.17 beta.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants