New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile/internal/types: lazy/incremental type expansion #25056

Open
mdempsky opened this Issue Apr 24, 2018 · 4 comments

Comments

Projects
None yet
3 participants
@mdempsky
Member

mdempsky commented Apr 24, 2018

The compiler now supports an indexed data format for package export data that enables efficient random-access of package data. We already use this to implement lazy expansion of declarations and inline bodies, but we still currently fully expand types whenever they're referenced. I expect in larger projects that a lot of deeply nested details are never needed and we could avoid reading that data into memory.

I think the steps are largely:

  1. Hide remaining Type fields behind setter/getter methods.
  2. Create a new TSTUB Type with a way to register an expander function that rewrites the Type into a real Go type.
  3. Add a Type.expand method that checks for Etype==TSTUB and calls the expander function.
  4. Insert calls to expand() at the top of every(?) public Type method.
  5. Update iimport.go to create TSTUB types instead of normal types.

Another useful (but non-critical) step would be to review the Type API and try to prune/simplify unnecessary methods. This would reduce how many places we need to instrument with expand calls.

@mdempsky mdempsky added the ToolSpeed label Apr 24, 2018

@mdempsky mdempsky added this to the Go1.11 milestone Apr 24, 2018

@mdempsky

This comment has been minimized.

Member

mdempsky commented Apr 24, 2018

Optimistically tagging for 1.11, but freeze is one week away, so this will probably slip.

@gopherbot

This comment has been minimized.

gopherbot commented Apr 24, 2018

Change https://golang.org/cl/109138 mentions this issue: cmd/compile/internal/types: replace Type.Val with Type.Elem

gopherbot pushed a commit that referenced this issue Apr 24, 2018

cmd/compile/internal/types: replace Type.Val with Type.Elem
This reduces the API surface of Type slightly (for #25056), but also
makes it more consistent with the reflect and go/types APIs.

Passes toolstash-check.

Change-Id: Ief9a8eb461ae6e88895f347e2a1b7b8a62423222
Reviewed-on: https://go-review.googlesource.com/109138
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

@mdempsky mdempsky modified the milestones: Go1.11, Go1.12 May 1, 2018

@mdempsky

This comment has been minimized.

Member

mdempsky commented May 1, 2018

Not happening for 1.11.

@griesemer

This comment has been minimized.

Contributor

griesemer commented Oct 30, 2018

Also not happening for 1.12.

@griesemer griesemer modified the milestones: Go1.12, Go1.13 Oct 30, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment