New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: make available as a library #15108

Open
ghost opened this Issue Apr 4, 2016 · 10 comments

Comments

Projects
None yet
8 participants
@ghost

ghost commented Apr 4, 2016

It would be nice if there was an interface to the gc. I had extreme difficulty reading through the source code (maybe due to the fact it was transpiled from C), but I've come to the conclusion that it is pretty unrealistic to use the gc for anything, barring use of the go tool. How difficult would it be to give it an API and/or add it to the stdlib?

@minux

This comment has been minimized.

Member

minux commented Apr 4, 2016

@ghost

This comment has been minimized.

ghost commented Apr 4, 2016

Well my specific usecase is for a personal project in which I have a set of *ast.Packages that I transform given some directives. Currently codegen is the way to go but I would like to be able to feed my ast to gc directly if possible.

@bradfitz

This comment has been minimized.

Member

bradfitz commented Apr 4, 2016

This is too big for the standard library. See https://golang.org/doc/faq#x_in_std

We're still trying to clean up the compiler after the transition from C. Turning it into a library is quite a long way away yet.

@bradfitz bradfitz added the LongTerm label Apr 4, 2016

@bradfitz bradfitz changed the title from make gc more accessible to cmd/compile: make available as a library Apr 4, 2016

@mdempsky mdempsky added this to the Unplanned milestone Apr 4, 2016

@sbinet

This comment has been minimized.

Member

sbinet commented Aug 23, 2016

what about just giving a very high level API so one can embed the whole go gc compiler into someone's pure-go binary?

we wouldn't commit to any gc API, just this very high-level API.
something along the lines of:

package toolchain // golang.org/x/exp/go/toolchain

import "go/build"

type Toolchain interface {
   Compile(pkg *build.Package) (ofile string, err error)
   Name() string // name of this toolchain (e.g. "gc", "gccgo")
}

// open a registered toolchain (a-la database/sql)
func Open(name string) (Toolchain , error)
@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Aug 23, 2016

Serious question: what is the real advantage of a high level API over simply exec.Command("go", "build", "x.go")?

@sbinet

This comment has been minimized.

Member

sbinet commented Aug 23, 2016

it's all in the same binary, so you can scp my-binary fresh-box:/usr/bin/. w/o any Go installation.
well, at least that's what I'd envision.

@minux

This comment has been minimized.

Member

minux commented Aug 23, 2016

@griesemer

This comment has been minimized.

Contributor

griesemer commented Aug 23, 2016

"Compiler as a library" is intriguing and I have brought this idea up internally very early on in the Go project. That said, we're far off from such a reality as @bradfitz pointed out already, and there's a lot of details (many more then meets the eye at this point) that would have to be solved.

Let's consider this as what it is, a long-term vision. No need to expand this into a long-running thread that doesn't go anywhere.

@johanbrandhorst

This comment has been minimized.

Member

johanbrandhorst commented Jul 16, 2018

I think this should be reconsidered in light of the new WASM architecture being introduced in 1.11. This kind of work would allow things like the Go playground to be built using WASM, and other cool stuff detailed in #26384 (package-level splitting of payloads, like https://jsgo.io).

@dave

This comment has been minimized.

Contributor

dave commented Jul 19, 2018

It looks like this isn't going to happen any time soon, so for the last few days I've been working on a project to automatically modify the compiler codebase to turn it into a library.

The idea is that it'll wrap up all package level variables in a state object, and replace all filesystem os calls with a virtualisation layer. All these changes to the codebase will be performed by the tool, so as the compiler codebase is changed we can re-run the tool to update the output.

This is a huge project, but I've made some progress... and I haven't come across any terminal blockers yet. It should be noted that the output code won't be of a quality required to merge back into the Go compiler codebase, so further discussion in this issue probably isn't relevant.

I've summarized my progress so far: dave/forky#1 - best to post over there if you want to chat about it. I would love some help with the open issues!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment