-
Notifications
You must be signed in to change notification settings - Fork 17.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
x/tools/gopls: improve support for authoring cgo packages #35721
Comments
A suggestion (not sure how viable it is) would be to use clangd if it's available and proxy the C code to it. |
That's probably overkill, but does point to an approach I didn't mention: tightly integrate cgo processing into gopls so that we can mitigate the performance problems of running the actual cgo tool. As you imply, the expensive part of cgo is learning about the C code. If we could separate that part from the code generation part, we could cache the C information and run just code generation when .go files changed. That might be the ideal solution, but would require changes to the cgo tool's implementation. |
Definitely related. If the net result of that change was that GoFiles was [x.go, y.go] and CompiledGoFiles was [x.go,y.go,$GOCACHE/.../_cgo_gotypes.go] per #16623 (comment) then I think we would be in much better shape. |
@heschik To confirm, by "much better shape" do you mean you think there would still be further work to do to address this issue? (Seems like this would address #35720 too, actually.) Looking at CLs 33677 and 33678 again, I think they're probably in reasonable shape to polish up for the next release cycle. I think the main issues are:
|
Further work: Yes -- we'd still have to regenerate the generated file sometimes. But that's a much more manageable concern. #35720: I think we want a fix for that before 1.15 is released. Otherwise I'd say yeah. If you have a set of patches that work against master, even if the API is rough, I'm happy to work to get |
@heschik FYI, I just rebased CL 33677 and fixed merge conflicts. I verified that tests still pass, but haven't done anything more extensive than that. I don't see CompiledGoFiles when I run |
Nevermind. Read the docs and realized it needs to be invoked as |
I think However, for the short term, here's what I think you can do within
|
https://go-review.googlesource.com/c/tools/+/208264 is an ugly hack to do the above. It passes |
Updated CL 33677 for parity with CL 43970. With CL 33677 and CL 208264 in place, I can run:
and confirm that it's processing the original .go sources, and it's type-checking okay. Edit: GOPACKAGESDRIVER=off is necessary because I simply hacked the golist driver implementation. The proper solution here will probably require extending the go/packages driver interface. |
@heschik I think CLs 33677 and 208264 should be in good shape if you want to play with them in the context of gopls. |
I'm just about done with #35720, and then I'm excited to give the patches a try. Should be Monday. |
So, this is a bit of a good news/bad news situation. The good news is that @mdempsky's patches work beautifully, and features like autocomplete start working in cgo files with no changes to gopls whatsoever. As I said before, there's still some work to re-trigger cgo compilation when appropriate, and the The bad news is that these are changes to the compiler and standard library. As such, they won't be ready until Go 1.15, scheduled for about 8 months from now. Hopefully we can land them early in the cycle so that people can try them out. If there's demand, I can write up some instructions on how to set up an environment with local patches even now. |
It's only a standard library change, so it should be possible to backport the changes via vendoring if gopls users want to use it before 1.15. |
I don't think that's a good option in module mode, unfortunately. |
@heschik Fair enough. I'm still very much a modules noob, so I defer to your expertise on how best to address the issue. I mostly wanted to clarify that compiler changes aren't involved here. I'd be curious what happens for "jump to definition" on a C.foo symbol. I'm guessing it jumps to the _cgo_gotypes.go file? We could maybe get cgo to emit //line directives to make it point into the C preamble and/or header files if that would be more useful. |
Wow, that would be so useful. There are many people who use |
This comment has been minimized.
This comment has been minimized.
In Arch Linux You should write Thank You, it works like magic! |
Not sure why this is marked for 1.15. Moving to backlog. |
Because it depended on go/types changes, but those have landed, at least enough for us, for now. |
@heschik: Will we be able to close this when Go 1.15 is released, or will there still be further work to do here? |
#39072 is still a problem, but the fix is mostly or entirely in the stdlib, so I think it'd be reasonable to close this one if you want. |
Oh ok, well let's keep this open for tracking as long as there are still open issues. Should we milestone #39072 for 1.16? |
Done. |
0.4.3 Disable the fillstruct analysis by default. We recently uncovered some performance issues with the analysis, leading us to disable it by default. Once those issues are resolved, we will enable it by default again. You can still enable it by adding the following to your VS Code settings: "gopls": { "analyses": { "fillstruct": true, } } gopls/v0.4.2 Significant memory improvements (@heschik). Dependency test variants and vendored packages are no longer considered "workspace packages". Smart autocompletion for "append" (@muirdm). A "fill struct" code action to suggest populating a struct literal with default values (@luciolas, @joshbaum). Better cgo support with Go 1.15 (@heschik). Learn more: golang/go#35721 (comment). Code lens to run Go tests directly in the editor (@martskins). Currently opt-in: "gopls": { "codelens": { "test": true, } } Improved folding in composite literals (@joshbaum). Pop-up suggestion to run go mod vendor when inconsistent vendoring detected (@stamblerre). Respect GOPRIVATE for all document links and links on hover (@findleyr). A full list of issues resolved in this release can be found in the gopls/v0.4.2 milestone. gopls/dev.go2go: You can use the new go2go prototype with gopls. See golang/go#39619.
i'm seeing the "error" in the import "C" line and the "regenerate cgo definitions" in vscode but nothing happens when i click it, no errors, no nothing at all. Is there a way to force it? a console command or something? |
@Zincr0 It would be good to add a progress notification, but right now there's no feedback. The only thing that should happen is that any changes you made in the cgo-using code will be reflected, resulting in errors going away or appearing as appropriate. Are you seeing changes not take effect? Can you be specific about what you're seeing vs. what you expect to see? |
I'm importing a static library and getting an error in the 'import "C"' line. Clicking regenerate cgo definitions doesn't make the linter error go away, i'm guessing that maybe all of this is for for embedded C code only, and not for static libraries? |
The only errors regenerating cgo would fix are compiler errors related to the Whatever you're seeing, I'm not sure it's related to this issue. Please file a new issue, particularly including the exact error. |
Created #40595 |
No. If your code compiles at the command line, but you're getting an error from |
Forked from #31561. This issue is not for users of cgo packages; that issue is #35720.
Beta instructions: #35721 (comment) contains instructions on how to build a version of Go and gopls that should work well for authoring cgo packages.
Files that import "C" are not really valid Go. They depend heavily on extra information that is produced by the go tool when it builds the package. They can be parsed, but not type checked, though there is some rudimentary support for working around the issues by setting the
go/types.Config.FakeImportC
flag.Many people have suggested that gopls use the package's GoFiles rather than its CompiledGoFiles. The GoFiles are the original (invalid) files. Using those may give a slightly better experience for people editing the package, but it will completely break anything that depends on the type information, since the package will no longer type check. We would have to evaluate the effects very carefully before making that change in the current architecture.
We could consider type checking cgo packages twice, once with the real code and once with the user's code. That would be a significant architectural change.
Alternatively, we could run the cgo tool as the .go files change to produce the real code. This is a much more robust approach, but the cost of running cgo processing may be too high, especially on large packages.
No matter what, this is a large project that will need a lot of testing and thought. It's unlikely we'll make huge improvements here very soon.
The text was updated successfully, but these errors were encountered: