-
Notifications
You must be signed in to change notification settings - Fork 172
cue get go does not work consistently #621
Comments
After trying a few variations, I came to the conclusion that for reproducible behavior, |
I have a fix here: https://cue-review.googlesource.com/c/cue/+/8021 Not pretty or ideal... ended up embedding and temp writing the files to disk... Maybe some of the other Go compiler API could remove this requirement? Not sure |
Interesting. Thanks for identifying the issue! |
@verdverm - just so that I'm clear, can you explain from the repro above what you believe to be not working? What version of Go are you using? |
With go 1.13, following the instructions above, you get the correct behavior; that is,
Use go > 1.13, e.g., 1.15.6, then we get instead:
This happens with cue 0.3.0-alpha6, 0.3.0-beta1 and 0.3.0-beta2 |
@NicolasRouquette - and just to confirm, the Edit: critical edit |
We probably need to state clearly somewhere what versions of Go are supported. Is there a reason you are using Go 1.13? (asking in the context of https://golang.org/doc/devel/release.html#policy) |
Ok, here's my now complete analysis. Example 1For completeness, here is a
Note this test does not include a Analysis based on the above:
Example 2Now consider:
i.e. we add a
Conclusion
Lines 99 to 105 in f0c025c
|
Noting an offline conversation with @mpvl. Next steps:
|
One further aspect we will need to consider, that isn't purely a function of the changes in Go 1.16 but is exacerbated by it. Consider the following:
With Go That is to say, the side effects of Go There is a precedent however for keeping track of such dependencies: tools. The current best practice for keeping track of tool dependencies is as follows (per golang/go#25922 (comment)): create a build-ignored
This advice works for all Go versions, and should probably have been the advice from the beginning (otherwise you have an unstable This is a point to consider for the docs/Go 1.16 docs for the |
@myitcv Thanks for improving our earlier analysis; LGTM. I also want to note that during my analysis, I found a potentially confounding problem.
|
Thanks, @NicolasRouquette - yes, that point is covered in #621 (comment):
With the loading of the embedded With the loading of the arguments to |
When 'cue get go' is run from a directory with a go vendor directory, and Cue is not within the dependencies and vendor directory, then Cue fails to load its interface types, ignoring a nested error from the golang.org/x/tools/go/packages.Load function. This changes checks for and returns these nested errors. It also embeds the interface files content so that this part of the code works regardless of modules and vendor dirs. Fixes #621 Change-Id: I630e2cb167c05d63186aa356eaf70a58749b61c5
When 'cue get go' is run from a directory with a go vendor directory, and Cue is not within the dependencies and vendor directory, then Cue fails to load its interface types, ignoring a nested error from the golang.org/x/tools/go/packages.Load function. This changes checks for and returns these nested errors. It also embeds the interface files content so that this part of the code works regardless of modules and vendor dirs. Fixes #621 Change-Id: I630e2cb167c05d63186aa356eaf70a58749b61c5
FYI: This bug is still present in cue-0.3.0-beta3. |
That's expected - the CL that fixes this is in progress. This issue will close automatically when the CL is merged. |
'cue get go' loads interface types from cmd/cue/cmd/interfaces during initialisation. Go types that satisfy these interface types are then used as special cases for the target CUE definition types - see the complete logic in (*extractor).altType. Given the nature of 'cue get go', it is assumed that the command is run in the context of a Go module. That is to say, 'go env GOMOD' will be non-empty. (It doesn't seem worth adding GOPATH support given that modules will be GA in Go 1.16) Currently, 'cue get go' assumes that the package cuelang.org/go/cmd/cue/cmd/interfaces will be available. That is to say, either cuelang.org/go is the main module or, more likely, that cuelang.org/go is a module dependency of the main module. However, this is an unnecessary restriction and one that is potentially incorrect. After all, the user is using a specific version of cmd/cue - loading a different version of cuelang.org/go seems likely to lead to version-skew related problems. This change switches cmd/cue to instead load cuelang.org/go/cmd/cue/cmd/interfaces via a simple embedding of the GoFiles in that package. As part of the same change we surface all loading and type checking errors for either the embedded cuelang.org/go/cmd/cue/cmd/interfaces or the arguments to 'cue get go'. With thanks to @verdverm and @NicolasRouquette for the analysis and suggestions that went into this fix. Drops GOPATH support. Fixes #621 Fixes #668 Change-Id: I630e2cb167c05d63186aa356eaf70a58749b61c5
'cue get go' loads interface types from cmd/cue/cmd/interfaces during initialisation. Go types that satisfy these interface types are then used as special cases for the target CUE definition types - see the complete logic in (*extractor).altType. Given the nature of 'cue get go', it is assumed that the command is run in the context of a Go module. That is to say, 'go env GOMOD' will be non-empty. (It doesn't seem worth adding GOPATH support given that modules will be GA in Go 1.16) Currently, 'cue get go' assumes that the package cuelang.org/go/cmd/cue/cmd/interfaces will be available. That is to say, either cuelang.org/go is the main module or, more likely, that cuelang.org/go is a module dependency of the main module. However, this is an unnecessary restriction and one that is potentially incorrect. After all, the user is using a specific version of cmd/cue - loading a different version of cuelang.org/go seems likely to lead to version-skew related problems. This change switches cmd/cue to instead load cuelang.org/go/cmd/cue/cmd/interfaces via a simple embedding of the GoFiles in that package. As part of the same change we surface all loading and type checking errors for either the embedded cuelang.org/go/cmd/cue/cmd/interfaces or the arguments to 'cue get go'. With thanks to @verdverm and @NicolasRouquette for the analysis and suggestions that went into this fix. Drops GOPATH support. Fixes #621 Fixes #668 Change-Id: I630e2cb167c05d63186aa356eaf70a58749b61c5
Currently, 'cue get go' loads interface types from cmd/cue/cmd/interfaces during initialisation. Go types that satisfy these interface types are then used as special cases for the target CUE definition types - see the complete logic in (*extractor).altType. Given the nature of 'cue get go' (i.e. that we are looking to import CUE definitions from Go code), it is assumed that the command will be run in the context of a main Go module. That is to say, 'go env GOMOD' will be non-empty. The package arguments to 'cue get go' (and their dependencies) will be resolved via the main Go module. Currently, 'cue get go' assumes that the package cuelang.org/go/cmd/cue/cmd/interfaces will be available. That is to say, either cuelang.org/go is the main module or, more likely, that cuelang.org/go is a module dependency of the main module. However, this is an unnecessary restriction and one that is potentially incorrect. After all, the user is using a specific version of cmd/cue - loading a different version of cuelang.org/go seems likely to lead to version-skew related problems. This change switches cmd/cue to instead load cuelang.org/go/cmd/cue/cmd/interfaces via a simple embedding of the GoFiles in that package. As part of the same change we surface all loading and type checking errors for either the embedded cuelang.org/go/cmd/cue/cmd/interfaces or the arguments to 'cue get go'. We also migrate the TestGetGo to be a testscript test: cmd/cue/cmd/testdata/script/get_go_types.txt This test encompasses all the checks for the various rules/edge cases of type mappins in 'cue get go', e.g. the mapping of certain Go types to _ or string. We also introduce more basic tests that verify the behaviour of 'cue get go' when given a main module package, as well as a non-main module package (a dependency that should already be resolvable via the main Go module). Note: it doesn't seem worth adding/testing GOPATH support given that modules will be GA in Go 1.16. Indeed we have not had explicit GOPATH tests for some time/ever. This change also signifies an end to our support for GOPATH - if it works it will be by accident. In a later change we will enforce the entirely reasonable requirement that the main Go module (which we are using to resolve the arguments to 'cue get go') and the main CUE module into which we are writing the result of 'cue get go' (either within the main module or the cue.mod/pkg directory) are aligned in terms of their module paths. This will allow us to drop the --local flag as part of our renaming 'cue get go' to 'cue import go'. With thanks to @verdverm and @NicolasRouquette for the analysis and suggestions that went into this fix. Fixes #621 Change-Id: I630e2cb167c05d63186aa356eaf70a58749b61c5
Currently, 'cue get go' loads interface types from cmd/cue/cmd/interfaces during initialisation. Go types that satisfy these interface types are then used as special cases for the target CUE definition types - see the complete logic in (*extractor).altType. Given the nature of 'cue get go' (i.e. that we are looking to import CUE definitions from Go code), it is assumed that the command will be run in the context of a main Go module. That is to say, 'go env GOMOD' will be non-empty. The package arguments to 'cue get go' (and their dependencies) will be resolved via the main Go module. Currently, 'cue get go' assumes that the package cuelang.org/go/cmd/cue/cmd/interfaces will be available. That is to say, either cuelang.org/go is the main module or, more likely, that cuelang.org/go is a module dependency of the main module. However, this is an unnecessary restriction and one that is potentially incorrect. After all, the user is using a specific version of cmd/cue - loading a different version of cuelang.org/go seems likely to lead to version-skew related problems. This change switches cmd/cue to instead load cuelang.org/go/cmd/cue/cmd/interfaces via a simple embedding of the GoFiles in that package. As part of the same change we surface all loading and type checking errors for either the embedded cuelang.org/go/cmd/cue/cmd/interfaces or the arguments to 'cue get go'. We also migrate the TestGetGo to be a testscript test: cmd/cue/cmd/testdata/script/get_go_types.txt This test encompasses all the checks for the various rules/edge cases of type mappins in 'cue get go', e.g. the mapping of certain Go types to _ or string. We also introduce more basic tests that verify the behaviour of 'cue get go' when given a main module package, as well as a non-main module package (a dependency that should already be resolvable via the main Go module). Note: it doesn't seem worth adding/testing GOPATH support given that modules will be GA in Go 1.16. Indeed we have not had explicit GOPATH tests for some time/ever. This change also signifies an end to our support for GOPATH - if it works it will be by accident. In a later change we will enforce the entirely reasonable requirement that the main Go module (which we are using to resolve the arguments to 'cue get go') and the main CUE module into which we are writing the result of 'cue get go' (either within the main module or the cue.mod/pkg directory) are aligned in terms of their module paths. This will allow us to drop the --local flag as part of our renaming 'cue get go' to 'cue import go'. We leave the enforcing of setting GOFLAGS=-mod=readonly whilst loading the arguments to 'cue get go' to a later change. This is preparation towards 'cue import go' but will require a bit more work. With thanks to @verdverm and @NicolasRouquette for the analysis and suggestions that went into this fix. Fixes #621 Change-Id: I630e2cb167c05d63186aa356eaf70a58749b61c5
Currently, 'cue get go' loads interface types from cmd/cue/cmd/interfaces during initialisation. Go types that satisfy these interface types are then used as special cases for the target CUE definition types - see the complete logic in (*extractor).altType. Given the nature of 'cue get go' (i.e. that we are looking to import CUE definitions from Go code), it is assumed that the command will be run in the context of a main Go module. That is to say, 'go env GOMOD' will be non-empty. The package arguments to 'cue get go' (and their dependencies) will be resolved via the main Go module. Currently, 'cue get go' assumes that the package cuelang.org/go/cmd/cue/cmd/interfaces will be available. That is to say, either cuelang.org/go is the main module or, more likely, that cuelang.org/go is a module dependency of the main module. However, this is an unnecessary restriction and one that is potentially incorrect. After all, the user is using a specific version of cmd/cue - loading a different version of cuelang.org/go seems likely to lead to version-skew related problems. This change switches cmd/cue to instead load cuelang.org/go/cmd/cue/cmd/interfaces via a simple embedding of the GoFiles in that package. As part of the same change we surface all loading and type checking errors for either the embedded cuelang.org/go/cmd/cue/cmd/interfaces or the arguments to 'cue get go'. We also migrate the TestGetGo to be a testscript test: cmd/cue/cmd/testdata/script/get_go_types.txt This test encompasses all the checks for the various rules/edge cases of type mappins in 'cue get go', e.g. the mapping of certain Go types to _ or string. We also introduce more basic tests that verify the behaviour of 'cue get go' when given a main module package, as well as a non-main module package (a dependency that should already be resolvable via the main Go module). Note: it doesn't seem worth adding/testing GOPATH support given that modules will be GA in Go 1.16. Indeed we have not had explicit GOPATH tests for some time/ever. This change also signifies an end to our support for GOPATH - if it works it will be by accident. In a later change we will enforce the entirely reasonable requirement that the main Go module (which we are using to resolve the arguments to 'cue get go') and the main CUE module into which we are writing the result of 'cue get go' (either within the main module or the cue.mod/pkg directory) are aligned in terms of their module paths. This will allow us to drop the --local flag as part of our renaming 'cue get go' to 'cue import go'. We leave the enforcing of setting GOFLAGS=-mod=readonly whilst loading the arguments to 'cue get go' to a later change. This is preparation towards 'cue import go' but will require a bit more work. With thanks to @verdverm and @NicolasRouquette for the analysis and suggestions that went into this fix. Fixes #621 Change-Id: I630e2cb167c05d63186aa356eaf70a58749b61c5
This issue has been migrated to cue-lang/cue#621. For more details about CUE's migration to a new home, please see cue-lang/cue#1078. |
ceu get go
does not load its own interfaces when a go vendor directory is present and CUE is not in the dependencies.The error is in
initInterfaces()
(https://github.com/cuelang/cue/blob/master/cmd/cue/cmd/get_go.go#L287)packages.Load does not return an error, but there are errors in
p[0].Errors
We should be checking for these errors where this function is used.
repro:
possible resolutions:
toTop
andtoText
interfaces(1) I'd generally prefer to have a solution that does not require the user to have Cue as a dependency. This seems like unpleasant DX, essentially telling them they need to have an artificial dependency via overly complex instructions.
(2) could be an easy solution. However, I'm leery of modifying the Config so that Cue code is fetched automatically. This might make some SecOps people unhappy / break in corp intranets.
(3) Config has an
Overlay
we could use to embed these files as strings. This seems like the best way to go.We will also want to catch the errors in the other locations and provide an error message there.
What do others think?
The text was updated successfully, but these errors were encountered: