-
Notifications
You must be signed in to change notification settings - Fork 17.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
spec: disallow anonymous interface cycles #56103
Comments
I've written several Go tools and I agree with your conclusion: I've never seen this kind of cycle in Go code, and I'm pretty sure that at least one of my tools would fail catastrophically if presented with one. |
Note that this doesn't speak to their usefulness, only to how much code would break if this change were made. People might have used cyclic interfaces more often if they didn't break tools. The existence of bug reports suggests that people have at least tried to use them. As a tool author I'm in favour of this change, but I am conveniently ignoring that this is a backwards incompatible change. But ISTM that recently we've taken to allowing those if they don't actually affect any existing code… |
Is this only limited to result types? How about |
No, they're not.
No. |
Could you elaborate more on why they are not equivalent? |
OK, I got it. One is named, the other is not. |
Does the proposed expand rule requires all named interfaces to be expanded to unnamed ones? |
All interface type literals must expand to canonical, embedding-free representations. |
Any indication how unhappy the bug reporters were about this? Was it a curiosity, or a real impediment? |
For Staticcheck, only one issue has been filed because of this problem (dominikh/go-tools#310). I believe the snippet was extracted from real code, but the issue has gone unfixed since 2018 and the user hasn't complained yet, so I can only assume that they worked around it. |
Maybe, but I think there's a few points:
|
That's my feeling too. I wish this feature would go away, but generally it's the compiler-writer's job to absorb the complexity to make the (idealized) spec simpler, and if the spec doesn't have to disallow this special case, that's one less thing the user needs to understand. (Let's ignore for now that the spec doesn't say quite what it should about recursive types.) Matthew has shown empirically that a truly tiny number of Go programs use this feature, but even if all of them are excessively complex and could be easily rewritten, that figure is still not zero. . . . We tool writers had been operating under the assumption that cycles always arose via named types, and thus it would sufficient to break cycles at named types, such as I in the example:
But what we discovered is that in traversing the method m we end up back at m without ever seeing the name I. That's because types.Interface.Methods iterates over the method set of Perhaps there are other ways to make clients more aware that Interface.Methods is qualitatively different, and to help provide them with the information they need to make their algorithms robust against cycles. For example, last week my coworkers and I spent between us probably a couple of hours on a cyclic-interface problem in the golang.org/x/tools/go/types/typeutil.hasher type. In that instance, if the type checker API had exposed the information that interface type was self-referential it would have made the solution trivial. |
This proposal has been added to the active column of the proposals project |
Will this compile under the new rules? type B interface { I }
type I interface { m() interface { B } } What about this? type B = interface{ I }
type I interface{ m() interface{ B } } And this? type B = interface{ I }
type I interface{ m() B } |
@DmitriyMV All 3 of those examples are cyclic. These would be disallowed by the new rules. |
@DmitriyMV I proposed "it should be possible to finitely expand all embedded interfaces."
has two embedded interfaces,
Expanding the first embedded interface again (now the first
This process will clearly never terminate in a finite number of steps. A similar process unfolds given your other two examples. So no, none of those would compile. -- Note: This rule naturally subsumes the spec's "An interface type T may not embed any type element that is, contains, or embeds T, recursively." For example, the spec demonstrates with an example of:
Trying to expand The same reasoning applies to |
I have three things. 1: This is the best simple, but non-trivial, example of something that uses this paradigm I could come up with type Nexter interface {
Next(Input) (interface { Nexter }, error)
Done() Output
} If this is a valid example, I can see this as a rather useful chaining pattern. 2: I think that this would be a backwards incompatible change under the strict definitions of the compatibility promise. I am however, personally in favor of the "loosening" of the compatibility promise that has happened recently. I feel it has made the language better overall. Therefore I think it would be ok to disallow it in regards to the promise. 3: While I think it would be ok to be be disallowed, I don't think it should be disallowed. I dont feel that the reasoning of "They causes a lot of trouble for tools" is very strong. I think the tooling should strive to match the language, not the other way around. I also echo @dominikh in that I suspect if the tooling was better around them, they might be a more common pattern. |
@deefdragon But why would you write the interface that way instead of: type Nexter interface {
Next(Input) (Nexter, error)
Done() Output
} Did you happen to try writing a type that actually implements your Nexter interface? (I assume not, because of the syntax errors.)
By definition, if the pattern is "useful" then it must have some "uses." But as I noted in the initial comment, that does not seem to be the case in practice.
Yes, it's easy to say tools should be better and do more, but as tools authors we have finite engineering resources to spread around different user demands. In practice, no one actually uses this feature, yet the complexity it imposes hinders us from implementing compiler optimizations that users would actually benefit from. |
Apologies on the syntax. That has been corrected. Regarding Regarding its uses, I can see uses for it in the back of my head, however, I cant come up with anything concrete or specific. Regarding your third point, I obviously misinterpreted the "Broken" section of your proposal. The way I read it did not get through to me the lengths to which it could improve things for an average go user in the background. Regarding my tooling comments. The point behind it was not to say that tooling has to be perfect. I fully recognize how difficult that would be, and how little time people have, (a large portion of which they are already donating to the development of these tools (Thank you for that btw)). My intended point was that tooling not being a perfect match to the spec is, to me, not a very good reason to change the language on its own. I am more on the fence about this staying than I was before. I still lean towards keeping it, and still think this could be a useful pattern if explored, but I don't deal with chaining function calls enough to make a concrete argument or have a concrete example. |
What about the case of "node" / "graph" / "list" / "chaining" interfaces ?
I did not provided the necessary methods of I never wrote code like this, and never used code like this, but I would be very surprised that no-one is doing it, and I would find it sensical. That would also be disallowed ? Even
|
@folays I proposed "it should be possible to finitely expand all embedded interfaces." None of your examples involve embedded interfaces, so they're already expanded. They would continue to be allowed. |
One [maybe?] advantage of allowing cyclic interfaces to be anonymous is that they do not require a definition of the interface to be imported in order to be implemented. This is a hyped property of Go's interfaces in other contexts. A signature that takes or returns a named type does need to be imported even if the named type is an interface. The simplest example I can think of to illustrate this point is weird: https://go.dev/play/p/_1JCj2upF-R
I do not think anybody wants to write or read code like this [outside of intellectual exercises]. If the above is banned, folks would still have the option of creating a package just to export the interface definition. That does actually sound better than reading and maintaining the above example even if it requires importing the interface to implement it. |
This boils down to whether we want to guarantee that every expression of the form 'interface { ... }' can have its embedded interfaces "compiled out" so that the interface expression can be written without any embedded interfaces. So this is OK: type B interface { x() I } Because it compiles out to: type B interface { x() I } But this is not: type B interface { I } because B compiles out to: type B interface { m() interface { B } } Making this guarantee would simplify tools and analysis and the compiler, and it might even simplify teaching Go to people if you can say that interfaces can always be expanded out this way to remove embedding. It's true that we've allowed this in some forms already, but @mdempsky says that complex cases are still buggy today. It also appears to be true that almost no one uses these forms, so if we did make the restriction, it seems that hardly any code would be affected. As @mdempsky reported, it looks like https://github.com/gozelus/zelus_rest/blob/master/core/db/db.go#L20 is the only problematic use in the entire public ecosystem, and the package appears to be unused, even in the rest of that module. If we do move forward with the restriction, we should have some way to disable the restriction for a release to let people adapt. Or maybe we should do one release where vet complains, followed by removing it entirely in the next release? Does anyone object to adopting this change and this rollout plan? |
Change https://go.dev/cl/445598 mentions this issue: |
SGTM. go.dev/cl/445598 changes cmd/compile to reject anonymous interface cycles, and adds a flag (spelled |
Given how very few uses we think there are (approximately zero), a compiler flag to opt-out sounds fine. If we thought a significant number of users were affected, we'd want to do a more gradual rollout with a vet check first. But it seems safe to skip that for now. So the plan sounds like: For Go 1.20, the compiler will reject these interface cycles by default, but you can build with 'go build -gcflags=all=-d=interfacecycles' to keep old code building. If people report significant amounts of breakage to us during the release candidates, we would back out this change. For Go 1.22, the -d=interfacecycles flag will be removed, and old code will simply not build anymore. Using Go 1.22 gives people a year to update their code and helps with people who may skip a version. Normally we would not ever want to break old code, but all the evidence we have suggests that this will break no real code at all. If people do report breakages to us, we can push the flag removal out to a later release, like Go 1.24. Do I have that right? |
SGTM. I'm happy to wait until 1.22 before removing the opt-out to give more time to smoke out potentially affected users. |
Based on the discussion above, this proposal seems like a likely accept. |
No change in consensus, so accepted. 🎉 |
This CL changes cmd/compile to reject anonymous interface cycles like: type I interface { m() interface { I } } We don't anticipate any users to be affected by this change in practice. Nonetheless, this CL also adds a `-d=interfacecycles` compiler flag to suppress the error. And assuming no issue reports from users, we'll move the check into go/types and types2 instead. Updates #56103. Change-Id: I1f1dce2d7aa19fb388312cc020e99cc354afddcb Reviewed-on: https://go-review.googlesource.com/c/go/+/445598 Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Robert Griesemer <gri@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Matthew Dempsky <mdempsky@google.com>
This CL changes cmd/compile to reject anonymous interface cycles like: type I interface { m() interface { I } } We don't anticipate any users to be affected by this change in practice. Nonetheless, this CL also adds a `-d=interfacecycles` compiler flag to suppress the error. And assuming no issue reports from users, we'll move the check into go/types and types2 instead. Updates golang#56103. Change-Id: I1f1dce2d7aa19fb388312cc020e99cc354afddcb Reviewed-on: https://go-review.googlesource.com/c/go/+/445598 Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Robert Griesemer <gri@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Change https://go.dev/cl/464196 mentions this issue: |
Elaborate on what this means, to help avoid users unnecessarily worrying about whether their code will be affected. Updates golang/go#56103. Change-Id: Iab6eeb836e30905e507681fcc9ada94e9a3052dd Reviewed-on: https://go-review.googlesource.com/c/website/+/464196 Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Robert Griesemer <gri@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Go 1.20 and 1.21 have both been released with support for anonymous interface cycles disallowed by default. I see no evidence that it's affected any users:
If there are no objections, next week I'll remove the flag (i.e., disallow users from re-enabling anonymous interface cycles) and start removing the compiler backend code that endeavors to support them. I anticipate go/types can be simplified by dropping support for anonymous interface cycles too, but for now I'll leave the check in cmd/compile. |
Change https://go.dev/cl/550896 mentions this issue: |
Per the discussion on the issue, since no problems related to this appeared since Go 1.20, remove the ability to disable the check for anonymous interface cycles permanently. Adjust various tests accordingly. For #56103. Change-Id: Ica2b28752dca08934bbbc163a9b062ae1eb2a834 Reviewed-on: https://go-review.googlesource.com/c/go/+/550896 Run-TryBot: Robert Griesemer <gri@google.com> Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Change https://go.dev/cl/551417 mentions this issue: |
The spec change will not be in time anymore for 1.22 - that's ok. The actual restriction has been in the compiler since 1.21. Moving to 1.23. |
Per the discussion on the issue, since no problems related to this appeared since Go 1.20, remove the ability to disable the check for anonymous interface cycles permanently. Adjust various tests accordingly. For golang#56103. Change-Id: Ica2b28752dca08934bbbc163a9b062ae1eb2a834 Reviewed-on: https://go-review.googlesource.com/c/go/+/550896 Run-TryBot: Robert Griesemer <gri@google.com> Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TL;DR: We should disallow declarations like
type I interface { m() interface { I } }
. They cause a lot of trouble for tools, and no one uses them in practice.Background
The Go spec allows interface types to embed other declared interfaces. For example, package io declares ReadCloser using the type literal
interface { Reader; Closer }
. This type literal is identical tointerface { Read([]byte) (int, error); Close() error }
, but also tointerface { Reader; Close() error }
andinterface { Read([]byte) (int, error); Closer }
.Type identity is a core concept of the Go language, and it's useful for tools (e.g., type checkers, compilers, static analysis passes) to have a canonical representation that types can be referred to with (e.g., to ensure runtime type descriptors representing identical types in separate compilation units are deduplicated by the linker). For interface types, the natural canonical representation is the fully expanded form without any embedded interfaces.
However, this causes problems for some self-referential interface types. The simplest example is:
Tools would like to expand the
interface { I }
to a canonical, embedding-free representation. However, expanding it producesinterface { m() interface { I } }
, which again containsinterface { I }
and requires expansion. Handled naively, this process never terminates.The (accepted) type aliases proposal stated: "In contrast, aliases must be possible to “expand out”, and there is no way to expand out an alias like
type T = *T
."The issue there was the same: given
type T = U
, we want to replace (expand) all occurrences ofT
withU
to find a canonical type description. But ifU
is*T
, then this process never terminates either.Proposal: I propose the same principle should apply to interface embedding: it should be possible to finitely expand all embedded interfaces, and we should disallow declarations like
type I interface { m() interface { I } }
.Unused
Anonymous, cyclic interfaces appear unused in practice. I ran an analysis of every unique module path indexed by index.golang.org, and I found only 2 occurrences of cyclic interfaces:
However, neither of these packages appear to be imported anywhere, even within their own modules.
The first use case could be addressed by introducing a second named interface type like
type MySQLTx interface { MySQLDb; Rollback(); Commit() }
. (This would also allow https://github.com/gozelus/zelus_rest/blob/master/core/db/db.go#L48 to be changed tofunc (d *dbImp) Begin() MySQLTx { ... }
, instead of requiring the interface type literal to be repeated.)The second appears like it's meant to be a testdata file instead.
Broken
In the past, anonymous, cyclic interfaces have been a recurring issue that we've struggled to support: #10222, #16369, #25262, #29312.
Notably, packages whose package export data contained an anonymous interface cycle couldn't be imported prior to Go 1.7, because the old textual export data format couldn't handle them. (See #16369 (comment).)
x/tools/go/ssa.Hasher has been known to mishandle them since 2018 (#26863), yet no end users have clamored for it to be fixed. This feature underpins many x/tools features like callgraph construction, points-to analysis, SSA building, gopls completion suggestions, and staticcheck. Moreover, CL 439117 was started to fix this issue, but has stalled as yet more failure cases are identified. The options at the moment are either: (1) continue to ignore the issue, (2) simplify interface hashing (at the risk of introducing collisions in realistic Go code), or (3) implement a considerably more complex algorithm.
Finally, while waiting for my module analysis code to execute, I manually discovered many more implementation issues with anonymous, cyclic interfaces: #56045, #56046, #56055, #56056, #56057, #56059, #56061, #56062, #56063, #56065.
Why not tie to go.mod
go
version?In #3939, it's been proposed to remove
int
->string
conversions, because the semantics are surprising to new users. One recurring suggestion has been to tie this to thego
line in go.mod files: old modules would continue to compile successfully, whereas new modules would get an error instead to protect users from misuse.Could we do the same for anonymous interface cycles?
I argue no: the issue with anonymous interface cycles isn't that users accidentally use them (like
int
->string
conversions), but that tools authors have to deal with them at all. As long as anonymous interface cycles are allowed anywhere, all tools authors have to deal with them (e.g., CL 439117 above). Users are better served by letting tools authors focus on features that users actually care about and use.The text was updated successfully, but these errors were encountered: