-
Notifications
You must be signed in to change notification settings - Fork 17.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
spec: surprising rules for inheriting methods when declaring named types #41685
Comments
I agree that the behavior is kind of surprising, but I don't think it can be changed. The spec says in the section about struct types
And One phrasing that would be unambiguous for the section you quoted could be
Because the underlying type is always either a type-literal or a predeclared type, this covers both the interface-case and the embedding-case and says that if neither is the case, there are no methods. This probably still wouldn't be much clearer, though. So for clarity, a "in particular, this means that methods of embedded fields and methods of interface types are inherited" could be added. And in any case, an appropriate example with embedding should probably be added to the block following your quote. |
I get that changing the spec and breaking programs is very unlikely, but I wanted to bring it up because I find the original JSON example to be a behavior bug from the point of view of the user. I'm all for making the spec clearer or adding examples. It would have made my life easier trying to figure this out, at least. Having said that, I still find it concerning that overriding promoted methods in named types isn't final; they can be sidestepped pretty easily as I showed earlier. Which makes me worry about the scenario where the embedded type is unexported in an API, for example. |
One more thought before I head off: assuming that this is working as intended, beyond spec clarification, I would argue that a static analysis tool could also be nice. For example, any exposed type which embeds another type and overrides a method should trigger a warning, because any API user could sidestep that override by defining a new type. And I would imagine that the vast majority of API writers aren't realising that they are exposing two method sets that way. |
The language is working as I expect. I agree with the comments above that this would be hard to change. I also don't think that it would be helpful to have a warning about overriding a promoted method of an embedded type. That is a standard technique, in which the outer type wants to promote some methods and override some other methods. Clarifying the spec would of course be fine with me. I think the lesson to take away here is: embedded types should be used with care. Promoted methods can bite you in various ways. I still like my example from a few years ago at https://talks.golang.org/2014/compiling.slide#17 (the "run" button on that slide should work). |
My point is that the overriding appears to be entirely optional, as any consumer can work around it by defining a new type, even if they are in a third party package or don't have direct access to the embedded type.
I generally agree, but I don't think many Go developers will be aware to what extent it's a footgun. I hadn't realised that one could sidestep method overrides until today, for example, so I'm suddenly worried that multiple of the APIs I've written in the past with method overrides could have that issue. |
Understood. For cases where defensive programming is a concern, the fix is type privateEmbeddor struct { embedee }
func (privateEmbeddor) OverridingMethod() {}
type Embeddor struct { privateEmbeddor } |
In hindsight, I should have raised this issue as more of an experience report about how overriding methods is much easier to mess up than most Go developers think. I'm still not sure what the best solution to that today could be, beyond trying to clarify the spec and warn users that struct embedding should be used with more care (and showing a few pitfall examples). Maybe a blog post on golang.org, similar to how before there was a post covering the internals and common pitfalls around slices. |
A blog post sounds like a good idea. |
Who should I talk to about that? |
FWIW here's an example where this behaviour would genuinely have tripped me up. I've used this In the following example, I'd expect to see a JSON object (because the https://play.golang.org/p/j8HdbL3kfSA For the record, here's one way of working around the problem: https://play.golang.org/p/TT1BXg1rljR |
I was bitten by promoted methods from embedding a while ago (https://golang.org/issue/18480), but the behavior of defined types that @mvdan describes is new to me, and I don't think the spec supports it. In @mvdan's
we have a defined type, The spec is quite clear that a defined type "does not inherit any methods bound to the given type" and illustrates with an example with the same structure as Dan's:
Here, The full sentence in the spec is:
The two exceptions are clearly illustrated by the examples following. The first is
and the second is the usual
Those are the only two cases where a defined type inherits methods from its "given" type. So I see no justification in the spec for |
How do you come to that conclusion? IMO it is a type that is a struct, so it is a struct type. I had this discussion with someone on Twitter as well, so note that the spec in other places uses that interpretation of the term "struct type". For example
where "the base type" is the receiver type or what the receiver type points to - so it's always a defined type. Arguing that
Well, given that we are having an argument about how to correctly interpret it, it obviously isn't that clear :) FWIW, I'd argue that with something like embedding, if there is an ambiguity in the spec and all known implementations agree with one interpretation of it, then that interpretation should be considered correct. We are more than 10 years into go1, it's hard to argue that the language has been different all this time. If the issue was that implementations differed in behavior, then trying to lawyer around the spec might be useful. But the implementations agree, so we really should have a conversation around what the spec should be, to accurately describe the behavior as implemented. [edit] And FWIW, I did try to make some suggestions on what the spec might say in my comment above :) |
I actually agree with @Merovius because of the following line in the spec:
That is to say, given:
inherit any of the methods of |
The phrase "the method set of an interface type or of elements of a composite type remains unchanged" needs to be clarified to refer to underlying types. The example for interfaces that follows illustrates that, but not the one for composite types. |
I wonder: would it make sense to have a type Embedding struct {
*big.Int // ok
}
func (*Embedding) UnmarshalJSON(b []byte) error { return nil } // ok
type NewNamed Embedding // vet: method (*NewNamed).UnmarshalJSON refers to embedded (*big.Int).UnmarshalJSON, not (*Embedding).UnmarshalJSON |
The func (n *NewNamed) UnmarshalJSON(b []byte) error { return n.Int.UnmarshalJSON(b) } or func (n *NewNamed) UnmarshalJSON(b []byte) error { return (*Embedding)(n).UnmarshalJSON(b) } |
I agree that a static analysis check for when one bypasses method overriding would be very useful; I brought it up earlier in this thread, but didn't seem to get much support :) I imagine it might not fit vet in terms of frequency, in which case we could hand the idea to staticcheck. |
(@dominikh, FYI) |
@mvdan you asked for a warning for "any exposed type which embeds another type and overrides a method," which as @ianlancetaylor pointed out is a reasonable thing to do. @bcmills's suggestion is different. |
Right, my suggestion was more agressive because it intended to find this problem at the time of releasing an API with the potential flaw, not when the flaw is exposed at a later time when the API is consumed via a |
I do assume that most people will run |
Friendly ping :) Aside from that, I'm happy to repurpose this issue around clarifying the spec a little, either with clearer wording or with an added example. I also think that the vet check that @bcmills proposes would be worthwhile, and I think we should file a separate issue about that. Bryan, I'm happy to help co-author a short proposal if you're up for it. |
CC @stevetraut |
Whether a type is defined is orthogonal to its kind. For example, Thus in |
I am not sure whether the frequency will warrant a vet check or not. On one hand, this seems like something that could remain silent and not be caught by testing. OTOH I am not sure folks will combine these features that often and also have a method shadowed. Data would be helpful here. |
First, a piece of code that's minimized from a real bug we hit at work. What do you think this code does? https://play.golang.org/p/-FkcWvJzLZS
json.Unmarshal
onEmbedding
uses the method we declared directly on that named type, which just does nothing, so we end up with a nil error. All as expected so far.It gets trickier with the equivalent unmarshaling with
type NewNamed Embedding
. We get a nil pointer dereference panic, but that's not really important; what matters is that, as one can see in the panic stack trace, we callmath/big.(*Int).UnmarshalJSON
.type Embedding
is overriding the promotedUnmarshalJSON
, so it's troublesome that one can sidestep that entirely by simply doingtype T Embedding
.The other part about this behavior that surprises me is that, if we have a non-interface named type
Bar
, there's no way to tell iftype Foo Bar
will result inFoo
having an empty method set or not. I always thought that the only way to define a type which inherited methods was via eithertype T SomeInterface
ortype T struct { Bar }
. The following playground link further shows this inconsistency: https://play.golang.org/p/5uV8obYY8aRThe spec says:
From that text, I understand that
of an interface type
means ourtype T SomeInterface
example above, andof a composite type
meanstype T struct { Bar }
.I'm not sure whether to call this a spec bug or a request that it be clarified, but at the very least I find the behavior very unexpected, to the point that it can cause real bugs like the one I first showed in the issue. So I lean towards thinking that this is something we should fix in the implementation, not just clarifying the spec, because it feels like an unnecessary footgun.
I should note that I've checked both
encoding/json
andreflect
for bugs, but haven't found any; they seem to both follow what the compiler does.cc @griesemer @ianlancetaylor
The text was updated successfully, but these errors were encountered: