New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: ability to create and import methods for non-local types #21401

Closed
urandom opened this Issue Aug 11, 2017 · 31 comments

Comments

Projects
None yet
10 participants
@urandom

urandom commented Aug 11, 2017

This proposal is to allow method definitions for types, which are not create in the current package.

The current go spec specifically forbids defining methods for a type, unless the type is defined in the same package as its methods. This proposal is for relaxing these rules in certain cases.

  1. If a method for a non-local type is defined, that method should be valid for the package it is defined in. Lets say that we have the following definition:
package foo

func (s string) Index(s, sep int) int {
    ....
}

We will then be able to use "my string".Index(...) from anywhere within package foo

  1. A new declaration is defined for importing such extension methods defined in third-party packages:
ExtendDecl = "extend" ( ExtendSpec | "(" { ExtendSpec ; } ")" ) .
ExtendSpec = TypeList ImportPath .

When importing an extension, a list of types must be provided, corresponding to the methods one wishes to import. Importing multiple definitions of a method for a given type is an error. Moreover, since this is a type of import, there is no need to provide an import declaration for the same package, unless one wishes to use something else that package provides, like extra types/functions.

Example:

package foo

extend (
    string "strings"
)

func Action(action string) {
   action = action.Replace("desc", "name", -1)
}

Similarly to import declarations, extension declarations are valid for the file that uses them.

The incentive behind implementing this proposal is two-fold. First, when compared with the usability of packages like 'strings', it allows for more fluent and readable APIs:

import "strings"

func Actions(actions []string) {
     strings.Replace(
        strings.Replace(
              strings.Join(actions, "\n"),
              "desc", "name", -1,
       ), "alpha", "beta", -1)
}

vs.

extend (
     []string, string "strings"
)

func Actions(actions []string) {
    actions.Join("\n").
          Replace("desc", "name", -1).
          Replace("alpha", "beta", -1)
}

Second, it allows existing to support new interfaces by defining the missing methods. This is currently doable by defining a new type using the current one as an underlying type, and then delegating all the existing methods to the underlying type, which is itself quite tedious.

@gopherbot gopherbot added this to the Proposal milestone Aug 11, 2017

@gopherbot gopherbot added the Proposal label Aug 11, 2017

@randall77

This comment has been minimized.

Show comment
Hide comment
@randall77

randall77 Aug 11, 2017

Contributor

This allows two different packages to add different implementations of the same function to a type. For instance:

package a
type T int
func X(t T) {
   type I interface { F() }
    t.(I).F()
}

package b
func (t T) F() { println("B") }
a.X(T(0))

package c
func (t T) F() { println("C") }
a.X(T(0))

How does a.X know which F to call?

Contributor

randall77 commented Aug 11, 2017

This allows two different packages to add different implementations of the same function to a type. For instance:

package a
type T int
func X(t T) {
   type I interface { F() }
    t.(I).F()
}

package b
func (t T) F() { println("B") }
a.X(T(0))

package c
func (t T) F() { println("C") }
a.X(T(0))

How does a.X know which F to call?

@urandom

This comment has been minimized.

Show comment
Hide comment
@urandom

urandom Aug 11, 2017

I have already addressed this in the original proposal:

Importing multiple definitions of a method for a given type is an error

urandom commented Aug 11, 2017

I have already addressed this in the original proposal:

Importing multiple definitions of a method for a given type is an error

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Aug 11, 2017

Contributor

When and how is that error reported?

Also, if I extend string with the method String, and call fmt.Print(s) for some string value s, will the fmt package invoke the String method?

If it won't, then if I pass s as a value of type interface{} to some other package, and that package passes the interface{} value back to me by calling a different function in my package, what will happen if I use an interface conversion to fmt.Stringer and call the String method?

Contributor

ianlancetaylor commented Aug 11, 2017

When and how is that error reported?

Also, if I extend string with the method String, and call fmt.Print(s) for some string value s, will the fmt package invoke the String method?

If it won't, then if I pass s as a value of type interface{} to some other package, and that package passes the interface{} value back to me by calling a different function in my package, what will happen if I use an interface conversion to fmt.Stringer and call the String method?

@urandom

This comment has been minimized.

Show comment
Hide comment
@urandom

urandom Aug 11, 2017

@ianlancetaylor

When and how is that error reported?

It's the same rules as when you declare two functions with the same name in a given package, or do an import . "my/package", and "my/package" defines a function with the same name as another one in the current package.

Also, if I extend string with the method String, and call fmt.Print(s) for some string value s, will the fmt package invoke the String method?

Does the file that defies the Print function import that extension? Does it also check for Stringer before checking for string? If the answer is yes to both (it's currently no to both), then yes, String will be called.

If it won't, then if I pass s as a value of type interface{} to some other package, and that package
passes the interface{} value back to me by calling a different function in my package, what will happen if I use an interface conversion to fmt.Stringer and call the String method?

See my previous answer. The extension is scoped to the file that imports it, it does not taint the type for subsequent packages/files.Thus, nothing unexpected will happen.

urandom commented Aug 11, 2017

@ianlancetaylor

When and how is that error reported?

It's the same rules as when you declare two functions with the same name in a given package, or do an import . "my/package", and "my/package" defines a function with the same name as another one in the current package.

Also, if I extend string with the method String, and call fmt.Print(s) for some string value s, will the fmt package invoke the String method?

Does the file that defies the Print function import that extension? Does it also check for Stringer before checking for string? If the answer is yes to both (it's currently no to both), then yes, String will be called.

If it won't, then if I pass s as a value of type interface{} to some other package, and that package
passes the interface{} value back to me by calling a different function in my package, what will happen if I use an interface conversion to fmt.Stringer and call the String method?

See my previous answer. The extension is scoped to the file that imports it, it does not taint the type for subsequent packages/files.Thus, nothing unexpected will happen.

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Aug 11, 2017

Contributor

It's the same rules as when you declare two functions with the same name in a given package, or do an import . "my/package", and "my/package" defines a function with the same name as another one in the current package.

Well, no, because those examples are all reported as errors by the compiler. In Keith's example above, the compiler be invoked on each of those packages separately, so it will never see the erroneous code.

To be clear, I'm asking a question about implementation here. I understand that this is an error according to your proposal. I'm asking how it can be implemented.

If it won't, then if I pass s as a value of type interface{} to some other package, and that package
passes the interface{} value back to me by calling a different function in my package, what will happen if I use an interface conversion to fmt.Stringer and call the String method?

See my previous answer. The extension is scoped to the file that imports it, it does not taint the type for subsequent packages/files.Thus, nothing unexpected will happen.

I think it's quite odd that the same exact value with the same type would behave differently merely because I call, say

func Identity(v interface{}) interface{} { return v }

defined in some different package.

Contributor

ianlancetaylor commented Aug 11, 2017

It's the same rules as when you declare two functions with the same name in a given package, or do an import . "my/package", and "my/package" defines a function with the same name as another one in the current package.

Well, no, because those examples are all reported as errors by the compiler. In Keith's example above, the compiler be invoked on each of those packages separately, so it will never see the erroneous code.

To be clear, I'm asking a question about implementation here. I understand that this is an error according to your proposal. I'm asking how it can be implemented.

If it won't, then if I pass s as a value of type interface{} to some other package, and that package
passes the interface{} value back to me by calling a different function in my package, what will happen if I use an interface conversion to fmt.Stringer and call the String method?

See my previous answer. The extension is scoped to the file that imports it, it does not taint the type for subsequent packages/files.Thus, nothing unexpected will happen.

I think it's quite odd that the same exact value with the same type would behave differently merely because I call, say

func Identity(v interface{}) interface{} { return v }

defined in some different package.

@urandom

This comment has been minimized.

Show comment
Hide comment
@urandom

urandom Aug 11, 2017

Well, no, because those examples are all reported as errors by the compiler. In Keith's example above, the compiler be invoked on each of those packages separately, so it will never see the erroneous code.

In the above example, a panic would be produced, since X tries to assert t to the interface I without checking if it's safe to do so. From package a's standpoint, T doesn't define a method F. It's a runtime error, much like any other type assertion.

func Identity(v interface{}) interface{} { return v }

I'm sorry, I don't quite get how this example ties in with the proposal.

urandom commented Aug 11, 2017

Well, no, because those examples are all reported as errors by the compiler. In Keith's example above, the compiler be invoked on each of those packages separately, so it will never see the erroneous code.

In the above example, a panic would be produced, since X tries to assert t to the interface I without checking if it's safe to do so. From package a's standpoint, T doesn't define a method F. It's a runtime error, much like any other type assertion.

func Identity(v interface{}) interface{} { return v }

I'm sorry, I don't quite get how this example ties in with the proposal.

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Aug 11, 2017

Contributor

My comment about Identity is that in package p I define a package-specific method on T, I create a value v of type T, then I write v1 := identity.Identity(v).(T), and now v1 and v behave differently.

Contributor

ianlancetaylor commented Aug 11, 2017

My comment about Identity is that in package p I define a package-specific method on T, I create a value v of type T, then I write v1 := identity.Identity(v).(T), and now v1 and v behave differently.

@urandom

This comment has been minimized.

Show comment
Hide comment
@urandom

urandom Aug 11, 2017

@ianlancetaylor
Why do v1 and v behave differently in your example? To me, it appears that both of them will behave the same, since from your description everything is happening in the same package, and both v and v1 are of type T for which you've defined a new method in that same package.

urandom commented Aug 11, 2017

@ianlancetaylor
Why do v1 and v behave differently in your example? To me, it appears that both of them will behave the same, since from your description everything is happening in the same package, and both v and v1 are of type T for which you've defined a new method in that same package.

@urandom

This comment has been minimized.

Show comment
Hide comment
@urandom

urandom Aug 11, 2017

I would also like to add an addendum to the proposal:
Consider a file package a, which either defines, or imports an extension for type T (lets say the extra method is Foo), as well as a variable t T some where in that same file. It will have the method Foo. In an essence, t's static type would be T, and its dynamic type would be T + interface{ Foo() }. As per the Go spec, this variable is assignable back to T. And throughout it's lifetime, it would satisfy that interface.

Essentially, since it's dynamic type features the new interface, passing it to an external functions that expects such an interface is a valid call.
If it is however given to a function whose argument is of type T, that argument will no longer satisfy interface { Foo() }, since it is just a T and in that context, the extension hasn't been imported.

If we go back to the fmt.Println example, if t T had been extended with a String() method, Println wouldn't be able to call it, since it's argument is of type interface{}, and the static type is T. If, however, there was a function PrintStringer(s Stringer), passing t would be valid.

urandom commented Aug 11, 2017

I would also like to add an addendum to the proposal:
Consider a file package a, which either defines, or imports an extension for type T (lets say the extra method is Foo), as well as a variable t T some where in that same file. It will have the method Foo. In an essence, t's static type would be T, and its dynamic type would be T + interface{ Foo() }. As per the Go spec, this variable is assignable back to T. And throughout it's lifetime, it would satisfy that interface.

Essentially, since it's dynamic type features the new interface, passing it to an external functions that expects such an interface is a valid call.
If it is however given to a function whose argument is of type T, that argument will no longer satisfy interface { Foo() }, since it is just a T and in that context, the extension hasn't been imported.

If we go back to the fmt.Println example, if t T had been extended with a String() method, Println wouldn't be able to call it, since it's argument is of type interface{}, and the static type is T. If, however, there was a function PrintStringer(s Stringer), passing t would be valid.

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Aug 11, 2017

Contributor

OK, given a value v of type T with an extra method M(), what happens if I write

((interface {})(v)).(interface { M() })

? That is, I convert v to an empty interface value and then try to convert that value to an interface with the added method M?

Contributor

ianlancetaylor commented Aug 11, 2017

OK, given a value v of type T with an extra method M(), what happens if I write

((interface {})(v)).(interface { M() })

? That is, I convert v to an empty interface value and then try to convert that value to an interface with the added method M?

@urandom

This comment has been minimized.

Show comment
Hide comment
@urandom

urandom Aug 12, 2017

If that expression is within the scope of the extension, then this conversion will success, since the first part will be an interface{} with a dynamic type T, which still gets extended with the method M(), thus satisfying the interface.

On the other hand, if v was created where the extension was in scope, then passed to a scope without the extension, it depends what the argument type was. If the argument's type was interface { M() }, then the conversion will still succeed, since the result of the first part will have a dynamic type of T + interface { M() }, and is thus convertible back to that interface. However, if the argument type was T, then in an essence, v was assigned back to a value of it static type, thus losing the extension.

urandom commented Aug 12, 2017

If that expression is within the scope of the extension, then this conversion will success, since the first part will be an interface{} with a dynamic type T, which still gets extended with the method M(), thus satisfying the interface.

On the other hand, if v was created where the extension was in scope, then passed to a scope without the extension, it depends what the argument type was. If the argument's type was interface { M() }, then the conversion will still succeed, since the result of the first part will have a dynamic type of T + interface { M() }, and is thus convertible back to that interface. However, if the argument type was T, then in an essence, v was assigned back to a value of it static type, thus losing the extension.

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Aug 12, 2017

Contributor

Thanks. It seems to me that this behavior is confusing. An type has a method set, and that has a clear meaning in conjunction with interface types. With this proposal, the association of the method set and interface types depends crucially on how the values are passed around. To me that seems like an undesirable feature.

Contributor

ianlancetaylor commented Aug 12, 2017

Thanks. It seems to me that this behavior is confusing. An type has a method set, and that has a clear meaning in conjunction with interface types. With this proposal, the association of the method set and interface types depends crucially on how the values are passed around. To me that seems like an undesirable feature.

@urandom

This comment has been minimized.

Show comment
Hide comment
@urandom

urandom Aug 12, 2017

@ianlancetaylor
Why is it confusing? From the point of view of the outside function that expects a T, this is exactly what it is getting. It is perfectly clear in its intent. It might not even know about any existing extensions for type T, nor has it imported any . In fact, it would have been confusing if it was getting a T + interface { M() } instead, even though it was only requesting a T, which is itself a non-interface type.

From the standpoint of the current Go spec, my proposed behavior seems anything but confusing.

urandom commented Aug 12, 2017

@ianlancetaylor
Why is it confusing? From the point of view of the outside function that expects a T, this is exactly what it is getting. It is perfectly clear in its intent. It might not even know about any existing extensions for type T, nor has it imported any . In fact, it would have been confusing if it was getting a T + interface { M() } instead, even though it was only requesting a T, which is itself a non-interface type.

From the standpoint of the current Go spec, my proposed behavior seems anything but confusing.

@balasanjay

This comment has been minimized.

Show comment
Hide comment
@balasanjay

balasanjay Aug 12, 2017

Contributor

This seems loosely inspired by C#'s extension methods.

If Go did want to do something in this space (making function calls chainable), then another potential source of inspiration is Javascript's proposed "::" operator.

I could imagine making the expression expr::pkg.Function(arg1, arg2) syntactic sugar for pkg.Function(expr, arg1, arg2).

The strings example would then look like this:

func Actions(actions []string) string {
    return strings.Join(actions, "\n")::
          strings.Replace("desc", "name", -1)::
          strings.Replace("alpha", "beta", -1)
}

I've always thought extension methods are a neat idea, as they allow libraries to define a small core interface (IEnumerable in .NET's case) and have "fake methods" on that interface to allow a relatively expansive and ergonomic fluent API. I imagine a hypothetical future Go implementation of Apache Beam, or a hypothetical Go2 (with generics) implementation of functional transformations (map/reduce/filter/etc) would all benefit from such a structure.

That being said, with either proposal, I think the standard library would probably need to be updated. With the original proposal, to rewrite packages to make use of this functionality, or with the "::" mini-proposal, to possibly reorder parameter lists to put a logical element in the first position.

Contributor

balasanjay commented Aug 12, 2017

This seems loosely inspired by C#'s extension methods.

If Go did want to do something in this space (making function calls chainable), then another potential source of inspiration is Javascript's proposed "::" operator.

I could imagine making the expression expr::pkg.Function(arg1, arg2) syntactic sugar for pkg.Function(expr, arg1, arg2).

The strings example would then look like this:

func Actions(actions []string) string {
    return strings.Join(actions, "\n")::
          strings.Replace("desc", "name", -1)::
          strings.Replace("alpha", "beta", -1)
}

I've always thought extension methods are a neat idea, as they allow libraries to define a small core interface (IEnumerable in .NET's case) and have "fake methods" on that interface to allow a relatively expansive and ergonomic fluent API. I imagine a hypothetical future Go implementation of Apache Beam, or a hypothetical Go2 (with generics) implementation of functional transformations (map/reduce/filter/etc) would all benefit from such a structure.

That being said, with either proposal, I think the standard library would probably need to be updated. With the original proposal, to rewrite packages to make use of this functionality, or with the "::" mini-proposal, to possibly reorder parameter lists to put a logical element in the first position.

@bcmills

This comment has been minimized.

Show comment
Hide comment
@bcmills

bcmills Aug 25, 2017

Member

Your strings.Replace example is exactly the use-case that composition and/or currying addresses in functional languages.

The :: operator that @balasanjay mentions seems closely related to the semicolon operator in pseudocode notations (see https://en.wikipedia.org/wiki/Function_composition#Alternative_notations).

Another option would be to provide the traditional right-to-left compose operator, perhaps in conjuction with a placeholder variable:

func Actions(actions []string) {
	strings.Replace(_, "alpha", "beta", -1) ∘
	strings.Replace(_, "desc", "name", -1,) ∘
	strings.Join(actions, "\n")
}
Member

bcmills commented Aug 25, 2017

Your strings.Replace example is exactly the use-case that composition and/or currying addresses in functional languages.

The :: operator that @balasanjay mentions seems closely related to the semicolon operator in pseudocode notations (see https://en.wikipedia.org/wiki/Function_composition#Alternative_notations).

Another option would be to provide the traditional right-to-left compose operator, perhaps in conjuction with a placeholder variable:

func Actions(actions []string) {
	strings.Replace(_, "alpha", "beta", -1) ∘
	strings.Replace(_, "desc", "name", -1,) ∘
	strings.Join(actions, "\n")
}
@urandom

This comment has been minimized.

Show comment
Hide comment
@urandom

urandom Aug 25, 2017

@bcmills
Your and @balasanjay 's do not directly address the two goals of my proposal. Your syntax is not more readable, since the package "strings" is still everywhere, and you still have to give it some form of a first argument. Second, it does nothing to allow a foreign type to conform to an interface. You'd still have to manually define a new type, and manually delegate to all methods of the underlying type.

urandom commented Aug 25, 2017

@bcmills
Your and @balasanjay 's do not directly address the two goals of my proposal. Your syntax is not more readable, since the package "strings" is still everywhere, and you still have to give it some form of a first argument. Second, it does nothing to allow a foreign type to conform to an interface. You'd still have to manually define a new type, and manually delegate to all methods of the underlying type.

@as

This comment has been minimized.

Show comment
Hide comment
@as

as Aug 25, 2017

Contributor

You'd still have to manually define a new type, and manually delegate to all methods of the underlying type.

https://play.golang.org/p/hPueAuqdqK

Contributor

as commented Aug 25, 2017

You'd still have to manually define a new type, and manually delegate to all methods of the underlying type.

https://play.golang.org/p/hPueAuqdqK

@urandom

This comment has been minimized.

Show comment
Hide comment
@urandom

urandom Aug 25, 2017

@as
Embedding only works for structs. Methods can be on anything.

urandom commented Aug 25, 2017

@as
Embedding only works for structs. Methods can be on anything.

@balasanjay

This comment has been minimized.

Show comment
Hide comment
@balasanjay

balasanjay Aug 29, 2017

Contributor

We'll have to agree to disagree on readability with respect to including packages in names. In my opinion, naming the source of functions is a good thing (and easily avoided in both current-Go and a hypothetical future-Go, with a local var replace = strings.Replace).

With respect to your second goal: I found it quite strange that interface satisfaction changes depending on which package a call-site is in. Currently, the result of the expression a.(I) depends on the dynamic type of a and the static type I. This proposal introduces (or so it seems to me) another dependency to these expressions; specifically on top-level statements in the file. That seems like an unfortunate consequence to me, and I was trying to point out that it was possible to solve goal 1 without also solving goal 2.

Here's the tc39 proposal for the "::" operator, for the record: https://github.com/tc39/proposal-bind-operator#examples. (Only the first two examples are relevant to Go, since Go already has its own syntax for method extraction)

Contributor

balasanjay commented Aug 29, 2017

We'll have to agree to disagree on readability with respect to including packages in names. In my opinion, naming the source of functions is a good thing (and easily avoided in both current-Go and a hypothetical future-Go, with a local var replace = strings.Replace).

With respect to your second goal: I found it quite strange that interface satisfaction changes depending on which package a call-site is in. Currently, the result of the expression a.(I) depends on the dynamic type of a and the static type I. This proposal introduces (or so it seems to me) another dependency to these expressions; specifically on top-level statements in the file. That seems like an unfortunate consequence to me, and I was trying to point out that it was possible to solve goal 1 without also solving goal 2.

Here's the tc39 proposal for the "::" operator, for the record: https://github.com/tc39/proposal-bind-operator#examples. (Only the first two examples are relevant to Go, since Go already has its own syntax for method extraction)

@urandom

This comment has been minimized.

Show comment
Hide comment
@urandom

urandom Aug 29, 2017

Currently, the result of the expression a.(I) depends on the dynamic type of a and the static type I. This proposal introduces (or so it seems to me) another dependency to these expressions; specifically on top-level statements in the file.

If I understood you correctly, you're worried that since I has to be imported as a type, that's a problem. But that's how dealing with types works right now. You cannot reference I unless you import it with the top-level import statement. In fact, this interaction doesn't change at all with this proposal.

Here's the tc39 proposal for the "::" operator, for the record: https://github.com/tc39/proposal-bind-operator#examples. (Only the first two examples are relevant to Go, since Go already has its own syntax for method extraction)

While this is very similar, my problem is that we already have a syntax for defining methods for a type, and then for calling them. So instead of reusing that syntax, bringing it to its logical conclusion, this introduces another operator :: that achieves goal 1 in almost the same way. It just seems a bit wasteful to me.

urandom commented Aug 29, 2017

Currently, the result of the expression a.(I) depends on the dynamic type of a and the static type I. This proposal introduces (or so it seems to me) another dependency to these expressions; specifically on top-level statements in the file.

If I understood you correctly, you're worried that since I has to be imported as a type, that's a problem. But that's how dealing with types works right now. You cannot reference I unless you import it with the top-level import statement. In fact, this interaction doesn't change at all with this proposal.

Here's the tc39 proposal for the "::" operator, for the record: https://github.com/tc39/proposal-bind-operator#examples. (Only the first two examples are relevant to Go, since Go already has its own syntax for method extraction)

While this is very similar, my problem is that we already have a syntax for defining methods for a type, and then for calling them. So instead of reusing that syntax, bringing it to its logical conclusion, this introduces another operator :: that achieves goal 1 in almost the same way. It just seems a bit wasteful to me.

@bcmills

This comment has been minimized.

Show comment
Hide comment
@bcmills

bcmills Sep 7, 2017

Member

For the particular case of string, another alternative would be to move the contents of the strings package to become methods on the built-in string type. Then you'd get the code from your examples without the need to import the strings package at all.

Do you have concrete use-cases for the proposed feature other than the strings package?

Member

bcmills commented Sep 7, 2017

For the particular case of string, another alternative would be to move the contents of the strings package to become methods on the built-in string type. Then you'd get the code from your examples without the need to import the strings package at all.

Do you have concrete use-cases for the proposed feature other than the strings package?

@urandom

This comment has been minimized.

Show comment
Hide comment
@urandom

urandom Sep 8, 2017

@bcmill,

iirc, it was stated that the reason the built-in types don't have methods is because that would complicate the spec, as those methods would have to be written there (i believe this was stated in an AMA on reddit).

As for use cases from the stdlib itself, strings and bytes are the no-brainers. Then there's math and its subpackages. IMHO, the API of math/bits can be greatly simplified if it defined methods for uint{-,16,32,64} instead of having methods like Len{-,16,32,64}. strconv functions can be distributed to their types. unicode could extend rune. If we assume Go2 will also come with generics, the stdlib should also come with packages with helpful map and slice functions (map/reduce/filter/etc for anyone that needs them).

Then you have third party libraries. Some, like github.com/pkg/errors, can benefit substantially by this. Others, like the various utility functions (I assume) we all write will be of benefit for our own projects. Examples of these would be the functions that obtain a value from a context and map it to a certain type, and these are nowadays quite predominant.

urandom commented Sep 8, 2017

@bcmill,

iirc, it was stated that the reason the built-in types don't have methods is because that would complicate the spec, as those methods would have to be written there (i believe this was stated in an AMA on reddit).

As for use cases from the stdlib itself, strings and bytes are the no-brainers. Then there's math and its subpackages. IMHO, the API of math/bits can be greatly simplified if it defined methods for uint{-,16,32,64} instead of having methods like Len{-,16,32,64}. strconv functions can be distributed to their types. unicode could extend rune. If we assume Go2 will also come with generics, the stdlib should also come with packages with helpful map and slice functions (map/reduce/filter/etc for anyone that needs them).

Then you have third party libraries. Some, like github.com/pkg/errors, can benefit substantially by this. Others, like the various utility functions (I assume) we all write will be of benefit for our own projects. Examples of these would be the functions that obtain a value from a context and map it to a certain type, and these are nowadays quite predominant.

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Sep 8, 2017

Contributor
Contributor

davecheney commented Sep 8, 2017

@bcmills

This comment has been minimized.

Show comment
Hide comment
@bcmills

bcmills Sep 8, 2017

Member

None of the methods on "GET" would be available on Get.

Calling the functions from the strings package on such a type already requires a conversion:
https://play.golang.org/p/fyV2ehifxv.

From that standpoint, putting the methods on the type seems like a strict improvement: then the author of the package could choose to embed it to retain the methods.

type Method struct{ string }

var Get = Method{ "GET" }
Member

bcmills commented Sep 8, 2017

None of the methods on "GET" would be available on Get.

Calling the functions from the strings package on such a type already requires a conversion:
https://play.golang.org/p/fyV2ehifxv.

From that standpoint, putting the methods on the type seems like a strict improvement: then the author of the package could choose to embed it to retain the methods.

type Method struct{ string }

var Get = Method{ "GET" }
@bcmills

This comment has been minimized.

Show comment
Hide comment
@bcmills

bcmills Sep 8, 2017

Member

it was stated that the reason the built-in types don't have methods is because that would complicate the spec, as those methods would have to be written there

That may be so, but it seems like an appendix of string methods would complicate the spec far less than adding aspects or importable methods.


math/bits

x.Len() without the bits prefix seems too ambiguous. So does x.TrailingZeros() (trailing zeroes in what base?).

The bits package, unlike strings, really does add information to the names of its functions.


unicode

That's an interesting case, because rune is an alias for int32. Should importing the unicode package add an IsNumber method to all int32 values? That seems confusing.


strconv

The Format, Quote, and Unquote functions could reasonably be methods. The Is functions have the same int32/rune problem as for unicode.

The Append functions currently effectively have a receiver type of []byte. Would you attach the Append functions to []byte, or to the individual types to be appended?


github.com/pkg/errors

That one is a weird case because the operands are already interfaces, not concrete types. What happens if the underlying values already implement one of the methods you would be adding?

Member

bcmills commented Sep 8, 2017

it was stated that the reason the built-in types don't have methods is because that would complicate the spec, as those methods would have to be written there

That may be so, but it seems like an appendix of string methods would complicate the spec far less than adding aspects or importable methods.


math/bits

x.Len() without the bits prefix seems too ambiguous. So does x.TrailingZeros() (trailing zeroes in what base?).

The bits package, unlike strings, really does add information to the names of its functions.


unicode

That's an interesting case, because rune is an alias for int32. Should importing the unicode package add an IsNumber method to all int32 values? That seems confusing.


strconv

The Format, Quote, and Unquote functions could reasonably be methods. The Is functions have the same int32/rune problem as for unicode.

The Append functions currently effectively have a receiver type of []byte. Would you attach the Append functions to []byte, or to the individual types to be appended?


github.com/pkg/errors

That one is a weird case because the operands are already interfaces, not concrete types. What happens if the underlying values already implement one of the methods you would be adding?

@urandom

This comment has been minimized.

Show comment
Hide comment
@urandom

urandom Sep 8, 2017

@bcmills

math/bits

I don't think it's ambiguous. The TrailingZeros() examples will used its signature, much like the argument is of a different type for all TrailingZeros* functions right now. There's as much ambiguity in uint16 and uint32 both having a TrailingZeros() method, as there is in different types implementing an io.Reader interface, for example

unicode

The current rules for type aliases state that any methods defined for type T1 would be available for the alias type T2. I would say that not following this rule would be horribly confusing.

strconv

If Append were to be turned to a method, logically []byte would be the receiver. It's the slice that's getting appended here, the rest is the data that goes into the slice. People coming from virtually any other language that has arrays/slices and methods will be right at home here.

github.com/pkg/errors

From the point of view of github.com/pkg/errors, nothing will change. The receiver is an error interface, which only has an Error() method. There would be no collisions when defining the method. From the point of view of the package that defines a type that implements the error interface, there would still be no change. As we know, the implementation is implicit, the type itself has no idea what error actually is. If it so happens to implement Wrap on its own, there would be no conflicts, since consumers of that type see the type itself, and if it is passed/assigned to an error, from the point of view of the consumer it's just an interface with an Error method, plus the extended methods (such as Wrap), defined in github.com/pkg/errors. The consumer itself doesn't know about any extra types defined in the underlying type itself, which is consistent with the current behavior in Go, where a consumer of an io.Reader wouldn't know about any other methods the actual underlying type would have.

urandom commented Sep 8, 2017

@bcmills

math/bits

I don't think it's ambiguous. The TrailingZeros() examples will used its signature, much like the argument is of a different type for all TrailingZeros* functions right now. There's as much ambiguity in uint16 and uint32 both having a TrailingZeros() method, as there is in different types implementing an io.Reader interface, for example

unicode

The current rules for type aliases state that any methods defined for type T1 would be available for the alias type T2. I would say that not following this rule would be horribly confusing.

strconv

If Append were to be turned to a method, logically []byte would be the receiver. It's the slice that's getting appended here, the rest is the data that goes into the slice. People coming from virtually any other language that has arrays/slices and methods will be right at home here.

github.com/pkg/errors

From the point of view of github.com/pkg/errors, nothing will change. The receiver is an error interface, which only has an Error() method. There would be no collisions when defining the method. From the point of view of the package that defines a type that implements the error interface, there would still be no change. As we know, the implementation is implicit, the type itself has no idea what error actually is. If it so happens to implement Wrap on its own, there would be no conflicts, since consumers of that type see the type itself, and if it is passed/assigned to an error, from the point of view of the consumer it's just an interface with an Error method, plus the extended methods (such as Wrap), defined in github.com/pkg/errors. The consumer itself doesn't know about any extra types defined in the underlying type itself, which is consistent with the current behavior in Go, where a consumer of an io.Reader wouldn't know about any other methods the actual underlying type would have.

@scudette

This comment has been minimized.

Show comment
Hide comment
@scudette

scudette Feb 22, 2018

I am new to Go but would like to chime in on a real use case that I am trying to solve. The issue is that I am implementing a library (say "Serialize") for custom serialization of types. Say my library is used in order to serialize types that are obtained from another library (say "X"). I expect the glue code to teach my library how to serialize X types and then pass these instances directly to my library.

So say I make an interface like:

type Serializer interface {
   Serialize() map[string]interface{}
}

Then I want the glue code to call X's functions to obtain instances and just be able to run Serialize.Serialize(X.MakeInstance(....)) where X.MakeInstance() returns one of the instances in X (note that X has no idea their objects will be serialized so they may return say an instance of:

type XYZ struct {
    field X.Foo
}

In other words it is not enough for the glue to cast X.XYZ to a delegate type because this will not convert embedded types (which may be returned by the serializer). I would really like to be able to define this in glue code:

func (self x.XYZ) Serialize() map[string]interface{} {
    return map[string]interface{} {
       "X_Foo": self.Foo
    }
}

The best solution I came up with so far is to design my own type dispatcher system (which is certainly possible). So glue code can register specific serializers for different types in X and the dispatcher can select the correct serializer by asserting to the required type (or using the reflect package). But this seems a bit clunky (although this is what I ended up doing in python for the same purpose). When I first read about Go's method dispatch system it seemed exactly like what I was doing by hand and so seemed perfect - only to be disappointed by learning that it was not allowed to add interfaces to external types from other packages.

It seems to me that this restriction makes the Go type system no different from traditional class based OO designs because methods can not be added to types outside the package they are defined in making the types non-maliable - if I want to make a type conform with a new interface I need to extend it (just like a class based design) or embed it (making me explicitly dispatch all the methods I care about) and this does not even solve the problem of receiving the old type from external sources since I will need to cast those to the new type somehow.

scudette commented Feb 22, 2018

I am new to Go but would like to chime in on a real use case that I am trying to solve. The issue is that I am implementing a library (say "Serialize") for custom serialization of types. Say my library is used in order to serialize types that are obtained from another library (say "X"). I expect the glue code to teach my library how to serialize X types and then pass these instances directly to my library.

So say I make an interface like:

type Serializer interface {
   Serialize() map[string]interface{}
}

Then I want the glue code to call X's functions to obtain instances and just be able to run Serialize.Serialize(X.MakeInstance(....)) where X.MakeInstance() returns one of the instances in X (note that X has no idea their objects will be serialized so they may return say an instance of:

type XYZ struct {
    field X.Foo
}

In other words it is not enough for the glue to cast X.XYZ to a delegate type because this will not convert embedded types (which may be returned by the serializer). I would really like to be able to define this in glue code:

func (self x.XYZ) Serialize() map[string]interface{} {
    return map[string]interface{} {
       "X_Foo": self.Foo
    }
}

The best solution I came up with so far is to design my own type dispatcher system (which is certainly possible). So glue code can register specific serializers for different types in X and the dispatcher can select the correct serializer by asserting to the required type (or using the reflect package). But this seems a bit clunky (although this is what I ended up doing in python for the same purpose). When I first read about Go's method dispatch system it seemed exactly like what I was doing by hand and so seemed perfect - only to be disappointed by learning that it was not allowed to add interfaces to external types from other packages.

It seems to me that this restriction makes the Go type system no different from traditional class based OO designs because methods can not be added to types outside the package they are defined in making the types non-maliable - if I want to make a type conform with a new interface I need to extend it (just like a class based design) or embed it (making me explicitly dispatch all the methods I care about) and this does not even solve the problem of receiving the old type from external sources since I will need to cast those to the new type somehow.

@as

This comment has been minimized.

Show comment
Hide comment
@as

as Feb 23, 2018

Contributor

@scudette I'm not sure what you mean by dispatch in this context, but embedding a type in another type silently forwards all the public methods as long as the embedding is anonymous.

Contributor

as commented Feb 23, 2018

@scudette I'm not sure what you mean by dispatch in this context, but embedding a type in another type silently forwards all the public methods as long as the embedding is anonymous.

@scudette

This comment has been minimized.

Show comment
Hide comment
@scudette

scudette Feb 23, 2018

Caveat: A Go newbie coming from predominately C++ and Python background so maybe when I am saying here does not make any sense :-).

@as I guess I see that embedding a type in a struct anonymously is the same as deriving from a base class (since all methods are automatically forwarded, the type can grow extra methods and can even override methods from the embedded type). I was referring to dispatch as non-anonymous embedding which in other OO languages would be composition and would require explicit delegation.

It was just a misunderstanding on my part - after reading the Go documentation which describes the type system as being very different from other OO languages with method dispatch instead of class hierarchy I misunderstood it. IMHO the restriction of attaching methods to types only within their type definition packages basically boils the type system into being a traditional class based language with a different syntax but not fundamentally different:

type Foo struct;
func (self Foo)Method() bool {

}

Is equivalent to (albeit with a stricter type system):

class Foo:
  def Method(self):
      return True

And embedding is equivalent to subclassing:

type Bar struct {
    Foo
    something SomeType
}

func (self Bar)OtherMethod() bool {
...
}

Is the same as:

class Bar(Foo):
   def OtherMethod(self):
      ....

Since Bar has both Method() and OtherMethod() but this does not solve the original problem of what to do when someone hands you a Foo instance and you need to support the interface by attaching the OtherMethod() to it. Now we have to do a non-trivial copy operation to convert a Foo to a Bar just so we can appease the type system that the object complies with the interface. It would be much easier to just add the interface to the Foo class without having to modify the base class code or extend it, which IIUC is the proposal in this issue.

I agree this can get very confusing if the baseclass already has this method (which one should win?) but this is easy to detect during compilation and linking, and in practice probably wont matter because if two modules are trying to support a protocol on a type they should (theoretically) have the same implementation.

scudette commented Feb 23, 2018

Caveat: A Go newbie coming from predominately C++ and Python background so maybe when I am saying here does not make any sense :-).

@as I guess I see that embedding a type in a struct anonymously is the same as deriving from a base class (since all methods are automatically forwarded, the type can grow extra methods and can even override methods from the embedded type). I was referring to dispatch as non-anonymous embedding which in other OO languages would be composition and would require explicit delegation.

It was just a misunderstanding on my part - after reading the Go documentation which describes the type system as being very different from other OO languages with method dispatch instead of class hierarchy I misunderstood it. IMHO the restriction of attaching methods to types only within their type definition packages basically boils the type system into being a traditional class based language with a different syntax but not fundamentally different:

type Foo struct;
func (self Foo)Method() bool {

}

Is equivalent to (albeit with a stricter type system):

class Foo:
  def Method(self):
      return True

And embedding is equivalent to subclassing:

type Bar struct {
    Foo
    something SomeType
}

func (self Bar)OtherMethod() bool {
...
}

Is the same as:

class Bar(Foo):
   def OtherMethod(self):
      ....

Since Bar has both Method() and OtherMethod() but this does not solve the original problem of what to do when someone hands you a Foo instance and you need to support the interface by attaching the OtherMethod() to it. Now we have to do a non-trivial copy operation to convert a Foo to a Bar just so we can appease the type system that the object complies with the interface. It would be much easier to just add the interface to the Foo class without having to modify the base class code or extend it, which IIUC is the proposal in this issue.

I agree this can get very confusing if the baseclass already has this method (which one should win?) but this is easy to detect during compilation and linking, and in practice probably wont matter because if two modules are trying to support a protocol on a type they should (theoretically) have the same implementation.

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Mar 13, 2018

Contributor

Presumably an extended method is not able to access unexported fields or methods of the type.

You can already create a new type with additional methods by using type embedding:

type New struct { pkg.Old }
func (n *New)  NewMethod() { ... }

The additional complexity required in the spec and implementation to implement this proposal does not seem justified by the incremental gain of functionality beyond what you can already do in the language.

Contributor

ianlancetaylor commented Mar 13, 2018

Presumably an extended method is not able to access unexported fields or methods of the type.

You can already create a new type with additional methods by using type embedding:

type New struct { pkg.Old }
func (n *New)  NewMethod() { ... }

The additional complexity required in the spec and implementation to implement this proposal does not seem justified by the incremental gain of functionality beyond what you can already do in the language.

@hooluupog

This comment has been minimized.

Show comment
Hide comment
@hooluupog

hooluupog Sep 9, 2018

Correct me if I'm wrong,
One of goals of the proposal is making function calls chainable. The current solution is creating a new type with methods. But we can not reuse the methods already provided by standard libraries instead of reinventing wheels. Take a piece of code from Go 2 draft design document for example,

package slices

func Map(type T1, T2)(s []T1, f func(T1) T2) []T2 {...}
func Reduce(type T1, T2)(s []T1, initializer T2, f func(T2, T1) T2) T2 {...}
func Filter(type T)(s []T, f func(T) bool) []T {}

s := []int{1, 2, 3}
//  I have made some small changes here.
s1 := slices.Map(s, func(i int) int { return i + 1 }) 
evens := slices.Filter(s1, func(i int) bool { return i%2 == 0 })
sum := slices.Reduce(evens, 0, func(i, j int) int { return i + j })

Make the functions in slices package chainable,

sum := slices.Reduce(
         slices.Filter(
           slices.Map(
              s, func(i int) int { 
                 return i + 1 
           }), 
           func(i int) bool { 
              return i%2 == 0
         }), 
         0, func(i, j int) int { 
               return i + j 
       })

The readability is worse than the original codes which have to create temporary variables to save intermediate results.
The proposal suggests creating and importing methods for non-local types(inspired by extension methods) which would complicate the spec. Then we have to create new type with methods by ourselves. We hope to make functions in slices package chainable without bringing in too much complexity and sacrificing code readability. How about introducing Pipeline Operator in Go? With pipeline operator, we could achieve the same goal without the need to update the standard library or rewrite packages.

s := []int{1, 2, 3}
// x |> f is syntactic sugar for f(x). 
sum := s |> func(e []int) []int { 
               return slices.Map(e, func(i int) int {
                  return i + 1 
               })
            } |>
            func(e []int) []int {
                  return slices.Filter(e, func(i int) bool { 
                     return i%2 == 0 
                  })
            } |>
            func(e []int) int {
                  return slices.Reduce(e, 0, func(i, j int) int { 
                     return i + j 
                  })
            }

With light weight anonymous function,

sum := s |> ((e) => slices.Map(e, (i) => i + 1)) |>
            ((e) => slices.Filter(e, (i) => i%2 == 0)) |>
            ((e) => slices.Reduce(e, 0, (i,j) => i + j))

With a placeholder variable:

sum := s |> slices.Map( _, _ + 1) |>
            slices.Filter( _, _ %2 == 0) |>
            slices.Reduce( _, 0, _ + _ )

The strings example would then look like this:

import "strings"

func Actions(actions []string) {
     actions |> strings.Join( _, "\n") |>
                strings.Replace( _, "desc", "name", -1) |>
                strings.Replace( _, "alpha", "beta", -1)
     }

hooluupog commented Sep 9, 2018

Correct me if I'm wrong,
One of goals of the proposal is making function calls chainable. The current solution is creating a new type with methods. But we can not reuse the methods already provided by standard libraries instead of reinventing wheels. Take a piece of code from Go 2 draft design document for example,

package slices

func Map(type T1, T2)(s []T1, f func(T1) T2) []T2 {...}
func Reduce(type T1, T2)(s []T1, initializer T2, f func(T2, T1) T2) T2 {...}
func Filter(type T)(s []T, f func(T) bool) []T {}

s := []int{1, 2, 3}
//  I have made some small changes here.
s1 := slices.Map(s, func(i int) int { return i + 1 }) 
evens := slices.Filter(s1, func(i int) bool { return i%2 == 0 })
sum := slices.Reduce(evens, 0, func(i, j int) int { return i + j })

Make the functions in slices package chainable,

sum := slices.Reduce(
         slices.Filter(
           slices.Map(
              s, func(i int) int { 
                 return i + 1 
           }), 
           func(i int) bool { 
              return i%2 == 0
         }), 
         0, func(i, j int) int { 
               return i + j 
       })

The readability is worse than the original codes which have to create temporary variables to save intermediate results.
The proposal suggests creating and importing methods for non-local types(inspired by extension methods) which would complicate the spec. Then we have to create new type with methods by ourselves. We hope to make functions in slices package chainable without bringing in too much complexity and sacrificing code readability. How about introducing Pipeline Operator in Go? With pipeline operator, we could achieve the same goal without the need to update the standard library or rewrite packages.

s := []int{1, 2, 3}
// x |> f is syntactic sugar for f(x). 
sum := s |> func(e []int) []int { 
               return slices.Map(e, func(i int) int {
                  return i + 1 
               })
            } |>
            func(e []int) []int {
                  return slices.Filter(e, func(i int) bool { 
                     return i%2 == 0 
                  })
            } |>
            func(e []int) int {
                  return slices.Reduce(e, 0, func(i, j int) int { 
                     return i + j 
                  })
            }

With light weight anonymous function,

sum := s |> ((e) => slices.Map(e, (i) => i + 1)) |>
            ((e) => slices.Filter(e, (i) => i%2 == 0)) |>
            ((e) => slices.Reduce(e, 0, (i,j) => i + j))

With a placeholder variable:

sum := s |> slices.Map( _, _ + 1) |>
            slices.Filter( _, _ %2 == 0) |>
            slices.Reduce( _, 0, _ + _ )

The strings example would then look like this:

import "strings"

func Actions(actions []string) {
     actions |> strings.Join( _, "\n") |>
                strings.Replace( _, "desc", "name", -1) |>
                strings.Replace( _, "alpha", "beta", -1)
     }
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment