Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: Go 2: a function type for all signatures #41478

Open
siadat opened this issue Sep 18, 2020 · 5 comments
Open

proposal: Go 2: a function type for all signatures #41478

siadat opened this issue Sep 18, 2020 · 5 comments

Comments

@siadat
Copy link
Contributor

@siadat siadat commented Sep 18, 2020

(I expect this suggestion to be rejected, but I thought I put it out there for others to judge :)

Introduction

In various libraries, including text/template, kubernetes, and others, parts of the code are responsible for calling functions whose signatures the libraries do not know.

The purpose of this issue is to discuss briefly the idea of adding a generic type for all functions. This type would only indicate that its value is callable. It implies no restrictions on the signature of the function. This would lead to simpler Go code that is tool-friendly. Statistic analyzers would be able to find callers or building a callgraph that are otherwise hidden behind reflections.

Existing workarounds

At the moment, a callable is passed around as an empty interface{} value and later on invoked using the reflect package. We cannot assert this interface{} value because the library is unaware of the exact signature of the underlying function.

Reflection makes it harder for static analyzers to figure out what the program is doing. Pointer analysis algorithms are already computationally expensive (eg Andersen's algorithm is O(n3) in worse cases) and reflection adds to this cost.

Analogy with interface{}

As an analogy, consider interface values. A broker library (eg a caching or messaging library) receives values wrapped inside an empty interface{}, and later returns it without inspecting the underlying type of it.

Here, the new callable type would simplify the code that is responsible for calling functions with arbitrary signature.

How about generics?

Using Go2's generics, we would be able to supply the exact type of the function, however, the caller still needs to use reflection or assertion to call the function.

Look and feel

If we had a callable type, we would use it like this:

func Call(f callable) []interface{} {
	args := []interface{}{3.14, 16}
	rets := f(args)
	return rets
}

f is a callable and is called like a normal function. It accepts and returns a slice of interface{}s.

@gopherbot gopherbot added this to the Proposal milestone Sep 18, 2020
@gopherbot gopherbot added the Proposal label Sep 18, 2020
@ianlancetaylor ianlancetaylor changed the title proposal: Go2: a function type for all signatures proposal: Go 2: a function type for all signatures Sep 18, 2020
@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Sep 18, 2020

I don't understand the difference between the proposed callable type and the type func([]interface{}) []interface{}.

Perhaps you mean that any function at all could be assigned to a variable of type callable. Then something would convert the call using []interface{} for the parameters and results. But that seems equivalent to making callable be interface{} and using a small wrapper around reflect.Call. I don't understand what advantage we get from callable. You mention static analyzers, but how does it help a static analyzer to know that a value of type interface{} is callable? It still doesn't know what actual function it represents.

@OneOfOne
Copy link
Contributor

@OneOfOne OneOfOne commented Sep 18, 2020

I think he's trying to avoid reflect.Call's overhead, which improved over the years but still wouldn't be as efficient as the direct call.

But that wouldn't be really possible without varargs generics

// i *think* this is what he was trying to achieve, which isn't possible
func [inArgs ....any, outArgs ...any, F callable(...inArgs) (...outArgs)] Call(f F, args ...inArgs) (...outArgs) {
     return f(...args)
}
@siadat
Copy link
Contributor Author

@siadat siadat commented Sep 18, 2020

This is a big change for such a small advantage (which I will try to explain). So, I am going to close it. But for the sake of discussion, in the rest of this comment I'll pretend as if I am still defending it! lol

Perhaps you mean that any function at all could be assigned to a variable of type callable.

Yes. For example, f1 and f2 are callables:

var f1 callable = func() {}
var f2 callable = strings.Join
// var f3 callable = 3.14 // ERROR: float64 is not callable

rets1 := f1()
rets2 := f2([]interface{}{ []string{"a", "b"}, "+" })

fmt.Printf("%#v\n", rets1)
fmt.Printf("%#v\n", rets2)

// Output:
// []interface {}{}
// []interface {}{"a+b"}

You mention static analyzers, but how does it help a static analyzer to know that a value of type interface{} is callable? It still doesn't know what actual function it represents.

Yes, that's correct, static checkers do not know the exact function an interface{} represents in runtime, but there are pointer analysis algorithms for discovering the set of possible types an interface{} value could have. They could, for example, say that a variable of type interface{} at line 10 is either an int or an io.Reader. One of these algorithms is implemented in x/tools/go/pointer (Alan Donovan et al) and used by the guru pointsto command. Here is a test case.

These pointer analysis algorithms are admittedly slow. The one I linked works by walking through the SSA functions. However, it has to handle reflect.Calls in a way that is not as straightforward. I meant to reduce exactly this complexity (of handling reflect.Call). Plus the fact that removing reflect dependency would simplify codes that need to invoke functions with unknown signatures (eg caches, schedulers, the template example etc).

I think he's trying to avoid reflect.Call's overhead

I am trying to avoid reflect.Call, but not because of its runtime overhead. Instead, I was trying to argue that something like the callable would simplify the code and make it easier to analyze the code (by making all function invocations first-class and uniform, as opposed to some of them hiding behind reflect.Call).

😅

@siadat siadat closed this Sep 18, 2020
@siadat
Copy link
Contributor Author

@siadat siadat commented Sep 18, 2020

I noticed closed issues are hard to find in Github. Sorry about that. I am reopening it temporarily for a few days just to make it easier for you to find it. Feel free to close/open/comment any time you want :)

@siadat siadat reopened this Sep 18, 2020
@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Sep 30, 2020

Based on the discussion above, this is a likely decline. Leaving open for four weeks for final comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.