DIP: Function pointers and Delegate Parameters Inherit Attributes fro…

[WalterBright]

…m Function

[atilaneves]

What happens if the user doesn't want inherited attributes? @safe/@trusted/@System are easy and trivial to "revert", but the same can't be said for pure, nothrow or scope.

[WalterBright]

What happens if the user doesn't want inherited attributes?

Discussed on line 113. Also, scope wouldn't be inherited at all.

[atilaneves]

I don't know how I missed that, sorry.

[Geod24]

Only functions are mentioned here, nothing about templates functions.
Templated functions usually have their attributes inferred, and taking a delegate affect this inference. Given the prevalence of templated functions, how useful is this ?

Additionally, it is more magic to explain to the prospective user. I am not looking forward to explain the difference between the following:

/// Example 1
alias DG = void delegate (scope const char[]) @nogc;
void toString (scope DG sink) @safe;

/// Example 2
void toString (scope void delegate (scope const char[]) @nogc sink) @safe;

/// Example 3+4
@safe:
void toString (T) (const ref T val, scope void delegate (scope const char[]) @nogc sink);
void toString (T) (const ref T val, scope DG sink);

In this case, example 1 behaves differently from example 2, but the same as example 3 and 4.

Finally, one of the main problem we have encountered with delegates is not copy pasting attributes from the function to the delegate parameter, but the inability to have a non-templated function define its attributes as depending on the delegate.
Take the classic toString sink example from Throwable:

class Throwable
{
    void toString(scope void delegate(in char[]) sink) const { ... }
}

What I want, as a user, is the ability to pass a @safe delegate and for the compiler to understand toString is @safe. Likewise, if I pass a pure @nogc delegate, and so on. This is the real problem that is preventing many use cases. On the other hand, having to duplicate some attributes has never been a problem, and this DIP doesn't give us any new feature, it just brings a doubtful benefit for a corner case.

[Bolpat]

@Geod24 The case you describe with Throwable's toString — let's call it the sink issue: The toString function acts pure, nothrow, @safe and/or @nogc (any subset) if the sink delegate does. If it were for all of the attributes, you could make 16 overloads which do exactly the same and just differ in attributes — just to accommodate any attributes of the caller of toString.

The sink issue could be solved by slightly changing what the attributes mean. In the current state, a pure annotated function must act pure — always. That's reasonable as long it doesn't take function pointers or delegates. Maybe functions should be considered pure (or whatever of the four attributes) with respect to their function pointer or delegate parameters?
The rule of thumb would be: If you give aforementioned toString a pure sink, it acts pure. The caller of toString(sink) knows whether sink is pure or not — and therefore knows whether the call toString(sink) is pure or not.

I've thought about that some time ago and the edge cases of nested function types are a little mess. By nested function types, imagine if the sink parameter of toString didn't take a char[] but a function pointer or delegate. This is rare, rather theoretical and hard to reason about — but the language semantics would need to answer that.

It would still break some code: If an always-pure function gets a non-pure argument, that call wouldn't be considered pure anymore. Probably corner cases that twirl around function pointers. I have no idea how relevant those corner cases are.

[Bolpat]

Function lazy parameters are internally delegates. Those generated delegates somewhat inherit the function's attributes. And delegate vararg arrays (something of the form

RetType delegate(ArgTs) @attrib [] ...

for an arbitrary number of lazy parameters work the same when the delegates are constructed by the compiler. If you write the delegate out yourself, it doesn't work.

alias Dg = int delegate() @nogc @safe pure nothrow;
void funcWithLazies(Dg[] dgs...) @nogc @safe pure nothrow { }

int global = 10;

int func() @safe
{
    import std.stdio;
    writeln("func FTW!");
    return global++;
}

void main() @safe
{
    static assert( __traits(compiles, funcWithLazies(      func))); // 1
    static assert(!__traits(compiles, funcWithLazies(() => func))); // 2
}

In this example, 1 compiles because whether the call to func is e.g. pure is irrellevant. The call is considered to happen in main, not in funcWithLazies.
On the other hand. 2 does not compile because the delegate being explicit is only @safe, but neither of the other attributes required.

I have no knowledge of how the compiler handles attributes, but if it can do something like this already (lazy parameters in both ways), why not give the user this?
I could imagine a new storage class (calles maybe?) that can be applied to function parameters. If the calls parameter type contains function pointers or delegates (fp/d), the outermost fp/d layer inherits the attributes of the function. If attributes for the function are being inferred, the outermost fp/d type would considered pure, nothrow, @safe, and @nogc to not interfere with attribute inference.

int func(calls R delegate(Args)* dgp) pure { return (*dgp)(); }

is checked as if it were

int func(R delegate(Args) pure * dgp) pure { return (*dgp)(); }

Calling func with some fp/d would be valid (in terms of attributes) if calling the fp/d would be valid where func is called.
In the introductory example, funcWithLazies with its parameter annotated calls, the line 2 would compile: After the lowering of the vararg, the call is funcWithLazies([() => func()]). For it to pass (in terms of attributes) the delegate () => func() needs to be @safe (the attribute of main), but not the other attributes that funcWithLazies is annotated with.

I have no idea, how the compiler handles attributes, but this is what's already happening for line 1. Basically, the compiler would only allow you to use some of its magic.
The calls storage class can be inferred if the body of the function is present. The compiler can determine if the fp/d is being called or not. A counterexample where the fp/d would not be called is swap(fp1, fp2) for some fp1 and fp2 some fp/ds.
With the new storage class, no such code is broken. (In contrast to my comment above.)

[WalterBright]

Given the prevalence of templated functions, how useful is this ?

Template functions do have their limitations, such as they can't be virtual.

example 1 behaves differently from example 2

I don't see a difference.

What I want, as a user, is the ability to pass a @safe delegate and for the compiler to understand toString is @safe. Likewise, if I pass a pure @nogc delegate, and so on. This is the real problem that is preventing many use cases.

That is, indeed, precisely what templates are for, and this behavior (inference from templates) is relied on by much of Phobos.

having to duplicate some attributes has never been a problem

I encountered it in Phobos when I was trying to replace lazy parameters in order to avoid the problems lazy has with more advanced pointer tracking. I expect it will come up repeatedly with trying to obsolete lazy.

this DIP doesn't give us any new feature

It gives a path to replacing lazy, which has a murky specification, with a well-specified construct. It will render lazy obsolete, with an easy replacement.

[WalterBright]

@Bolpat I'm not sure what you're really proposing. Adding more attributes looks simple, but they always seem to cause unexpected problems. I'd like to try really hard to avoid new attributes.

[Geod24]

Template functions do have their limitations, such as they can't be virtual.

👍

I don't see a difference.

Sorry, I realize I was missing the attribute on the function.
The difference is that your approach is only from one side.
This only explores the conversion from void foo(void delegate() bar) to void foo(DG bar), as a mean to escape this mechanism. But what happens when the user refactors his/her code, and turns a delegate into an alias, and suddenly, things don't compile anymore ?

That is, indeed, precisely what templates are for, and this behavior (inference from templates) is relied on by much of Phobos.

I'm just going to quote you:

Template functions do have their limitations, such as they can't be virtual.

We had this discussion during out last meeting. @atilaneves seemed to have experienced the problem as well.

I encountered it in Phobos when I was trying to replace lazy parameters in order to avoid the problems lazy has with more advanced pointer tracking. I expect it will come up repeatedly with trying to obsolete lazy.

In the DIP, lazy is seen as a "nice side effect", but it seems to be the main motivation for this DIP.