Fix Issue 809 - Should be possible to convert lazy argument to delegate

[RazvanN7]

&dg will extract the delegate, if dg is an lazy argument. I'm not sure if this needs a spec PR or not; if it does, where would it be appropriate to add it? Maybe the lazy parameters section [1]?

[1] https://dlang.org/spec/function.html#lazy-params

[dlang-bot]

Thanks for your pull request and interest in making D better, @RazvanN7! We are looking forward to reviewing it, and you should be hearing from a maintainer soon.
Please verify that your PR follows this checklist:

• My PR is fully covered with tests (you can see the coverage diff by visiting the details link of the codecov check)
• My PR is as minimal as possible (smaller, focused PRs are easier to review than big ones)
• I have provided a detailed rationale explaining my changes
• New or modified functions have Ddoc comments (with Params: and Returns:)

Please see CONTRIBUTING.md for more information.

---

If you have addressed all reviews or aren't sure how to proceed, don't hesitate to ping us with a simple comment.

Bugzilla references

Auto-close	Bugzilla	Severity	Description
✓	809	enhancement	Should be possible to convert lazy argument to delegate

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub fetch digger
dub run digger -- build "master + dmd#10264"

[Geod24]

Love this change, but it needs a runnable test, not compilable.

[JinShil]

Maybe the lazy parameters section [1]?

Yeah, makes sense to me.

[aliak00]

Is it ok that exp.e1 here and following all of this could be mutated by lines 6247 and 6248 if it is a TOK.variable?

[RazvanN7]

It should be fine because it is only performing semantic analysis, thus enriching the AST. In order to be sure that you have a delegate there you need to do some transformations.

[RazvanN7]

@Geod24 Done.

[RazvanN7]

Spec change: dlang/dlang.org#2686

[JinShil]

I'm having second thoughts about this. Suppose we were to some day introduce rvalue references. Would we then want the & operator to return a reference to the return value?

[aliak00]

In addition to what JinShil said, I also think there's some inconsistency here. Currently taking the address of a delegate gives you a pointer to that delegate. If lazy is to be viewed as a delegate (as making sense according to the bug report) then there's an asymmetry between typeof(&lazyParam) and typeof(&delegateType). Is that ok?

[RazvanN7]

I'm having second thoughts about this. Suppose we were to some day introduce rvalue references. Would we then want the & operator to return a reference to the return value?

I think that if we were to introduce rvalue references (which I hope we will not), one solution might be to consider &dg as extracting the delegate and &dg() as having a reference to the return value.

If lazy is to be viewed as a delegate (as making sense according to the bug report) then there's an asymmetry between typeof(&lazyParam) and typeof(&delegateType). Is that ok?

Yes. Today, whenever you use lazyParam, the compiler interprets it as lazyParam() i.e. a call to a delegate; by using the & operator you specify to the compiler that you want to extract the delegate; you still have the call operator () to disambiguate whenever you want.

[aliak00]

Yes. Today, whenever you use lazyParam, the compiler interprets it as lazyParam() i.e. a call to a delegate; by using the & operator you specify to the compiler that you want to extract the delegate; you still have the call operator () to disambiguate whenever you want.

Mhm. Ok. Are you sure? Below when I reference d it does not seem to be interpreted as d(). And if it was, then this patch would change the meaning of what &d is right?

import std;

alias D = ref int delegate();

ref f(lazy D d) {
    return d();
}

void main() {
    int x = 3;
    ref int g() {
        return x;
    }
    f(&g);
    writeln(x);
}

Is lazy D d supposed to behave like a D? It doesn't seem

[RazvanN7]

@aliak00 I don't see any references to d in your example. Do you mean to g? If yes, then this patch does not touch that case, because it extracts the delegate from lazy params only.

Is lazy D d supposed to behave like a D? It doesn't seem

Not really. lazy D d is not an lvalue, whereas D d is.

[aliak00]

Hmm. Seems I didn't even finish my typing 😛 I did not mean g. d is referenced inside f. And if a reference to lazyParam is interpreted as lazyParam() then the reference to d should be interpreted as d() yes? Or did I misunderstand? In which case, &d should be &d()? and d() returns a delegate, which is addressable?

(the code i posted above uses d() and not d because the latter does not compile currently).

[RazvanN7]

Ok. I understand. Currently, how you've written the example it will not compile because d is a function call (however you write it d or d()), therefore an rvalue, and you cannot take the address of an rvalue.

With my patch, this does not change; you will have the same behavior, however, if you write return &d;, then you can extract the delegate of the lazy param and return a reference to it. How does that sound?

[aliak00]

Ah true. Sorry, the example does not compile yes. Yeah I understand the proposal. It's just that my understanding of what a lazy T should be is maybe incorrect. So:

import std;

void main() {
    f0( { return 3; } );
    f1( { return 3; } );
}

void f0(int delegate() x) {
    auto a = x;
    auto b = &x;
    pragma(msg, typeof(b));
}
void f1(lazy int delegate() x) {
    auto a = x;
    auto b = &x;
    pragma(msg, typeof(b));
}

To me, if we say that lazyParam is evaluated as lazyParam() then the above two functions should compile and produce the same result no? It doesn't right now (should it?) so my understanding seems incorrect.

[RazvanN7]

int delegate() x is different than lazy int delegate() x in the respect that in the first case x is an lvalue while in the latter it is an rvalue. That is the only difference. You can check this out in the spec [1], the 3rd rule: "A lazy parameter cannot be an lvalue.".

That is why your first example compiles (you can take the address of a delegate() variable), but the second one does not: the lazy parameter is an rvalue.

I hope this explanation makes things a bit clearer.

[1] https://dlang.org/spec/function.html#lazy-params

[aliak00]

Yep, finally got it! Thanks a lot @RazvanN7

[RazvanN7]

Don't mention it!

Test is now runnable

[Geod24]

Another thing which is not handled, much more fundamental:
At the moment lazy are scope delegate. Meaning they don't perform memory allocation.
If you can extract them, it means the extract delegate must be scope, and you need to add a test with -dip1000 otherwise it breaks safety.

[RazvanN7]

@Geod24 Nice catch! I added a fail compilation test.

[RazvanN7]

Autotester failure is unrelated.

[WalterBright]

Use isVarExp()

[WalterBright]

Use isVarExp()

[WalterBright]

Use isCallExp()

[WalterBright]

Use isVarExp(). Also, how do you know callExp.e1 is a VarExp?

[RazvanN7]

Because resolvePropertiesX replaces the VarExp with a CallExp for which e1 is the initial VarExp. (look at line 1302 in expressionsem.d)

[WalterBright]

I do not understand why "extracting a delegate" permanently alters the semantics?

[RazvanN7]

Expression semantic on an AddrExp will rewrite the AST so that we have the name of a delegate instead of variable; after that when resolve properties is called on the newly rewritten AST node, when it finds the var name it will try to replace it again as a CallExp, so we need to store the fact that the delegate was already extracted.

[WalterBright]

Use type.toBasetype().ty, not type.ty

[WalterBright]

See my comments.

[WalterBright]

I'm unsure as well that this won't simply produce other inconsistencies.

Is this a problem that actually needs solving? Note that the bugzilla issue gives a simple workaround.

[RazvanN7]

@WalterBright I made the changes. Thanks for your feedback.

Is this a problem that actually needs solving? Note that the bugzilla issue gives a simple workaround

It seems that folks want this as it is more expressive than the workaround.

[andralex]

Spec change: dlang/dlang.org#2686

I think this is worth moving forward with. cc @atilaneves

[andralex]

Let's see if @WalterBright is okay with the changes following the tech review.

On the semantics side, lazy has always been a black sheep - a delegate in disguise. Allowing extracting the underlying delegate brings it closer to normalcy.

[atilaneves]

I like it but I'll defer to @WalterBright.

[RazvanN7]

Ping.

[RazvanN7]

Another respectful ping on this.

Comments addressed

[thewilsonator]

Adding 72h -> merge