Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issue 18751: chunkBy predicate cannot access local variable. #6441

Closed
wants to merge 3 commits into from

Conversation

quickfur
Copy link
Member

The reason is that the inner range of ChunkByImpl must be a global template, otherwise we run into the infamous problem of nested templates being unable to access the local context of an alias template parameter.

The true fix is to fix the compiler to be able to handle this properly, but given the age of this issue, I'm not holding my breath on it. And in the meantime, chunkBy needs to be usable in current real-world code.

@dlang-bot
Copy link
Contributor

Thanks for your pull request, @quickfur!

Bugzilla references

Auto-close Bugzilla Severity Description
18751 normal chunkBy predicate cannot access local variable

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub fetch digger
dub run digger -- build "master + phobos#6441"

@@ -1705,6 +1705,63 @@ if (isInputRange!Range && !isForwardRange!Range)
}
}

// Inner range
struct ChunkByGroupImpl(alias eq, Range, Impl)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

private

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Copy link
Member

@wilzbach wilzbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI complains about @safe unittest + undocumented public symbols.

@@ -1883,6 +1887,19 @@ if (isForwardRange!Range)
assert(grp1.save.equal([1, 3, 5]));
}

// Issue 18751
unittest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ouch. I added @safe and suddenly dmd is complaining:

std/algorithm/iteration.d(1899): Error: @safe function std.algorithm.iteration.__unittest_L1891_C7 cannot call @system function std.algorithm.iteration.__unittest_L1891_C7.chunkBy!((i, j) => data[i] == data[j], int[]).chunkBy
std/algorithm/iteration.d(1899): Error: @safe function std.algorithm.iteration.__unittest_L1891_C7 cannot call @system destructor std.algorithm.iteration.__unittest_L1891_C7.ChunkByImpl!(__lambda1, int[]).ChunkByImpl.~this
std/algorithm/iteration.d(1900): Error: @safe function std.algorithm.iteration.__unittest_L1891_C7 cannot call @system function std.algorithm.comparison.equal!(equal).equal!(ChunkByImpl!(__lambda1, int[]), int[][]).equal
std/algorithm/iteration.d(1900): Error: @safe function std.algorithm.iteration.__unittest_L1891_C7 cannot call @system generated function std.algorithm.iteration.__unittest_L1891_C7.ChunkByImpl!(__lambda1, int[]).ChunkByImpl.__fieldPostblit

I've no idea what happened; I literally just copy-n-pasted the implementation from the original code pretty much untouched, except to add template arguments so that it will compile.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any ideas?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry. I forgot that chunkBy isn't @safe.
Just @System then :/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(this is due to its use of RefCounted under the hood which isn't @safe)

@quickfur
Copy link
Member Author

I can't figure it out. What is it about std.demangle that's causing the same error this PR is supposed to fix? How could it possibly have worked before?

@quickfur
Copy link
Member Author

I give up. I've no idea how to coax the compiler to accept the code in all cases. It seems that fixing one issue leads to another, and fixing the other leads back to the first problem. Any help would be appreciated. This issue is making chunkBy basically unusable in my code.

@JackStouffer
Copy link
Member

This looks like something that needs to be fixed in the compiler rather than the library. Voldemort types shouldn't (in a perfect world) affect this.

@quickfur
Copy link
Member Author

I agree, but this is a well-known current limitation. There have been a good number of Phobos PRs that move Voldemorts into private module-level templates precisely because of this problem.

However, what I don't understand is, why it's still failing even after I moved the Voldemort out, or worse yet, why it works as a nested struct but not as a separate template.

If this can be fixed in the compiler, it would be wonderful. In the meantime, though, chunkBy is essentially unusable in my code. :-/

@JackStouffer
Copy link
Member

I’d post this code to the bug report and move it to a DMD bug. I’ll close this as it seems the common way to fix this type of issue is a breaking change.

@quickfur
Copy link
Member Author

Fine. Though IIRC, the whole thing is one massive hack anyway (allowing alias parameters to bind to lambdas that access the caller's local context), so the chances of this actually getting fixed in dmd are pretty slim.

@quickfur quickfur deleted the issue18751 branch April 11, 2018 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants