Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Reduce the genericity of closures in the iterator traits #62429
By default, closures inherit the generic parameters of their scope,
This does make the closures more cumbersome to write, but it will
For some context, rayon-rs/rayon#671 reported that it is fairly easy to get parallel iterators to exceed the type length limit. I opened rayon-rs/rayon#673 to reduce genericity in a few specific cases, which seems to help. I decided it might be worthwhile to try this on the standard library too, if only so I could get some feedback on this approach.
I kind of wish the compiler could just figure this out, but that may be more difficult.
Have you had a chance to do any investigation to see if this benefit actually occurs, perhaps by looking at the generated LLVM IR or the resulting generated code? If so, consider adding a test for this to ensure that it isn't regressed in the future.
There are overflow tests for
I verified manually beforehand that such duplication was happening, and that they weren't afterward. And I know in rayon-rs/rayon#673 that similar changes did reduce the type length. But yes, I can and should add codegen tests specifically for these aspects.
The deduplication is now tested in codegen.
I had a hard time figuring out how to demonstrate the type length as-is, but I was able to do so with
It's interesting though, because even if I crank up
Ah, it failed on
// only-32bit too impatient for 2⁶⁴ items
Those are OK on 32-bit optimized builds, but too slow on
I've now forced optimizations in those specific tests.
Aug 15, 2019
added a commit
this pull request
Aug 15, 2019
Is there an issue that tracks any attempts to improve how closures are compiled, such that changes like these aren't necessary anymore? It's effective, but it's a lot of boilerplate code...
I'm also wondering whether a macro could give the same upsides without the visual clutter of turning all closures into functions. @cuviper Have you tried this, or thought about it?
@timvermeulen I don't have a good answer for either of those questions, and I agree it's a pain.
I suspect that improving how closures are compiled -- I guess inferring tighter generic parameters -- would be RFC territory. This would be sort of like RFC 2229 at the type level. If that's possible, I would be happy to see these changes go back to simpler closures again.
As for a macro, at least for
One saving grace is that I don't think we need to turn this PR into a general "avoid closures" advice for Rust developers. I think iterators are unusual in this respect, and especially iterator adaptors, to have complicated generic types but closures that only operate on a simpler item type. Parallel iterators in