Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add IntoIterator impl for arrays by value (for [T; N]) #65819

Conversation

LukasKalbertodt
Copy link
Member

@LukasKalbertodt LukasKalbertodt commented Oct 25, 2019

Current status (2020-12-30):
Const generics are stable now, so this PR is unblocked. The main remaining question is how to deal with the regressions caused by this change (see this comment). In the meantime, you can use IntoIterator::new which has been stabilized independently.

Status update comments:



Closes #25725

Initially part of #62959, this PR adds this impl:

impl<T, const N: usize> IntoIterator for [T; N]

TODO


The backwards compatibility problem

Adding this impl is not as straight-forward as it seems: there are some backwards compatibility hazards. In particular, due to autoref, this compiles today:

let array = [true; 3];
for x in array.into_iter() {
    let _ = *x; // x is a reference
}

With this change, that code wouldn't compile anymore, as x inside the loop would have the type bool and not &bool (like it does today). One should note that this only happens when using the .method call syntax with .into_iter(). It does not affect .iter() (different method) and it does not affect the for-loop syntax (does not involve autoref).

There has been some discussion in #49000 and in #62959. Most agree that a crater run would be very useful. That's what this PR is for. But the fact that this change didn't break anything in the compiler is already promising. (it did)

Arguments to add this impl despite the potential breakage:

  • It doesn't really make sense to call .into_iter(), as .iter() is shorter and the for loop desugars to into_iter() anyway. So hopefully no one used this in the real world. (people did use that in the real world)
  • RFC 1105 clearly specifies that "implementing any non-fundamental trait" is a "minor change". It also acknowledges that "implementing any existing trait can cause breakage", "However, as before, this kind of breakage is considered 'minor'".
  • @scottmcm wrote a comment that I (and apparently many others) completely agree with:

    My personal opinion: having this impl is so obviously the correct thing that I'd be willing to bend stability guarantees to have it, but we don't even need to because adding a new trait impl is an allowed change, no matter whether it breaks code. And the only code that it breaks, today, is code that was doing a.into_iter() when a.iter() would have done exactly the same thing for less typing, so any workaround needed to not trigger this change will make the code strictly better regardless.


CC @Centril @Mark-Simulacrum @cuviper

@rust-highfive
Copy link
Collaborator

@rust-highfive rust-highfive commented Oct 25, 2019

r? @KodrAus

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review label Oct 25, 2019
@jonas-schievink jonas-schievink added the relnotes label Oct 25, 2019
@jonas-schievink jonas-schievink added this to the 1.40 milestone Oct 25, 2019
@jonas-schievink
Copy link
Member

@jonas-schievink jonas-schievink commented Oct 25, 2019

Finally, if we want to add this impl, we need to decide if/what we do to limit the fallout of potential breakage, like adding a clippy lint or making this a built-in rustc warning.

The Clippy lint for this now exists, and I believe there was a crater run done with this impl too. We could also do another Crater run with this PR (the queue is empty right now).

@rust-highfive
Copy link
Collaborator

@rust-highfive rust-highfive commented Oct 25, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-10-25T16:08:39.0930075Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-10-25T16:08:39.1152691Z ##[command]git config gc.auto 0
2019-10-25T16:08:39.1236209Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-10-25T16:08:39.1301257Z ##[command]git config --get-all http.proxy
2019-10-25T16:08:39.1452944Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/65819/merge:refs/remotes/pull/65819/merge
---
2019-10-25T17:08:44.5564012Z .................................................................................................... 1600/9251
2019-10-25T17:08:50.2511175Z .................................................................................................... 1700/9251
2019-10-25T17:09:02.8269650Z ........................................................i...............i........................... 1800/9251
2019-10-25T17:09:10.4313493Z .................................................................................................... 1900/9251
2019-10-25T17:09:25.6518576Z ..............................................iiiii................................................. 2000/9251
2019-10-25T17:09:36.5052054Z .................................................................................................... 2200/9251
2019-10-25T17:09:39.0797158Z .................................................................................................... 2300/9251
2019-10-25T17:09:42.9956997Z .................................................................................................... 2400/9251
2019-10-25T17:10:06.1092397Z .................................................................................................... 2500/9251
---
2019-10-25T17:12:59.6538211Z .................................................i...............i.................................. 4800/9251
2019-10-25T17:13:08.7094825Z .................................................................................................... 4900/9251
2019-10-25T17:13:17.3151600Z .................................................................................................... 5000/9251
2019-10-25T17:13:23.6369622Z .................................................................................................... 5100/9251
2019-10-25T17:13:34.0180013Z ..................................................ii.ii...........i................................. 5200/9251
2019-10-25T17:13:43.8520642Z .................................................................................................... 5400/9251
2019-10-25T17:13:53.1465550Z .................................................................................................... 5500/9251
2019-10-25T17:14:01.0876503Z ....................i............................................................................... 5600/9251
2019-10-25T17:14:06.7785421Z .................................................................................................... 5700/9251
2019-10-25T17:14:06.7785421Z .................................................................................................... 5700/9251
2019-10-25T17:14:19.0161077Z .................................................................................................... 5800/9251
2019-10-25T17:14:30.1941961Z .................ii...i..ii...........i............................................................. 5900/9251
2019-10-25T17:14:52.0963618Z .................................................................................................... 6100/9251
2019-10-25T17:15:00.6647414Z .................................................................................................... 6200/9251
2019-10-25T17:15:00.6647414Z .................................................................................................... 6200/9251
2019-10-25T17:15:18.3466905Z ........................................i..ii....................................................... 6300/9251
2019-10-25T17:15:40.5878966Z .................................................................................................... 6500/9251
2019-10-25T17:15:42.7768238Z ......i............................................................................................. 6600/9251
2019-10-25T17:15:45.0184500Z .................................................................................i.................. 6700/9251
2019-10-25T17:15:47.7478050Z .................................................................................................... 6800/9251
---
2019-10-25T17:20:20.9768317Z  finished in 5.855
2019-10-25T17:20:20.9940752Z Check compiletest suite=codegen mode=codegen (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-10-25T17:20:21.1555520Z 
2019-10-25T17:20:21.1555748Z running 153 tests
2019-10-25T17:20:24.2934546Z i....iii......iii..iiii...i.............................i..i..................i....i...........ii.i. 100/153
2019-10-25T17:20:26.2404768Z .i.iiii..............i.........iii.i.........ii......
2019-10-25T17:20:26.2406705Z 
2019-10-25T17:20:26.2407929Z  finished in 5.246
2019-10-25T17:20:26.2647884Z Check compiletest suite=codegen-units mode=codegen-units (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-10-25T17:20:27.2336457Z 
---
2019-10-25T17:20:28.4669836Z  finished in 2.206
2019-10-25T17:20:28.4850648Z Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-10-25T17:20:28.6452250Z 
2019-10-25T17:20:28.6452690Z running 9 tests
2019-10-25T17:20:28.6453805Z iiiiiiiii
2019-10-25T17:20:28.6454336Z 
2019-10-25T17:20:28.6454374Z  finished in 0.160
2019-10-25T17:20:28.6647841Z Check compiletest suite=incremental mode=incremental (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-10-25T17:20:29.7344058Z 
---
2019-10-25T17:20:47.4021005Z  finished in 18.737
2019-10-25T17:20:47.4224274Z Check compiletest suite=debuginfo mode=debuginfo-gdb+lldb (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-10-25T17:20:47.5797524Z 
2019-10-25T17:20:47.5798270Z running 123 tests
2019-10-25T17:21:12.2987338Z .iiiii...i.....i..i...i..i.i.i..i.ii..i.i.....i..i....ii..........iiii..........i...ii...i.......ii. 100/123
2019-10-25T17:21:17.0330233Z i.i.i......iii.i.....ii
2019-10-25T17:21:17.0332830Z 
2019-10-25T17:21:17.0334358Z  finished in 29.611
2019-10-25T17:21:17.0346431Z Uplifting stage1 rustc (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-10-25T17:21:17.0347069Z Copying stage2 rustc from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
---
2019-10-25T17:33:35.5310013Z 
2019-10-25T17:33:35.5320316Z    Doc-tests core
2019-10-25T17:33:40.6045157Z 
2019-10-25T17:33:40.6046164Z running 2407 tests
2019-10-25T17:33:51.8372051Z ......iiiii......................................................................................... 100/2407
2019-10-25T17:34:02.8126534Z ................................................................................ii.................. 200/2407
2019-10-25T17:34:28.6200617Z ..i................................................................................................. 400/2407
2019-10-25T17:34:28.6200617Z ..i................................................................................................. 400/2407
2019-10-25T17:34:39.3266125Z .................................................i..i.................iiii.......................... 500/2407
2019-10-25T17:34:59.8491940Z .................................................................................................... 700/2407
2019-10-25T17:35:10.3956953Z .................................................................................................... 800/2407
2019-10-25T17:35:20.9406938Z .................................................................................................... 900/2407
2019-10-25T17:35:31.4092079Z .................................................................................................... 1000/2407
---
2019-10-25T17:39:34.5794224Z .................................................................................................... 500/763
2019-10-25T17:39:34.5802106Z ....................thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', src/libcore/result.rs:1165:5
2019-10-25T17:39:34.5802829Z ....thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', src/libcore/result.rs:1165:5
2019-10-25T17:39:34.5803328Z thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: "SendError(..)"', src/libcore/result.rs:1165:5
2019-10-25T17:39:34.5804201Z .......thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', src/libcore/result.rs:1165:5
2019-10-25T17:39:34.7625983Z ..........................................thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', src/libcore/result.rs:1165:5
2019-10-25T17:39:34.7656420Z ....thread '..<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', src/libcore/result.rs.:.1165:5
2019-10-25T17:39:34.7680978Z ...thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', src/libcore/result.rs:1165:5
2019-10-25T17:39:36.8198603Z ......................thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/mutex.rs:629:13
2019-10-25T17:39:36.8201490Z ....thread '<unnamed>' panicked at 'test panic in inner thread to poison mutex', src/libstd/sync/mutex.rs:584:13
2019-10-25T17:39:36.8211979Z .thread '<unnamed>' panicked at 'test panic in inner thread to poison mutex', src/libstd/sync/mutex.rs:561:13
2019-10-25T17:39:36.8220089Z .thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/mutex.rs:689:13
---
2019-10-25T17:39:46.1412044Z 
2019-10-25T17:39:46.1417111Z running 995 tests
2019-10-25T17:40:07.3540590Z i................................................................................................... 100/995
2019-10-25T17:40:19.3210688Z .................................................................................................... 200/995
2019-10-25T17:40:27.7094277Z ...................iii......i......i...i......i..................................................... 300/995
2019-10-25T17:40:33.4349727Z .................................................................................................... 400/995
2019-10-25T17:40:41.3556019Z ........................................i..i.................................ii..................... 500/995
2019-10-25T17:40:56.4501747Z .................................................................................................... 700/995
2019-10-25T17:40:56.4501747Z .................................................................................................... 700/995
2019-10-25T17:41:04.5584533Z .....................iiii........................................................................... 800/995
2019-10-25T17:41:20.1484174Z .................................................................................................... 900/995
2019-10-25T17:41:27.9344646Z ...........................................iiii................................................
2019-10-25T17:41:27.9345938Z 
2019-10-25T17:41:27.9390118Z  finished in 196.599
2019-10-25T17:41:27.9406845Z Testing term stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-10-25T17:41:28.1581321Z    Compiling term v0.0.0 (/checkout/src/libterm)
---
2019-10-25T17:41:30.3484127Z    Compiling test v0.0.0 (/checkout/src/libtest)
2019-10-25T17:41:30.9927882Z error[E0614]: type `types::TestType` cannot be dereferenced
2019-10-25T17:41:30.9928269Z    --> src/libtest/tests.rs:273:49
2019-10-25T17:41:30.9928501Z     |
2019-10-25T17:41:30.9928872Z 273 |         let result = time_test_failure_template(*test_type);
2019-10-25T17:41:30.9929271Z 
2019-10-25T17:41:30.9974967Z error[E0614]: type `types::TestType` cannot be dereferenced
2019-10-25T17:41:30.9975254Z    --> src/libtest/tests.rs:324:41
2019-10-25T17:41:30.9975478Z     |
2019-10-25T17:41:30.9975478Z     |
2019-10-25T17:41:30.9975738Z 324 |         let test_desc = typed_test_desc(*test_type);
2019-10-25T17:41:30.9976026Z 
2019-10-25T17:41:30.9976269Z error[E0614]: type `u128` cannot be dereferenced
2019-10-25T17:41:30.9976489Z    --> src/libtest/tests.rs:325:40
2019-10-25T17:41:30.9976671Z     |
2019-10-25T17:41:30.9976671Z     |
2019-10-25T17:41:30.9976944Z 325 |         let exec_time = test_exec_time(*time as u64);
2019-10-25T17:41:30.9977931Z 
2019-10-25T17:41:30.9978217Z error[E0614]: type `bool` cannot be dereferenced
2019-10-25T17:41:30.9978480Z    --> src/libtest/tests.rs:327:61
2019-10-25T17:41:30.9979150Z     |
2019-10-25T17:41:30.9979150Z     |
2019-10-25T17:41:30.9979506Z 327 |         assert_eq!(options.is_warn(&test_desc, &exec_time), *expected_warn);
2019-10-25T17:41:30.9979882Z 
2019-10-25T17:41:30.9985454Z error[E0614]: type `bool` cannot be dereferenced
2019-10-25T17:41:30.9985724Z    --> src/libtest/tests.rs:328:65
2019-10-25T17:41:30.9985945Z     |
2019-10-25T17:41:30.9985945Z     |
2019-10-25T17:41:30.9986386Z 328 |         assert_eq!(options.is_critical(&test_desc, &exec_time), *expected_critical);
2019-10-25T17:41:30.9986780Z 
2019-10-25T17:41:31.0347719Z error: aborting due to 5 previous errors
2019-10-25T17:41:31.0347955Z 
2019-10-25T17:41:31.0348284Z For more information about this error, try `rustc --explain E0614`.
---
2019-10-25T17:41:31.0670323Z   local time: Fri Oct 25 17:41:31 UTC 2019
2019-10-25T17:41:31.3586469Z   network time: Fri, 25 Oct 2019 17:41:31 GMT
2019-10-25T17:41:31.3586602Z == end clock drift check ==
2019-10-25T17:41:32.0118257Z 
2019-10-25T17:41:32.0223680Z ##[error]Bash exited with code '1'.
2019-10-25T17:41:32.0260040Z ##[section]Starting: Checkout
2019-10-25T17:41:32.0262235Z ==============================================================================
2019-10-25T17:41:32.0262286Z Task         : Get sources
2019-10-25T17:41:32.0262344Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@LukasKalbertodt
Copy link
Member Author

@LukasKalbertodt LukasKalbertodt commented Oct 25, 2019

And I thought I ran every test locally... Anyway, we now have an example of code that would break due to this change:

rust/src/libtest/tests.rs

Lines 311 to 329 in 23f890f

let test_vector = [
(TestType::UnitTest, unit.warn.as_millis() - 1, false, false),
(TestType::UnitTest, unit.warn.as_millis(), true, false),
(TestType::UnitTest, unit.critical.as_millis(), true, true),
(TestType::IntegrationTest, integration.warn.as_millis() - 1, false, false),
(TestType::IntegrationTest, integration.warn.as_millis(), true, false),
(TestType::IntegrationTest, integration.critical.as_millis(), true, true),
(TestType::DocTest, doc.warn.as_millis() - 1, false, false),
(TestType::DocTest, doc.warn.as_millis(), true, false),
(TestType::DocTest, doc.critical.as_millis(), true, true),
];
for (test_type, time, expected_warn, expected_critical) in test_vector.into_iter() {
let test_desc = typed_test_desc(*test_type);
let exec_time = test_exec_time(*time as u64);
assert_eq!(options.is_warn(&test_desc, &exec_time), *expected_warn);
assert_eq!(options.is_critical(&test_desc, &exec_time), *expected_critical);
}

This piece of code is interesting because we can guess how this code came to be: the variable is called test_vector, so it likely started out as Vec. But at some point the author changed it to a simple array (or they just forgot the vec!). Then some errors were printed by the compiler, which were fixed by adding a bunch of * dereference operations. But the author didn't notice the wrong/strange into_iter().

Now we have to see how often this exists in the real world.

@scottmcm
Copy link
Member

@scottmcm scottmcm commented Oct 25, 2019

Well, if the queue's empty let's get a check run to find things to go fix.

@bors try

@bors
Copy link
Contributor

@bors bors commented Oct 25, 2019

Trying commit d4d42a2 with merge b34fa46...

bors added a commit that referenced this issue Oct 25, 2019
…r=<try>

Add `IntoIterator` impl for arrays by value (`for [T; N]`)

Closes #25725

Initially part of #62959, this PR adds this impl:

```rust
impl<T, const N: usize> IntoIterator for [T; N]
where
    [T; N]: LengthAtMost32
```

Tests and some docs were adjusted. There are probably more places where docs need adjustment, but I postponed doing a proper search until we decided whether we want this. Which brings us to...

---

## The backwards compatibility problem

Adding this impl is not as straight-forward as it seems: there are some backwards compatibility hazards. In particular, due to autoref, this compiles today:

```rust
let array = [true; 3];
for x in array.into_iter() {
    let _ = *x; // x is a reference
}
```

With this change, that code wouldn't compile anymore, as `x` inside the loop would have the type `bool` and not `&bool` (like it does today). One should note that this only happens when using the `.method` call syntax with `.into_iter()`. It does not affect `.iter()` (different method) and it does not affect the `for`-loop syntax (does not involve autoref).

There has been some discussion in #49000 and in #62959. Most agree that a crater run would be very useful. That's what this PR is for. But the fact that this change didn't break anything in the compiler is already promising.

### Arguments to add this `impl` despite the potential breakage:
- It doesn't really make sense to call `.into_iter()`, as `.iter()` is shorter and the `for` loop desugars to `into_iter()` anyway. So hopefully no one used this in the real world.
- [RFC 1105](https://github.com/nox/rust-rfcs/blob/master/text/1105-api-evolution.md) clearly specifies that "implementing any non-fundamental trait" is a "minor change". It also acknowledges that "implementing any existing trait can cause breakage", "However, as before, this kind of breakage is considered 'minor'".
- @scottmcm wrote [a comment](#62959 (comment)) that I (and apparently many others) completely agree with:
  > My personal opinion: having this impl is so obviously the correct thing that I'd be willing to bend stability guarantees to have it, but we don't even need to because adding a new trait impl is an allowed change, no matter whether it breaks code. And the only code that it breaks, today, is code that was doing `a.into_iter()` when `a.iter()` would have done exactly the same thing for less typing, so any workaround needed to not trigger this change will make the code strictly better regardless.

---

Finally, *if* we want to add this impl, we need to decide if/what we do to limit the fallout of potential breakage, like adding a clippy lint or making this a built-in rustc warning.

CC @Centril @Mark-Simulacrum @cuviper
@bors
Copy link
Contributor

@bors bors commented Oct 26, 2019

☀️ Try build successful - checks-azure
Build commit: b34fa46 (b34fa46cc6fa5d46c6adcb041ebd28c7f7a374f3)

@varkor
Copy link
Member

@varkor varkor commented Oct 26, 2019

@craterbot check

@craterbot
Copy link
Collaborator

@craterbot craterbot commented Oct 26, 2019

👌 Experiment pr-65819 created and queued.
🤖 Automatically detected try build b34fa46
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-crater and removed S-waiting-on-review labels Oct 26, 2019
@varkor varkor added the F-const_generics label Oct 26, 2019
@craterbot
Copy link
Collaborator

@craterbot craterbot commented Oct 28, 2019

🚧 Experiment pr-65819 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot
Copy link
Collaborator

@craterbot craterbot commented Oct 30, 2019

🎉 Experiment pr-65819 is completed!
📊 5360 regressed and 1 fixed (74234 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the blacklist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-review and removed S-waiting-on-crater labels Oct 30, 2019
@scottmcm
Copy link
Member

@scottmcm scottmcm commented Oct 30, 2019

Lots of regressions, though as a positive clippy is catching at least the first one I saw multiple times (such as https://crater-reports.s3.amazonaws.com/pr-65819/try%23b34fa46cc6fa5d46c6adcb041ebd28c7f7a374f3/reg/cargo-bundle-0.4.0/log.txt):

error: this .into_iter() call is equivalent to .iter() and will not move the array
   --> src\decoder.rs:235:47
    |
235 |                     for (i, &table) in tables.into_iter().enumerate() {
    |                                               ^^^^^^^^^ help: call directly: `iter`
    |
    = note: `#[deny(clippy::into_iter_on_array)]` on by default
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#into_iter_on_array

https://github.com/kaksmet/jpeg-decoder/blob/c8e25252/src/decoder.rs#L235
And submitted image-rs/jpeg-decoder#116 to fix it.

@hellow554
Copy link
Contributor

@hellow554 hellow554 commented Oct 30, 2019

Another issue one can see a lot: (e.g. here)

error[E0308]: mismatched types
  --> /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/lazy_static-1.1.0/build.rs:13:19
   |
13 |         .filter(|&&f| f)
   |                   ^^
   |                   |
   |                   expected bool, found reference
   |                   help: you can probably remove the explicit borrow: `f`
   |
   = note: expected type `bool`
              found type `&_`

error: aborting due to previous error

tesuji added a commit to tesuji/ring that referenced this issue Oct 30, 2019
See clippy::into_iter lint and rust-lang/rust#65819
@nikomatsakis
Copy link
Contributor

@nikomatsakis nikomatsakis commented Apr 13, 2021

What about upgrading to the new edition? Would cargo fix be able to upgrade this automatically so that the code still compiled with the same behavior?

We've already implemented the idea, but indeed we did not handle migration. I don't see that being a big issue though. One simple rewrite would be to I think the rewrite would be to add .as_slice(). I'll mention it on the PR (#84147)

EDIT: Never mind, the PR already handles this by rewriting into_iter() to iter()

@cuviper
Copy link
Member

@cuviper cuviper commented Apr 13, 2021

The preferred fix in most cases is to change arr.into_iter() to arr.iter(), but that's a different type so it may not always work.

The existing lint already offers that fix. I don't know what you mean about different types, because it returns slice::Iter either way.

@digama0
Copy link
Contributor

@digama0 digama0 commented Apr 14, 2021

Oh that's great then. The only case where a blind transformation on eligible into_iter() calls to iter() would get in trouble would be when the receiver has type T with an inherent iter() method and a deref to [U; N], but this seems pretty unlikely to occur in practice.

@craterbot
Copy link
Collaborator

@craterbot craterbot commented Apr 16, 2021

🚧 Experiment pr-65819 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@xfix
Copy link
Contributor

@xfix xfix commented Apr 16, 2021

Oh that's great then. The only case where a blind transformation on eligible into_iter() calls to iter() would get in trouble would be when the receiver has type T with an inherent iter() method and a deref to [U; N], but this seems pretty unlikely to occur in practice.

I'm not sure how could that be an issue considering Deref works on references, which means that into_iter called on a structure that implements Deref to an array type will try to call <&[U; N]>::into_iter first even if <[U; N]>::into_iter exists, avoiding the compatibility issue entirely. Because this isn't one of cases that will be affected by this change, it won't be incorrectly fixed by cargo fix.

@cuviper
Copy link
Member

@cuviper cuviper commented Apr 16, 2021

@xfix

Deref works on references, which means that into_iter called on a structure that implements Deref to an array type will try to call <&[U; N]>::into_iter first even if <[U; N]>::into_iter exists, avoiding the compatibility issue entirely.

You're in good company to think it works like that (#84147 (comment)), but no, method resolution tries to use the value behind the dereference first, as this playground shows. I don't think this needs to block #84147, but we could file an issue for 2021 to expand the scope of the array_into_iter lint and its fix.

@cuviper
Copy link
Member

@cuviper cuviper commented Apr 16, 2021

BTW @LukasKalbertodt I did cherry-pick two of your commits from this PR into #84147, but not:

  • e6bccc9 Remove array_into_iter lint
    • The lint is still needed for 2015/2018 editions, especially as a migration fix
  • 30f0ded Use the IntoIterator array impl in several places across library/*
    • We can't use array.into_iter() to get values yet, at least until library/ moves to 2021
    • Some of those changes would be fine though, like using arrays for implicit IntoIterator args

@digama0
Copy link
Contributor

@digama0 digama0 commented Apr 17, 2021

@xfix Here's an example of the condition I was talking about (playground). Pretend that Array<T, N> is the actual type [T; N] here. We've wrapped an array with a type Foo that has a deref to [u8; 12] and an inherent Foo::iter method. The lint would trigger here because we're calling <&[u8; 12]>::into_iter(), but the normal fix of changing .into_iter() into .iter() doesn't work because method autoref will cause it to call Foo::iter() instead (which has the wrong type). In order to get the right iter we need to write (*foo).iter() instead. (And in general that might have to be (***foo).iter() or whatever to avoid inherent impls on any of the intervening types.)

@craterbot
Copy link
Collaborator

@craterbot craterbot commented Apr 19, 2021

🎉 Experiment pr-65819 is completed!
📊 4688 regressed and 11 fixed (153667 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the blacklist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-review and removed S-waiting-on-crater labels Apr 19, 2021
@LukasKalbertodt
Copy link
Member Author

@LukasKalbertodt LukasKalbertodt commented Apr 22, 2021

I didn't have a lot of time to analyze this latest report, but from the looks of it, not a ton has changed compared to the previous run.

And just like the report from August 2020, it looks like the vast majority of regressions is caused by a few regressed dependencies. Almost all of which have already released a fix, usually as a minor version bump. Thus, most of these broken projects are just one cargo update away from working again. For more information, see this comment.

What this shows, I think, is that there are just a lot of projects that are not actively maintained anymore. E.g. one-time hobby projects. We cannot expect those to ever get updated and it would be a shame if those projects wouldn't compile anymore in the future. That's why I'm really glad we have #84147 now which avoids this breakage altogether.

I think it's a bit unfortunate that this kind of method resolution trick was only properly considered/investigated a few weeks ago. If we had known about this earlier (and how comparatively easy it is to implement), this could have saved quite a bit of work. But overall I'm just glad we have this solution now so that we can have the impl without breaking the world. So thanks everyone involved in that!

Finally, I guess I can close this PR now? (After 1.5 years :D)
#84147 cherry picked all relevant commits and will likely be merged soon.

bors added a commit to rust-lang-ci/rust that referenced this issue Apr 25, 2021
…matsakis,m-ou-se

Cautiously add IntoIterator for arrays by value

Add the attribute described in rust-lang#84133, `#[rustc_skip_array_during_method_dispatch]`, which effectively hides a trait from method dispatch when the receiver type is an array.

Then cherry-pick `IntoIterator for [T; N]` from rust-lang#65819 and gate it with that attribute. Arrays can now be used as `IntoIterator` normally, but `array.into_iter()` has edition-dependent behavior, returning `slice::Iter` for 2015 and 2018 editions, or `array::IntoIter` for 2021 and later.

r? `@nikomatsakis`
cc `@LukasKalbertodt` `@rust-lang/libs`
bors added a commit to rust-lang-ci/rust that referenced this issue Apr 25, 2021
…matsakis,m-ou-se

Cautiously add IntoIterator for arrays by value

Add the attribute described in rust-lang#84133, `#[rustc_skip_array_during_method_dispatch]`, which effectively hides a trait from method dispatch when the receiver type is an array.

Then cherry-pick `IntoIterator for [T; N]` from rust-lang#65819 and gate it with that attribute. Arrays can now be used as `IntoIterator` normally, but `array.into_iter()` has edition-dependent behavior, returning `slice::Iter` for 2015 and 2018 editions, or `array::IntoIter` for 2021 and later.

r? `@nikomatsakis`
cc `@LukasKalbertodt` `@rust-lang/libs`
@m-ou-se
Copy link
Member

@m-ou-se m-ou-se commented Apr 26, 2021

@LukasKalbertodt Thanks a lot for all the work and effort you put into this! Thanks to your work, such as the lint you added, the analysis of the crater runs, the implementation of array::IntoIter, and tirelessly continuing to push this forward all this time, we now have a stable IntoIterator for [T; N] implementation in Rust 1.53. 🎉 🎉 🎉

m-ou-se added a commit to m-ou-se/rust that referenced this issue Apr 28, 2021
Point out that behavior might be switched on 2015 and 2018 too one day

Reword documentation to make it clear that behaviour can be switched on older editions too, one day in the future. It doesn't *have* to be switched, but I think it's good to have it as an option and re-evaluate it a few months/years down the line when e.g. the crates that showed up in crater were broken by different changes in the language already.

cc rust-lang#25725, rust-lang#65819, rust-lang#66145, rust-lang#84147 , and rust-lang#84133 (comment)
jackh726 added a commit to jackh726/rust that referenced this issue Apr 28, 2021
Point out that behavior might be switched on 2015 and 2018 too one day

Reword documentation to make it clear that behaviour can be switched on older editions too, one day in the future. It doesn't *have* to be switched, but I think it's good to have it as an option and re-evaluate it a few months/years down the line when e.g. the crates that showed up in crater were broken by different changes in the language already.

cc rust-lang#25725, rust-lang#65819, rust-lang#66145, rust-lang#84147 , and rust-lang#84133 (comment)
jackh726 added a commit to jackh726/rust that referenced this issue Apr 29, 2021
Point out that behavior might be switched on 2015 and 2018 too one day

Reword documentation to make it clear that behaviour can be switched on older editions too, one day in the future. It doesn't *have* to be switched, but I think it's good to have it as an option and re-evaluate it a few months/years down the line when e.g. the crates that showed up in crater were broken by different changes in the language already.

cc rust-lang#25725, rust-lang#65819, rust-lang#66145, rust-lang#84147 , and rust-lang#84133 (comment)
jackh726 added a commit to jackh726/rust that referenced this issue Apr 29, 2021
Point out that behavior might be switched on 2015 and 2018 too one day

Reword documentation to make it clear that behaviour can be switched on older editions too, one day in the future. It doesn't *have* to be switched, but I think it's good to have it as an option and re-evaluate it a few months/years down the line when e.g. the crates that showed up in crater were broken by different changes in the language already.

cc rust-lang#25725, rust-lang#65819, rust-lang#66145, rust-lang#84147 , and rust-lang#84133 (comment)
jackh726 added a commit to jackh726/rust that referenced this issue Apr 29, 2021
Point out that behavior might be switched on 2015 and 2018 too one day

Reword documentation to make it clear that behaviour can be switched on older editions too, one day in the future. It doesn't *have* to be switched, but I think it's good to have it as an option and re-evaluate it a few months/years down the line when e.g. the crates that showed up in crater were broken by different changes in the language already.

cc rust-lang#25725, rust-lang#65819, rust-lang#66145, rust-lang#84147 , and rust-lang#84133 (comment)
jackh726 added a commit to jackh726/rust that referenced this issue Apr 29, 2021
Point out that behavior might be switched on 2015 and 2018 too one day

Reword documentation to make it clear that behaviour can be switched on older editions too, one day in the future. It doesn't *have* to be switched, but I think it's good to have it as an option and re-evaluate it a few months/years down the line when e.g. the crates that showed up in crater were broken by different changes in the language already.

cc rust-lang#25725, rust-lang#65819, rust-lang#66145, rust-lang#84147 , and rust-lang#84133 (comment)
jackh726 added a commit to jackh726/rust that referenced this issue Apr 29, 2021
Point out that behavior might be switched on 2015 and 2018 too one day

Reword documentation to make it clear that behaviour can be switched on older editions too, one day in the future. It doesn't *have* to be switched, but I think it's good to have it as an option and re-evaluate it a few months/years down the line when e.g. the crates that showed up in crater were broken by different changes in the language already.

cc rust-lang#25725, rust-lang#65819, rust-lang#66145, rust-lang#84147 , and rust-lang#84133 (comment)
@mimoo
Copy link

@mimoo mimoo commented Oct 20, 2021

It seems like there are slight differences between this and the previous std::array::IntoIter (which doesn't show signs of deprecation btw).

This will work:

use itertools::Itertools; // 0.10.1

fn main() {
    let a = [1u8, 2, 3];
    let b: (u64, u64, u64) = std::array::IntoIter::new(a).map(Into::into).next_tuple().unwrap();
}

But this won't:

use itertools::Itertools; // 0.10.1

fn main() {
    let a = [1u8, 2, 3];
    let b: (u64, u64, u64) = a.into_iter().map(Into::into).next_tuple().unwrap();
}

With error:

the trait From<&u8> is not implemented for u64

it looks like into_iter() behaves like iter() in this case.

@jhpratt
Copy link
Contributor

@jhpratt jhpratt commented Oct 20, 2021

@mimoo That is very much intentional to avoid significant breakage. It will behave the way you're expecting in Rust 2021, which will be released tomorrow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
F-const_generics needs-fcp S-waiting-on-review T-libs-api
Projects
None yet
Development

Successfully merging this pull request may close these issues.