Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Rc<str> and Rc<[T]> instead of Rc<String> and Rc<Vec<T>> #49645

Closed
wants to merge 3 commits into from

Conversation

birkenfeld
Copy link
Contributor

@birkenfeld birkenfeld commented Apr 4, 2018

to save a bit of memory and pointer chasing.

For the Vec<T> case, there is one instance each of get_mut and make_mut that preclude the use of the slice type:

  • the get_mut one does push, so I left it as is
  • the make_mut one does a reverse, which would be fine but Lrc<[T]> does not impl Clone (should it?) - however, the reverse seems to be unnecessary - please check if the solution makes sense

Had some unrelated-looking local test failures, let's see if if CI gets them too.

@kennytm
Copy link
Member

kennytm commented Apr 4, 2018

@bors try

I'd like to check the perf. cc @Mark-Simulacrum

@bors
Copy link
Contributor

bors commented Apr 4, 2018

⌛ Trying commit 639abc9 with merge 7ee153e...

bors added a commit that referenced this pull request Apr 4, 2018
Use Rc<str> and Rc<[T]> instead of Rc<String> and Rc<Vec<T>>

to save a bit of memory and pointer chasing.

For the `Vec<T>` case, there is one instance each of `get_mut` and `make_mut` that preclude the use of the slice type:

* the `get_mut` one does `push`, so I left it as is
* the `make_mut` one does a `reverse`, which would be fine but `Lrc<[T]>` does not impl `Clone` (should it?) - however, the reverse seems to be unnecessary - please if the solution makes sense

Had some unrelated-looking local test failures, let's see if if CI gets them too.
@TimNN
Copy link
Contributor

TimNN commented Apr 4, 2018

Your PR failed on Travis. Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
Resolving deltas: 100% (611482/611482), completed with 4870 local objects.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN.

@birkenfeld
Copy link
Contributor Author

birkenfeld commented Apr 4, 2018

@TimNN Questionable fragment ;) Fixing the test now...

@@ -2341,14 +2341,14 @@ impl<'a, K, V> OccupiedEntry<'a, K, V> {
/// use std::collections::hash_map::{Entry, HashMap};
/// use std::rc::Rc;
///
/// let mut map: HashMap<Rc<String>, u32> = HashMap::new();
/// map.insert(Rc::new("Stringthing".to_string()), 15);
/// let mut map: HashMap<Rc<str>, u32> = HashMap::new();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why these doc changes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we shouldn't promote use of Rc<String>, rather show that Rc<str> is now feasible and preferable.

@TimNN
Copy link
Contributor

TimNN commented Apr 4, 2018

Your PR failed on Travis. Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[00:00:44] configure: rust.quiet-tests     := True
---
[00:44:51] ..........................................................................i.........................
[00:44:56] .................i..................................................................................
---
[00:45:31] ............................................................................................i.......
[00:45:38] ................................................................i...................................
---
[00:46:32] .............................................i......................................................
---
[00:50:22] .............................i......................................................................
[00:50:36] ..............................................................i.....................................
[00:50:52] ...............................................i....................................................
[00:51:11] ....................................................................................................
[00:51:33] ....................................................................................................
[00:51:54] ....................................................................................................
[00:52:19] .i...............................................................................................i..
[00:52:46] ....................................................................................test [run-pass] run-pass/mir_heavy_promoted.rs has been running for over 60 seconds
[00:52:55] ................
[00:53:25] ....................................................................................................
[00:54:02] .............................................................ii.....................................
[00:54:45] ........................i....................................................i.ii.....test [run-pass] run-pass/saturating-float-casts.rs has been running for over 60 seconds
[00:54:50] ..............
[00:55:30] .....................................................................................iiiiiii........
---
[00:57:30] ....................................i...............................................................
[00:57:38] ....................................................................................................
[00:57:45] ..................i............................................................ii.iii...............
[00:57:52] ....................................................................................................
[00:58:00] ........i..............................i............................................................
[00:58:08] ....................................................................................................
[00:58:14] ....................i...............................................................................
[00:58:22] ....................................................................................................
[00:58:32] ....................................................................................................
[00:58:42] ....................................................................................................
[00:58:53] ....................................................................................................
[00:59:06] ....................................................................................................
[00:59:16] .............i......................................................................................
[00:59:26] ................i..ii...............................................................................
[00:59:36] ....................................................................................................
[00:59:45] ....................................................................................................
[00:59:55] ..................................................................................i.................
[01:00:05] ............................i.......................................................................
---
[01:00:41] ............................i.......................................................................
[01:00:43] ....................................................................i...............................
[01:00:44] ................i.......................................................
---
[01:00:58] ...........i........................
---
[01:01:28] i...i..ii....i.............ii........iii......i..i...i...ii..i..i..ii.....
---
[01:01:31] i.......i......................i......
---
[01:02:08] iiii.......i..i........i..i.i.............i..........iiii...........i...i..........ii.i.i.......ii..
[01:02:09] ....ii...
---
[01:04:05] ...........................F................................................test [run-pass] run-pass-fulldeps/myriad-closures.rs has been running for over 60 seconds
---
[01:06:46] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/run-pass-fulldeps/issue-35829.rs" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps" "--target=x86_64-unknown-linux-gnu" "-C" "prefer-dynamic" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps/issue-35829.stage2-x86_64-unknown-linux-gnu" "-Crpath" "-O" "-Zmiri" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps/issue-35829.stage2-x86_64-unknown-linux-gnu.aux"
---
[01:06:46] 40 |     let byte_string_lit_kind = LitKind::ByteStr(Lrc::from(b"one"));
[01:06:46]    |                                                 ^^^^^^^^^^^^^^^^^ expected slice, found &[u8; 3]
[01:06:46]    |
[01:06:46]    = note: expected type `std::rc::Rc<[u8]>`
[01:06:46]               found type `std::rc::Rc<&[u8; 3]>`
[01:06:46]
[01:06:46] error[E0308]: mismatched types
[01:06:46]   --> /checkout/src/test/run-pass-fulldeps/issue-35829.rs:45:53
[01:06:46]    |
[01:06:46] 45 |     let raw_byte_string_lit_kind = LitKind::ByteStr(Lrc::from(b"#\"two\"#"));
[01:06:46]    |                                                     ^^^^^^^^^^^^^^^^^^^^^^^ expected slice, found &[u8; 7]
[01:06:46]    |
[01:06:46]    = note: expected type `std::rc::Rc<[u8]>`
[01:06:46]               found type `std::rc::Rc<&[u8; 7]>`
---
[01:06:46] thread '[run-pass] run-pass-fulldeps/issue-35829.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2901:9
---
[01:06:46] thread 'main' panicked at 'Some tests failed', tools/compiletest/src/main.rs:478:22
[01:06:46] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/run-pass-fulldeps" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "run-pass" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-3.9/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Zmiri -Zunstable-options " "--target-rustcflags" "-Crpath -O -Zmiri -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "3.9.1\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
[01:06:46] expected success, got: exit code: 101
[01:06:46]
[01:06:46]
[01:06:46] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[01:06:46] Build completed unsuccessfully in 0:22:40
[01:06:46] Makefile:58: recipe for target 'check' failed
[01:06:46] make: *** [check] Error 1
---
$ ls -lat $HOME/Library/Logs/DiagnosticReports/
ls: cannot access /home/travis/Library/Logs/DiagnosticReports/: No such file or directory
travis_time:end:0937c08a:start=1522848293114439242,finish=1522848293122767658,duration=8328416
travis_fold:end:after_failure.2
travis_fold:start:after_failure.3
travis_time:start:06d03e2c
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
find: `/home/travis/Library/Logs/DiagnosticReports': No such file or directory
travis_time:end:06d03e2c:start=1522848293130930559,finish=1522848293138896969,duration=7966410
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:07c8b8bc
$ dmesg | grep -i kill
[   10.789901] init: failsafe main process (1095) killed by TERM signal

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN.

@kennytm
Copy link
Member

kennytm commented Apr 4, 2018

The try build is successful. Again, the reply is missing, probably because new commits are pushed in between. Anyway, please schedule 7ee153eb4c4a2421b4eb1475248cec535fd981a9 (try) vs 5758c2dd14fd29caf7c7bb2123eb6b23443b9233 (master) @Mark-Simulacrum

@TimNN
Copy link
Contributor

TimNN commented Apr 4, 2018

Your PR failed on Travis. Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[00:00:48] configure: rust.quiet-tests     := True
---
[00:40:32] ..........................................................................i.........................
[00:40:38] .................i..................................................................................
---
[00:41:13] ............................................................................................i.......
[00:41:20] ................................................................i...................................
---
[00:42:14] .............................................i......................................................
---
[00:46:05] .............................i......................................................................
[00:46:19] ..............................................................i.....................................
[00:46:35] ...............................................i....................................................
[00:46:55] ....................................................................................................
[00:47:17] ....................................................................................................
[00:47:38] ....................................................................................................
[00:48:04] .i...............................................................................................i..
[00:48:30] .................................................................................test [run-pass] run-pass/mir_heavy_promoted.rs has been running for over 60 seconds
[00:48:40] ...................
[00:49:10] ....................................................................................................
[00:49:46] .............................................................ii.....................................
[00:50:31] ........................i....................................................i.ii.....test [run-pass] run-pass/saturating-float-casts.rs has been running for over 60 seconds
[00:50:36] ..............
[00:51:16] .....................................................................................iiiiiii........
---
[00:53:15] ....................................i...............................................................
[00:53:23] ....................................................................................................
[00:53:30] ..................i............................................................ii.iii...............
[00:53:38] ....................................................................................................
[00:53:45] ........i..............................i............................................................
[00:53:53] ....................................................................................................
[00:54:00] ....................i...............................................................................
[00:54:08] ....................................................................................................
[00:54:18] ....................................................................................................
[00:54:29] ....................................................................................................
[00:54:39] ....................................................................................................
[00:54:53] ....................................................................................................
[00:55:02] .............i......................................................................................
[00:55:12] ................i..ii...............................................................................
[00:55:22] ....................................................................................................
[00:55:32] ....................................................................................................
[00:55:42] ..................................................................................i.................
[00:55:53] ............................i.......................................................................
---
[00:56:30] ...........................i........................................................................
[00:56:32] ....................................................................i...............................
[00:56:33] ................i.......................................................
---
[00:56:48] ...........i........................
---
[00:57:17] i...i..ii....i.............ii........iii......i..i...i...ii..i..i..ii.....
---
[00:57:20] i.......i......................i......
---
[00:57:59] iiii.......i..i........i..i.i.............i..........iiii...........i...i..........ii.i.i.......ii..
[00:58:00] ....ii...
---
[00:59:56] ..........................F...............................................test [run-pass] run-pass-fulldeps/myriad-closures.rs has been running for over 60 seconds
---
[01:02:31] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/run-pass-fulldeps/issue-35829.rs" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps" "--target=x86_64-unknown-linux-gnu" "-C" "prefer-dynamic" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps/issue-35829.stage2-x86_64-unknown-linux-gnu" "-Crpath" "-O" "-Zmiri" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps/issue-35829.stage2-x86_64-unknown-linux-gnu.aux"
---
[01:02:31] error[E0277]: the trait bound `[u8]: std::marker::Sized` is not satisfied
[01:02:31]   --> /checkout/src/test/run-pass-fulldeps/issue-35829.rs:40:49
[01:02:31]    |
[01:02:31] 40 |     let byte_string_lit_kind = LitKind::ByteStr(Lrc::from(b"one"[..]));
[01:02:31]    |                                                 ^^^^^^^^^ `[u8]` does not have a constant size known at compile-time
[01:02:31]    |
[01:02:31]    = help: the trait `std::marker::Sized` is not implemented for `[u8]`
[01:02:31]    = note: required by `std::convert::From::from`
[01:02:31]
[01:02:31] error[E0277]: the trait bound `[u8]: std::marker::Sized` is not satisfied
[01:02:31]   --> /checkout/src/test/run-pass-fulldeps/issue-35829.rs:45:53
[01:02:31]    |
[01:02:31] 45 |     let raw_byte_string_lit_kind = LitKind::ByteStr(Lrc::from(b"#\"two\"#"[..]));
[01:02:31]    |                                                     ^^^^^^^^^ `[u8]` does not have a constant size known at compile-time
[01:02:31]    |
[01:02:31]    = help: the trait `std::marker::Sized` is not implemented for `[u8]`
[01:02:31]    = note: required by `std::convert::From::from`
---
[01:02:31] thread '[run-pass] run-pass-fulldeps/issue-35829.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2901:9
---
[01:02:31] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/run-pass-fulldeps" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass-fulldeps" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "run-pass" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-3.9/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Zmiri -Zunstable-options " "--target-rustcflags" "-Crpath -O -Zmiri -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "3.9.1\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
[01:02:31] expected success, got: exit code: 101
[01:02:31]
[01:02:31]
[01:02:31] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[01:02:31] Build completed unsuccessfully in 0:22:46
[01:02:31] make: *** [check] Error 1
[01:02:31] Makefile:58: recipe for target 'check' failed
---
$ ls -lat $HOME/Library/Logs/DiagnosticReports/
ls: cannot access /home/travis/Library/Logs/DiagnosticReports/: No such file or directory
travis_time:end:0f6924b2:start=1522856025478968573,finish=1522856025491582484,duration=12613911
travis_fold:end:after_failure.2
travis_fold:start:after_failure.3
travis_time:start:09afa380
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
find: `/home/travis/Library/Logs/DiagnosticReports': No such file or directory
travis_time:end:09afa380:start=1522856025500653423,finish=1522856025506840925,duration=6187502
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:005f66be
$ dmesg | grep -i kill
[   10.398145] init: failsafe main process (1095) killed by TERM signal

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN.

@pietroalbini pietroalbini added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Apr 4, 2018
@Mark-Simulacrum
Copy link
Member

Try build queued for perf.

Copy link
Contributor

@nikomatsakis nikomatsakis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so r=me for the compiler stuff, but I don't feel qualified to r+ the changes to the libstd docs

@nikomatsakis
Copy link
Contributor

r? @steveklabnik for the changes to hashmap example etc

@kennytm kennytm added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Apr 4, 2018
@kennytm
Copy link
Member

kennytm commented Apr 5, 2018

Perf result: https://perf.rust-lang.org/compare.html?start=5758c2dd14fd29caf7c7bb2123eb6b23443b9233&end=7ee153eb4c4a2421b4eb1475248cec535fd981a9&stat=instructions%3Au

Ignoring the spurious style-servo (clean incremental) test, there is almost no change in performance. The max-rss fluctuates between ±2% which seems within statistical noise.

@bors
Copy link
Contributor

bors commented Apr 5, 2018

☔ The latest upstream changes (presumably #49696) made this pull request unmergeable. Please resolve the merge conflicts.

There is one instace where Lrc::get_mut().unwrap() is used on
the reference to make a push, this cannot be converted.

One instance of Lrc::make_mut() was removed -- the side effect of
changing the original vector cannot have mattered since make_mut()
might have cloned it.
@birkenfeld
Copy link
Contributor Author

Updated.

@bors
Copy link
Contributor

bors commented Apr 7, 2018

☔ The latest upstream changes (presumably #49672) made this pull request unmergeable. Please resolve the merge conflicts.

@nikomatsakis
Copy link
Contributor

nikomatsakis commented Apr 10, 2018

Hmm. I'm torn on this change.

I wouldn't expect a real performance win -- as we saw on perf, this is basically in the noise, which is what I would expect. (Note also that even when judged from a "micro-efficiency" perspective, this is not an obvious win: Lrc<[T]> is a fat pointer, so it takes more space to store, though it does save on a dereference to access.)

My bigger concern though is that constructing a Lrc<Vec<..>> is easy: you do Lrc::new(..). Constructing a Lrc<[]> is also easy but less obvious (Lrc::from, it turns out, but I wouldn't have guessed that this will work). So this seems to make the functioning of the system just a bit more opaque.

Note all that we don't really get much increased flexibility here. e.g.,, given a Lrc<String>, it's not like we can make multiple derived Lrc values for different subsets of the string, right? We still only get a Lrc<str>.

So in summary:

  • More or less a wash from an efficient point-of-view
  • Arguably more elegant
  • But arguably less obvious

cc @rust-lang/compiler -- thoughts?

@oli-obk
Copy link
Contributor

oli-obk commented Apr 10, 2018

👍 for elegance

@eddyb
Copy link
Member

eddyb commented Apr 10, 2018

Note all that we don't really get much increased flexibility here. e.g.,, given a Lrc<String>, it's not like we can make multiple derived Lrc values for different subsets of the string, right?

Yeah - you need owning_ref for that (which adds another pointer).

@kennytm
Copy link
Member

kennytm commented Apr 10, 2018

Actually I would have expected a performance loss because constructing a Rc<[T]> from Vec<T> involves copying the entire array, while creating a Rc<Vec<T>> only needs to move 3 pointers. The perf result shows nothing is impacted, so I'm +0 on merging this.

@Zoxc
Copy link
Contributor

Zoxc commented Apr 10, 2018

I've been wanting to get rid of the Lrcs in queries. The way to achieve that would be to allocate results in arenas, so instead of Lrc<T>, queries would use &'tcx T. This would mean we would need a special way to construct query results anyway instead of just using Lrc::new.

@pietroalbini
Copy link
Member

Ping from triage! What's the status of this PR?

1 similar comment
@pietroalbini
Copy link
Member

Ping from triage! What's the status of this PR?

@birkenfeld
Copy link
Contributor Author

birkenfeld commented Apr 23, 2018

Looks like the change is not unanimous, and a significant part is likely to be refactored away, so I'll close it.

@birkenfeld birkenfeld closed this Apr 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet