Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize copying large ranges of undefmask blocks #58556

Merged
merged 8 commits into from Mar 15, 2019

Conversation

Projects
None yet
9 participants
@oli-obk
Copy link
Contributor

oli-obk commented Feb 18, 2019

Hopefully fixes #58523

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

rust-highfive commented Feb 18, 2019

r? @pnkfelix

(rust_highfive has picked a reviewer for you, use r? to override)

@oli-obk

This comment has been minimized.

Copy link
Contributor Author

oli-obk commented Feb 18, 2019

@bors try

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Feb 18, 2019

⌛️ Trying commit e30f55f with merge ab1e694...

bors added a commit that referenced this pull request Feb 18, 2019

Auto merge of #58556 - oli-obk:imperative_recursion, r=<try>
Optimize copying large ranges of undefmask blocks

Hopefully fixes #58523
@oli-obk

This comment has been minimized.

Copy link
Contributor Author

oli-obk commented Feb 18, 2019

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Feb 18, 2019

Success: Queued ab1e694 with parent d215d95, comparison URL.

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Feb 18, 2019

☀️ Test successful - checks-travis
State: approved= try=True

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Feb 18, 2019

Finished benchmarking try commit ab1e694

@oli-obk

This comment has been minimized.

Copy link
Contributor Author

oli-obk commented Feb 19, 2019

Local tests have shown that my stage 1 compiler needs 8 seconds for

fn main() {
    (&[0; 1 << 28]);
}

while my stage 0 compiler needs 23 seconds.

I'm not sure how to add such a test to the perf test suite without causing significant slowdown of the entire suite.

For smaller arrays (1 << 24) it's not as pronounced, but it's still less than one second consistently with stage 1 and around 2 seconds for stage 0.

@oli-obk

This comment has been minimized.

Copy link
Contributor Author

oli-obk commented Feb 20, 2019

@bors try

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Feb 20, 2019

⌛️ Trying commit 5aacd4c with merge 5f39473...

bors added a commit that referenced this pull request Feb 20, 2019

Auto merge of #58556 - oli-obk:imperative_recursion, r=<try>
Optimize copying large ranges of undefmask blocks

Hopefully fixes #58523
@oli-obk

This comment has been minimized.

Copy link
Contributor Author

oli-obk commented Feb 20, 2019

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Feb 20, 2019

Success: Queued 5f39473 with parent f66e469, comparison URL.

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Feb 20, 2019

☀️ Test successful - checks-travis
State: approved= try=True

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Feb 20, 2019

Finished benchmarking try commit 5f39473

@oli-obk

This comment has been minimized.

Copy link
Contributor Author

oli-obk commented Feb 21, 2019

perf shows a small (less than 1.5%) but across the board improvement

Once rust-lang-nursery/rustc-perf#349 is merged, we should see a big improvement, but I don't see a reason to wait for that.

@joshlf

This comment has been minimized.

Copy link
Contributor

joshlf commented Mar 4, 2019

With rust-lang-nursery/rustc-perf#349 merged, is this ready to move forward?

@oli-obk

This comment has been minimized.

Copy link
Contributor Author

oli-obk commented Mar 4, 2019

@bors try

@oli-obk

This comment has been minimized.

Copy link
Contributor Author

oli-obk commented Mar 4, 2019

@bors ping

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Mar 4, 2019

😪 I'm awake I'm awake

@oli-obk

This comment has been minimized.

Copy link
Contributor Author

oli-obk commented Mar 4, 2019

@bors try

@oli-obk

This comment has been minimized.

Copy link
Contributor Author

oli-obk commented Mar 4, 2019

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Mar 4, 2019

Success: Queued af4ce58 with parent a9da8fc, comparison URL.

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Mar 4, 2019

☀️ Try build successful - checks-travis
Build commit: af4ce58

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Mar 7, 2019

Finished benchmarking try commit af4ce58

@oli-obk

This comment has been minimized.

Copy link
Contributor Author

oli-obk commented Mar 7, 2019

@pnkfelix this is ready for review and perf looks green

pub fn new(size: Size) -> Self {
pub const BLOCK_SIZE: u64 = 64;

pub fn new(size: Size, state: bool) -> Self {

This comment has been minimized.

Copy link
@pnkfelix

pnkfelix Mar 12, 2019

Member

I infer that state here is interpreted as { true => defined, false => undefined }, (right?)

You might consider adding a comment above the header saying so. (My initial interpretation of "undef mask" was that if the bit is true, then it is undefined)

// across block boundaries
if new_state {
// set bita..64 to 1
self.blocks[blocka] |= u64::max_value() << bita;

This comment has been minimized.

Copy link
@pnkfelix

pnkfelix Mar 12, 2019

Member

(aside: i'm sort of amazed libstd doesn't have named methods for these operations; I would think turning big ranges of bits on or off within a uN (or better still, arbitrary arrays or vectors of [uN]), would be so common as to have higher-level methods than shifts and bitwise-or-masking.)

@pnkfelix

This comment has been minimized.

Copy link
Member

pnkfelix commented Mar 12, 2019

Discussed some implementation details with @oli-obk on zulip, namely about the motivation for the run-length encoded form of the undef-mask used in copy_undef_mask.

I'm satisfied that the optimization tends to help more than it hurts.

@pnkfelix

This comment has been minimized.

Copy link
Member

pnkfelix commented Mar 12, 2019

@bors r+

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Mar 12, 2019

📌 Commit 2a1eb1c has been approved by pnkfelix

Mark-Simulacrum added a commit to Mark-Simulacrum/rust that referenced this pull request Mar 14, 2019

Rollup merge of rust-lang#58556 - oli-obk:imperative_recursion, r=pnk…
…felix

Optimize copying large ranges of undefmask blocks

Hopefully fixes rust-lang#58523

bors added a commit that referenced this pull request Mar 14, 2019

Auto merge of #58556 - oli-obk:imperative_recursion, r=pnkfelix
Optimize copying large ranges of undefmask blocks

Hopefully fixes #58523
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Mar 14, 2019

⌛️ Testing commit 2a1eb1c with merge 8a04250...

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Mar 14, 2019

💔 Test failed - checks-travis

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

rust-highfive commented Mar 14, 2019

The job i686-gnu-nopt of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[02:59:03] test fmt/mod.rs - fmt::LowerExp (line 891) ... ok
[02:59:03] test fmt/mod.rs - fmt::LowerExp (line 899) ... ok
[02:59:03] test fmt/mod.rs - fmt::LowerHex (line 757) ... ok
[02:59:04] test fmt/mod.rs - fmt::LowerHex (line 746) ... ok
The job exceeded the maximum time limit for jobs, and has been terminated.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@kennytm

This comment has been minimized.

Copy link
Member

kennytm commented Mar 15, 2019

@bors retry

3 hour timeout?

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Mar 15, 2019

⌛️ Testing commit 2a1eb1c with merge 52e8856...

bors added a commit that referenced this pull request Mar 15, 2019

Auto merge of #58556 - oli-obk:imperative_recursion, r=pnkfelix
Optimize copying large ranges of undefmask blocks

Hopefully fixes #58523
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Mar 15, 2019

☀️ Test successful - checks-travis, status-appveyor
Approved by: pnkfelix
Pushing 52e8856 to master...

@bors bors added the merged-by-bors label Mar 15, 2019

@bors bors merged commit 2a1eb1c into rust-lang:master Mar 15, 2019

1 check passed

homu Test successful
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.