Skip to content

refactor: replace InternedString with Cow in IndexPackage #15559

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 16, 2025

Conversation

0xPoe
Copy link
Member

@0xPoe 0xPoe commented May 18, 2025

What does this PR try to resolve?

ref #14834

As described in the issue, we want to move IndexPackage into cargo-util-schemas. However, it contains InternedString fields, which we don't want to expose as part of the public API.
This PR replaces InternedString with Cow.

And also, as @weihanglo's suggested, I implemented the From/Into trait to simplify code.

From Cow tarit implementation: 42f593f (#15559)

Replace InternedString with into() across the whole codebase: 5f90098 (#15559)
I am unsure if it worsens the readability. Feel free to comment.

How should we test and review this PR?

It shouldn't change or break any tests.

Benchmark 1: #15559 (comment)
Benchmark 2: #15559 (comment)

Additional information

None

@rustbot rustbot added the A-registries Area: registries label May 18, 2025
@0xPoe 0xPoe changed the title refactor: replace InternedString with Cow in IndexPackage WIP: refactor: replace InternedString with Cow in IndexPackage May 18, 2025
@0xPoe 0xPoe force-pushed the rustin-patch-cow branch from eaf777e to f290a45 Compare May 18, 2025 20:32
@0xPoe 0xPoe force-pushed the rustin-patch-cow branch 2 times, most recently from 5fdaca9 to de4a534 Compare May 19, 2025 20:45
@0xPoe 0xPoe changed the title WIP: refactor: replace InternedString with Cow in IndexPackage refactor: replace InternedString with Cow in IndexPackage May 19, 2025
@0xPoe 0xPoe marked this pull request as ready for review May 19, 2025 20:50
@rustbot
Copy link
Collaborator

rustbot commented May 19, 2025

r? @ehuss

rustbot has assigned @ehuss.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 19, 2025
@0xPoe 0xPoe assigned weihanglo and unassigned ehuss May 19, 2025
@ehuss
Copy link
Contributor

ehuss commented May 19, 2025

I would recommend at least doing some benchmarking using our bench suite. https://github.com/rust-lang/cargo/tree/master/benches contains some documentation, and particularly the resolve bench. cargo bench -p benchsuite --bench resolve is a good place to start (while you are figuring out how to run the benchmarks you can do an individual workspace like resolve_ws/rust to speed things up), and Comparing implementation explains how you can do a comparison report against master.

@0xPoe 0xPoe force-pushed the rustin-patch-cow branch from de4a534 to c5b799e Compare May 22, 2025 14:41
@0xPoe

This comment was marked as outdated.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we could change InternString::from_cow to something like:

impl<'a> From<Cow<'a, str>> for InternedString {
    fn from(cs: Cow<'a, str>) -> Self {
        let mut cache = interned_storage();
        let s = cache.get(cs.as_ref()).copied().unwrap_or_else(|| {
            let s = cs.into_owned().leak();
            cache.insert(s);
            s
        });

        InternedString { inner: s }
    }
}

So in other places we can rely on Into<InternedString>.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@weihanglo Do you think it makes sense to replace all instances of InternedString::new with .into() where possible? There are quite a few usages of InternedString::new.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine with wherever reasonable. Could also leave them for future

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine with wherever reasonable. Could also leave them for future

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will update all of them in a separate commit.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have done it in this commit. 5f90098 (#15559).
I am unsure if it worsens the readability. Maybe we shouldn't do it at all 😄

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine with either. Looks like this is ready for review/merge?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. It is ready for review. Thank you!

@0xPoe 0xPoe force-pushed the rustin-patch-cow branch from c5b799e to 42eb272 Compare May 23, 2025 14:24
@0xPoe 0xPoe marked this pull request as draft May 23, 2025 20:17
@rustbot rustbot added the A-rebuild-detection Area: rebuild detection and fingerprinting label May 26, 2025
@0xPoe 0xPoe force-pushed the rustin-patch-cow branch from 94aac2e to 5fe108d Compare May 26, 2025 14:54
@0xPoe
Copy link
Member Author

0xPoe commented May 27, 2025

I’ve tested this change several times using the resolve_ws/rust benchmark suite.

My test env:

  Model Name:	MacBook Pro
  Chip:	Apple M4 Max
  Total Number of Cores:	14 (10 performance and 4 efficiency)
  Memory:	36 GB

Test result:
master commit: HEAD is now at c2c636a fix(toml): Remove workaround for rustc frontmatter support (#15570)
master: cargo bench -p benchsuite --bench resolve -- resolve_ws/rust --save-baseline master --measurement-time 50 --sample-size 500
patch: cargo bench -p benchsuite --bench resolve -- resolve_ws/rust --baseline master --measurement-time 50 --sample-size 500

First round:

Benchmark Time (mean) Change Outliers
resolve_ws/rust 91.910 ms -0.9313% 31 (6.20%) - 14 high mild, 16 high severe
resolve_ws/rust-ws-inherit 92.021 ms +0.7503% 43 (8.60%) - 14 high mild, 22 high severe

Second round:

Benchmark Time (mean) Change Outliers
resolve_ws/rust 92.142 ms +1.7303% 33 (6.60%) - 10 high mild, 23 high severe
resolve_ws/rust-ws-inherit 91.193 ms -0.0707% 30 (6.00%) - 8 high mild, 22 high severe

Third round:

Benchmark Time (mean) Change Outliers
resolve_ws/rust 91.398 ms +1.2693% 28 (5.60%) - 8 high mild, 20 high severe
resolve_ws/rust-ws-inherit 90.558 ms +0.5349% 26 (5.20%) - 8 high mild, 18 high severe

Fourth round:

Benchmark Time (mean) Change Outliers
resolve_ws/rust 91.193 ms +0.8302% 26 (5.20%) - 11 high mild, 15 high severe
resolve_ws/rust-ws-inherit 90.591 ms +0.8267% 34 (6.80%) - 6 high mild, 17 high severe

@0xPoe
Copy link
Member Author

0xPoe commented Jun 1, 2025

I’ve tested this change several times using the resolve_ws/tikv benchmark suite.

My test env:

  Model Name:	MacBook Pro
  Chip:	Apple M4 Max
  Total Number of Cores:	14 (10 performance and 4 efficiency)
  Memory:	36 GB

Test result:
master commit: HEAD is now at c2c636a fix(toml): Remove workaround for rustc frontmatter support (#15570)
master: cargo bench -p benchsuite --bench resolve -- resolve_ws/tikv --save-baseline master --measurement-time 100 --sample-size 600
patch: cargo bench -p benchsuite --bench resolve -- resolve_ws/tikv --baseline master --measurement-time 100 --sample-size 600

First round:

Benchmark Time (mean) Change Outliers
resolve_ws/tikv 131.32 ms -2.4203% 42 (7.00%) - 18 high mild, 24 high severe

Second round:

Benchmark Time (mean) Change Outliers
resolve_ws/tikv 133.14 ms +0.8323% 37 (6.17%) - 14 high mild, 23 high severe

Third round:

Benchmark Time (mean) Change Outliers
resolve_ws/tikv 134.06 ms +0.9986% 30 (5.00%) - 10 high mild, 20 high severe

Fourth round:

Benchmark Time (mean) Change Outliers
resolve_ws/tikv 135.27 ms +2.3135% 106 (17.67%) - 71 high mild, 35 high severe

@rustbot rustbot added A-cache-messages Area: caching of compiler messages A-cli Area: Command-line interface, option parsing, etc. A-crate-dependencies Area: [dependencies] of any kind A-dependency-resolution Area: dependency resolution and the resolver A-features2 Area: issues specifically related to the v2 feature resolver A-interacts-with-crates.io Area: interaction with registries A-manifest Area: Cargo.toml issues labels Jun 14, 2025
0xPoe added 2 commits June 14, 2025 23:38
Signed-off-by: 0xPoe <techregister@pm.me>
Signed-off-by: 0xPoe <techregister@pm.me>
@0xPoe 0xPoe force-pushed the rustin-patch-cow branch from 18fe056 to feaf55f Compare June 14, 2025 21:39
…odebase

Signed-off-by: 0xPoe <techregister@pm.me>
@0xPoe 0xPoe force-pushed the rustin-patch-cow branch from feaf55f to 5f90098 Compare June 14, 2025 21:43
@0xPoe 0xPoe marked this pull request as ready for review June 15, 2025 09:37
Copy link
Member Author

@0xPoe 0xPoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔢 Self-check (PR reviewed by myself and ready for feedback)

  • Code compiles successfully

  • Benchmark Tests

  • All tests pass

  • PR title and description updated

  • Documentation PR created (or confirmed not needed)

  • PR size is reasonable

@0xPoe 0xPoe requested a review from weihanglo June 15, 2025 09:43
Copy link
Member

@weihanglo weihanglo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The difference shown in the benchmark seems insignificant, so going to merge this. Thank you so much!

@weihanglo weihanglo added this pull request to the merge queue Jun 16, 2025
Merged via the queue into rust-lang:master with commit f6cea42 Jun 16, 2025
24 checks passed
@0xPoe 0xPoe deleted the rustin-patch-cow branch June 16, 2025 20:42
@0xPoe
Copy link
Member Author

0xPoe commented Jun 16, 2025

Thanks for your review! 💚 💙 💜 💛 ❤️

jhpratt added a commit to jhpratt/rust that referenced this pull request Jun 17, 2025
Update cargo

6 commits in fc1518ef02b77327d70d4026b95ea719dd9b8c51..2251525ae503fa196f6d7f9ce6d32eccb2d5f044
2025-06-06 04:49:44 +0000 to 2025-06-16 22:01:27 +0000
- feat: Add custom completer for `cargo remove &lt;TAB&gt;` (rust-lang/cargo#15662)
- chore(deps): update msrv (3 versions) to v1.85 (rust-lang/cargo#15668)
- refactor: replace InternedString with Cow in IndexPackage (rust-lang/cargo#15559)
- highlight the correct words (rust-lang/cargo#15659)
- CHANGELOG.md: typo (rust-lang/cargo#15660)
- Use `Not::not` rather than a custom `is_false` function (rust-lang/cargo#15645)
@rustbot rustbot modified the milestone: 1.89.0 Jun 18, 2025
rust-timer added a commit to rust-lang/rust that referenced this pull request Jun 18, 2025
Rollup merge of #142632 - ehuss:update-cargo, r=ehuss

Update cargo

6 commits in fc1518ef02b77327d70d4026b95ea719dd9b8c51..2251525ae503fa196f6d7f9ce6d32eccb2d5f044
2025-06-06 04:49:44 +0000 to 2025-06-16 22:01:27 +0000
- feat: Add custom completer for `cargo remove &lt;TAB&gt;` (rust-lang/cargo#15662)
- chore(deps): update msrv (3 versions) to v1.85 (rust-lang/cargo#15668)
- refactor: replace InternedString with Cow in IndexPackage (rust-lang/cargo#15559)
- highlight the correct words (rust-lang/cargo#15659)
- CHANGELOG.md: typo (rust-lang/cargo#15660)
- Use `Not::not` rather than a custom `is_false` function (rust-lang/cargo#15645)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-cache-messages Area: caching of compiler messages A-cli Area: Command-line interface, option parsing, etc. A-crate-dependencies Area: [dependencies] of any kind A-dependency-resolution Area: dependency resolution and the resolver A-features2 Area: issues specifically related to the v2 feature resolver A-interacts-with-crates.io Area: interaction with registries A-manifest Area: Cargo.toml issues A-profiles Area: profiles A-rebuild-detection Area: rebuild detection and fingerprinting A-registries Area: registries A-testing-cargo-itself Area: cargo's tests A-workspaces Area: workspaces Command-add Command-info Command-metadata Command-rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants