Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Only key by Cargo.toml and Cargo.lock files of workspace members #180

Merged
merged 2 commits into from Dec 3, 2023
Merged

Only key by Cargo.toml and Cargo.lock files of workspace members #180

merged 2 commits into from Dec 3, 2023

Conversation

max-heller
Copy link
Contributor

@max-heller max-heller commented Nov 25, 2023 •
edited

Currently, we glob for all Cargo.toml and Cargo.lock files anywhere in a workspace, but this can pick up false positives (see #179). This PR switches to only using Cargo.toml files of workspace members (determined using cargo metadata --no-deps) and per-workspace Cargo.lock files.

Closes #179

@max-heller max-heller mentioned this pull request Nov 25, 2023
Closed
@max-heller max-heller force-pushed the cargo-manifests branch 2 times, most recently from a41534f to 86473ee Compare November 25, 2023 14:38
Swatinem
Swatinem reviewed Nov 25, 2023
View reviewed changes
Copy link
Owner

@Swatinem Swatinem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks good, in theory.
however previously we had some problems with cargo metadata building everything within the workspace eagerly (on cache "restore"), which negated the whole purpose of the cache.

I’m wondering if the --no-deps flag might change that behavior or not, or if a more uptodate version of cargo does not suffer from this. But I am very reluctant to try and find out. Let me run a couple of tests for this, as the problem with cargo metadata was reproducible on https://github.com/getsentry/relay, and I can try with that on monday.

src/config.ts Outdated Show resolved Hide resolved
@max-heller
Copy link
Contributor Author

however previously we had some problems with cargo metadata building everything within the workspace eagerly (on cache "restore"), which negated the whole purpose of the cache.

I’m wondering if the --no-deps flag might change that behavior or not

My guess is that it wouldn't be building anything, but downloading dependencies from the registry to analyze their manifests. The documentation for --no-deps says it "[doesn't] fetch dependencies", which sounds like what we want.

@max-heller
Copy link
Contributor Author

the problem with cargo metadata was reproducible on https://github.com/getsentry/relay, and I can try with that on monday

Did you get a chance to test this?

@max-heller max-heller mentioned this pull request Dec 1, 2023
Merged
max-heller added a commit to max-heller/mdbook-pandoc that referenced this pull request Dec 1, 2023
@max-heller
ci: fix release workflow (#13)
1278134 
The `rust-cache` action in the release workflow currently breaks because this repo's submodules contain a lot of `Cargo.toml` and `Cargo.lock` files. This will be fixed by Swatinem/rust-cache#180, but for now this PR disables cloning submodules during the release workflow.
@Swatinem
Copy link
Owner

Swatinem commented Dec 3, 2023

Sorry for the delay, I wasn’t able to reproduce the original problem with cargo metadata locally. Even back then it wasn’t so easy to reproduce I remember. Maybe in the meantime things might have changed. I will merge this and then do some tests in CI.

@Swatinem Swatinem merged commit d30f114 into Swatinem:master Dec 3, 2023
22 checks passed
@max-heller max-heller deleted the cargo-manifests branch December 3, 2023 11:01
@max-heller max-heller mentioned this pull request Jan 11, 2024
Merged
max-heller added a commit to max-heller/mdbook-pandoc that referenced this pull request Jan 11, 2024
@max-heller
ci: update cargo-dist to v0.7.1 (#56)
a8f673c 
This includes Swatinem/rust-cache#180 and allows using the unmodified `cargo-dist` release workflow
github-merge-queue bot pushed a commit to knope-dev/knope that referenced this pull request Jan 12, 2024
@renovate
chore(deps): update dependency swatinem/rust-cache to v2.7.2 (#778)
e70bd1e 
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [Swatinem/rust-cache](https://togithub.com/Swatinem/rust-cache) |
minor | `v2` -> `v2.7.2` |

---

### Release Notes

<details>
<summary>Swatinem/rust-cache (Swatinem/rust-cache)</summary>

###
[`v2.7.2`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.7.2)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.7.0...v2.7.2)

#### What's Changed

- Update action runtime to `node20` by
[@&#8203;rhysd](https://togithub.com/rhysd) in
[Swatinem/rust-cache#175
- Only key by `Cargo.toml` and `Cargo.lock` files of workspace members
by [@&#8203;max-heller](https://togithub.com/max-heller) in
[Swatinem/rust-cache#180

#### New Contributors

- [@&#8203;rhysd](https://togithub.com/rhysd) made their first
contribution in
[Swatinem/rust-cache#175
- [@&#8203;max-heller](https://togithub.com/max-heller) made their first
contribution in
[Swatinem/rust-cache#180

**Full Changelog**:
Swatinem/rust-cache@v2.7.1...v2.7.2

###
[`v2.7.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.7.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.6.2...v2.7.0)

##### What's Changed

- Fix save-if documentation in readme by
[@&#8203;rukai](https://togithub.com/rukai) in
[Swatinem/rust-cache#166
- Support for `trybuild` and similar macro testing tools by
[@&#8203;neysofu](https://togithub.com/neysofu) in
[Swatinem/rust-cache#168

##### New Contributors

- [@&#8203;rukai](https://togithub.com/rukai) made their first
contribution in
[Swatinem/rust-cache#166
- [@&#8203;neysofu](https://togithub.com/neysofu) made their first
contribution in
[Swatinem/rust-cache#168

**Full Changelog**:
Swatinem/rust-cache@v2.6.2...v2.7.0

###
[`v2.6.2`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.6.2)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.6.1...v2.6.2)

##### What's Changed

- dep: Use `smol-toml` instead of `toml` by
[@&#8203;NobodyXu](https://togithub.com/NobodyXu) in
[Swatinem/rust-cache#164

**Full Changelog**:
Swatinem/rust-cache@v2...v2.6.2

###
[`v2.6.1`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.6.1)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.6.0...v2.6.1)

-   Fix hash contributions of `Cargo.lock`/`Cargo.toml` files.

###
[`v2.6.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.6.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.5.1...v2.6.0)

##### What's Changed

- Add "buildjet" as a second `cache-provider` backend
[@&#8203;joroshiba](https://togithub.com/joroshiba) in
[Swatinem/rust-cache#154
-   Clean up sparse registry index.
-   Do not clean up src of `-sys` crates.
-   Remove `.cargo/credentials.toml` before saving.

##### New Contributors

- [@&#8203;joroshiba](https://togithub.com/joroshiba) made their first
contribution in
[Swatinem/rust-cache#154

**Full Changelog**:
Swatinem/rust-cache@v2.5.1...v2.6.0

###
[`v2.5.1`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.5.1)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.5.0...v2.5.1)

-   Fix hash contribution of `Cargo.lock`.

###
[`v2.5.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.5.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.4.0...v2.5.0)

##### What's Changed

- feat: Rm workspace crates version before caching by
[@&#8203;NobodyXu](https://togithub.com/NobodyXu) in
[Swatinem/rust-cache#147
- feat: Add hash of `.cargo/config.toml` to key by
[@&#8203;NobodyXu](https://togithub.com/NobodyXu) in
[Swatinem/rust-cache#149

##### New Contributors

- [@&#8203;NobodyXu](https://togithub.com/NobodyXu) made their first
contribution in
[Swatinem/rust-cache#147

**Full Changelog**:
Swatinem/rust-cache@v2.4.0...v2.5.0

###
[`v2.4.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.4.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.3.0...v2.4.0)

-   Fix cache key stability.
- Use 8 character hash components to reduce the key length, making it
more readable.

###
[`v2.3.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.3.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.2.1...v2.3.0)

- Add `cache-all-crates` option, which enables caching of crates
installed by workflows.
- Add installed packages to cache key, so changes to workflows that
install rust tools are detected and cached properly.
-   Fix cache restore failures due to upstream bug.
-   Fix `EISDIR` error due to globed directories.
- Update runtime `@actions/cache`, `@actions/io` and dev `typescript`
dependencies.
- Update `npm run prepare` so it creates distribution files with the
right line endings.

###
[`v2.2.1`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.2.1)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.2.0...v2.2.1)

- Update `@actions/cache` dependency to fix usage of `zstd` compression.

###
[`v2.2.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.2.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.1.0...v2.2.0)

- Add new `save-if` option to always restore, but only conditionally
save the cache.

###
[`v2.1.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.1.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.0.2...v2.1.0)

- Only hash `Cargo.{lock,toml}` files in the configured workspace
directories.

###
[`v2.0.2`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.0.2)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.0.1...v2.0.2)

-   Avoid calling cargo metadata on pre-cleanup.
-   Added `prefix-key`, `cache-directories` and `cache-targets` options.

###
[`v2.0.1`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.0.1)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2...v2.0.1)

- Primarily just updating dependencies to fix GitHub deprecation
notices.

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/knope-dev/knope).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMjcuMCIsInVwZGF0ZWRJblZlciI6IjM3LjEyNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
zharinov pushed a commit to zharinov/verver that referenced this pull request Feb 28, 2024
@renovate
chore(deps): update swatinem/rust-cache action to v2.7.3 (#39)
d529c8d 
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [Swatinem/rust-cache](https://togithub.com/Swatinem/rust-cache) |
action | minor | `v2.5.1` -> `v2.7.3` |

---

### Release Notes

<details>
<summary>Swatinem/rust-cache (Swatinem/rust-cache)</summary>

###
[`v2.7.3`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.7.3)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.7.2...v2.7.3)

- Work around upstream problem that causes cache saving to hang for
minutes.

**Full Changelog**:
Swatinem/rust-cache@v2.7.2...v2.7.3

###
[`v2.7.2`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.7.2)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.7.1...v2.7.2)

##### What's Changed

- Update action runtime to `node20` by
[@&#8203;rhysd](https://togithub.com/rhysd) in
[Swatinem/rust-cache#175
- Only key by `Cargo.toml` and `Cargo.lock` files of workspace members
by [@&#8203;max-heller](https://togithub.com/max-heller) in
[Swatinem/rust-cache#180

##### New Contributors

- [@&#8203;rhysd](https://togithub.com/rhysd) made their first
contribution in
[Swatinem/rust-cache#175
- [@&#8203;max-heller](https://togithub.com/max-heller) made their first
contribution in
[Swatinem/rust-cache#180

**Full Changelog**:
Swatinem/rust-cache@v2.7.1...v2.7.2

###
[`v2.7.1`](https://togithub.com/Swatinem/rust-cache/compare/v2.7.0...v2.7.1)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.7.0...v2.7.1)

###
[`v2.7.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.7.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.6.2...v2.7.0)

##### What's Changed

- Fix save-if documentation in readme by
[@&#8203;rukai](https://togithub.com/rukai) in
[Swatinem/rust-cache#166
- Support for `trybuild` and similar macro testing tools by
[@&#8203;neysofu](https://togithub.com/neysofu) in
[Swatinem/rust-cache#168

##### New Contributors

- [@&#8203;rukai](https://togithub.com/rukai) made their first
contribution in
[Swatinem/rust-cache#166
- [@&#8203;neysofu](https://togithub.com/neysofu) made their first
contribution in
[Swatinem/rust-cache#168

**Full Changelog**:
Swatinem/rust-cache@v2.6.2...v2.7.0

###
[`v2.6.2`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.6.2)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.6.1...v2.6.2)

##### What's Changed

- dep: Use `smol-toml` instead of `toml` by
[@&#8203;NobodyXu](https://togithub.com/NobodyXu) in
[Swatinem/rust-cache#164

**Full Changelog**:
Swatinem/rust-cache@v2...v2.6.2

###
[`v2.6.1`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.6.1)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.6.0...v2.6.1)

-   Fix hash contributions of `Cargo.lock`/`Cargo.toml` files.

###
[`v2.6.0`](https://togithub.com/Swatinem/rust-cache/releases/tag/v2.6.0)

[Compare
Source](https://togithub.com/Swatinem/rust-cache/compare/v2.5.1...v2.6.0)

##### What's Changed

- Add "buildjet" as a second `cache-provider` backend
[@&#8203;joroshiba](https://togithub.com/joroshiba) in
[Swatinem/rust-cache#154
-   Clean up sparse registry index.
-   Do not clean up src of `-sys` crates.
-   Remove `.cargo/credentials.toml` before saving.

##### New Contributors

- [@&#8203;joroshiba](https://togithub.com/joroshiba) made their first
contribution in
[Swatinem/rust-cache#154

**Full Changelog**:
Swatinem/rust-cache@v2.5.1...v2.6.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/zharinov/verver).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi4yNy4xIiwidXBkYXRlZEluVmVyIjoiMzcuMTI3LjAiLCJ0YXJnZXRCcmFuY2giOiJtYWluIn0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Swatinem Swatinem Swatinem left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Post job cleanup errors with "argument list too long" in repository with many Cargo.toml/Cargo.lock files
2 participants
@max-heller @Swatinem