Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: Remove leading newline in vendor output #11273

Merged
merged 1 commit into from
Oct 22, 2022
Merged

fix: Remove leading newline in vendor output #11273

merged 1 commit into from
Oct 22, 2022

Conversation

epage
Copy link
Contributor

@epage epage commented Oct 21, 2022

This supersedes #11271

@rust-highfive
Copy link

r? @weihanglo

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 21, 2022
@epage epage mentioned this pull request Oct 21, 2022
Closed
weihanglo
weihanglo approved these changes Oct 22, 2022
View reviewed changes
Copy link
Member

@weihanglo weihanglo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the upstream fix!

@weihanglo
Copy link
Member

@bors r+

@bors
Copy link
Collaborator

bors commented Oct 22, 2022

📌 Commit a9f704a has been approved by weihanglo

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 22, 2022
@bors
Copy link
Collaborator

bors commented Oct 22, 2022

⌛ Testing commit a9f704a with merge 071eeaf...

@bors
Copy link
Collaborator

bors commented Oct 22, 2022

☀️ Test successful - checks-actions
Approved by: weihanglo
Pushing 071eeaf to master...

@bors bors merged commit 071eeaf into rust-lang:master Oct 22, 2022
@weihanglo weihanglo mentioned this pull request Oct 22, 2022
Merged
@epage epage deleted the toml branch October 22, 2022 12:46
bors pushed a commit to rust-lang-ci/rust that referenced this pull request Oct 22, 2022
@weihanglo
Update cargo
9926f6e 
5 commits in 3ff044334f0567ce1481c78603aeee7211b91623..071eeaf210708219a5a1b2c4728ca2f97df7f2ae
2022-10-17 20:25:00 +0000 to 2022-10-22 01:17:55 +0000

- fix: Remove leading newline in vendor output (rust-lang/cargo#11273)
- Fix publishing with a dependency on a sparse registry (rust-lang/cargo#11268)
- Add missing edition (rust-lang/cargo#11265)
- fix(publish): Check remote git registry more than once post-publish (rust-lang/cargo#11255)
- Fix typo (rust-lang/cargo#11258)
bors added a commit to rust-lang-ci/rust that referenced this pull request Oct 22, 2022
@bors
Auto merge of rust-lang#103400 - weihanglo:update-cargo, r=weihanglo
5a0f454 
Update cargo

5 commits in 3ff044334f0567ce1481c78603aeee7211b91623..071eeaf210708219a5a1b2c4728ca2f97df7f2ae 2022-10-17 20:25:00 +0000 to 2022-10-22 01:17:55 +0000

- fix: Remove leading newline in vendor output (rust-lang/cargo#11273)
- Fix publishing with a dependency on a sparse registry (rust-lang/cargo#11268)
- Add missing edition (rust-lang/cargo#11265)
- fix(publish): Check remote git registry more than once post-publish (rust-lang/cargo#11255)
- Fix typo (rust-lang/cargo#11258)

r? `@ghost`
@ehuss ehuss added this to the 1.66.0 milestone Oct 30, 2022
arnout pushed a commit to buildroot/buildroot that referenced this pull request Oct 31, 2022
@yann-morin-1998
support/download/cargo-post-process: cargo output for vendor config
04154a6 
Use the output of `cargo vendor` to generate the vendor configuration.

Fixes the need to patch the generated configuration if there are
non-crates.io dependencies.

Note:
  `cargo vendor` currently prints a newline before it prints the
  needed configuration.

  This is fixed in +nightly, will end up in +stable soon and must
  be considered when updating cargo.
  See: rust-lang/cargo#11273

  Until then it is needed to remove this first line to make sure
  that the contents of .cargo/config will be the same as they were
  generated with the earlier version of the script. Thus, the
  hashes of the packages that use this script remain the same.

Signed-off-by: Simon Richter <simon.richter@ptwdosimetry.com>
[yann.morin.1998@free.fr: add comment in rust-bin.mk and rust.mk]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
@ehuss ehuss mentioned this pull request Dec 11, 2022
Merged
arnout pushed a commit to buildroot/buildroot that referenced this pull request Jan 10, 2023
@yann-morin-1998
support/download: fix cargo vendoring
ac2b26f 
Commit de4cf25 (package/{rust, rust-bin}: bump to version 1.66.0)
forgot, despite the big comment above the version strings, to confirm
that the vendoring was still working.

Previously, we were adding the vendoring equivalence manually, but in
commit 04154a6 (support/download/cargo-post-process: cargo output
for vendor config), we switched to using the output of "cargo vendor"
(on stdout) to support cases were the vendoring equivalence would be
more complex (e.g. when using crates not hosted on crates.io).

With rust until and including 1.65.0, "cargo vendor" would output (for
crates.io crates) the same output as our manual fixups, except it was
preceded by an empty line. So, to avoid recompting all our hashes, we
added a tweak to strip away the leading empty line in 04154a6.

But rust 1.66.0 includes [0] which changes the output (on stdout) of
"cargo vendor", where the first empty line is no longer emitted.

This means that our tweak for rust 1.65.0 now strips out an important
part of the cargo vendor output, which renders the archives invalid, and
thus generates different archives, which fail to validate against our
hashes.

Fix this by doing what the comment in the post-process helper states,
and just keep the whole output of "cargo vendor", by just removing the
"tail --lines=+2". Since that comment is no longer meaningful, we drop
it too.

Now, all our 6 cargo-based packages, as well as our 5 python packages
that have rust code, can be vendored again, without changing our hashes,
but most importantly, with valid archives.

Still, we keep the comment above the versions strings, in the hope that
a future bumper will notice and be more careful at validating the
vendoring.

[0] rust-lang/cargo#11273

Fixes:
    http://autobuild.buildroot.org/results/bea/beac7674bbc9fd2f8777b5861f65afee9c485753/  (bat)
    http://autobuild.buildroot.org/results/d1e/d1ec1ebbde115628a4b8b9099544347242a97c1c/  (dust)
    http://autobuild.buildroot.org/results/f96/f968be895be9ca98b314fdd688ef8d3bdf4e5dfb/  (hyerfine)
    http://autobuild.buildroot.org/results/a0c/a0cdb6cc9493f5248d98f98b13da854e12adc2be/  (ripgrep)
    ... and so many others...

Reported-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: James Hilliard <james.hilliard1@gmail.com>
Cc: Simon Richter <simon.richter@ptwdosimetry.com>
Reviewed-by: James Hilliard <james.hilliard1@gmail.com>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Tested-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
ccrisan pushed a commit to ccrisan/thingos that referenced this pull request Mar 30, 2023
@ccrisan
support/download/cargo-post-process: cargo output for vendor config
1212f87 
Use the output of `cargo vendor` to generate the vendor configuration.

Fixes the need to patch the generated configuration if there are
non-crates.io dependencies.

Note:
  `cargo vendor` currently prints a newline before it prints the
  needed configuration.

  This is fixed in +nightly, will end up in +stable soon and must
  be considered when updating cargo.
  See: rust-lang/cargo#11273

  Until then it is needed to remove this first line to make sure
  that the contents of .cargo/config will be the same as they were
  generated with the earlier version of the script. Thus, the
  hashes of the packages that use this script remain the same.

Signed-off-by: Simon Richter <simon.richter@ptwdosimetry.com>
[yann.morin.1998@free.fr: add comment in rust-bin.mk and rust.mk]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@weihanglo weihanglo weihanglo approved these changes

Assignees

@weihanglo weihanglo

Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Milestone
1.66.0
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@epage @rust-highfive @weihanglo @bors @ehuss