Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-enable compatibility with readonly CARGO_HOME #6940

Merged
merged 1 commit into from May 14, 2019

Conversation

Projects
None yet
6 participants
@alexcrichton
Copy link
Member

commented May 14, 2019

Previously Cargo would attempt to work as much as possible with a
previously filled out CARGO_HOME, even if it was mounted as read-only.
In #6880 this was regressed as a few global locks and files were always
attempted to be opened in writable mode.

This commit fixes these issues by correcting two locations:

  • First the global package cache lock has error handling to allow
    acquiring the lock in read-only mode inaddition to read/write mode. If
    the read/write mode failed due to an error that looks like a readonly
    filesystem then we assume everything in the package cache is readonly
    and we switch to just acquiring any lock, this time a shared readonly
    one. We in theory aren't actually doing any synchronization at that
    point since it's all readonly anyway.

  • Next when unpacking package we're careful to issue a stat call
    before opening a file in writable mode. This way our preexisting guard
    to return early if a package is unpacked will succeed before we open
    anything in writable mode.

Closes #6928

@rust-highfive

This comment has been minimized.

Copy link

commented May 14, 2019

r? @nrc

(rust_highfive has picked a reviewer for you, use r? to override)

@alexcrichton

This comment has been minimized.

Copy link
Member Author

commented May 14, 2019

For posterity, the way I tested this was:

$ cargo build --features vendored-openssl
$ docker run \
  -v `rustc --print sysroot`:/rust:ro \
  -e RUSTC=/rust/bin/rustc \
  -w /src \
  -it \
  -v `pwd`:/src \
  -v $HOME/.cargo:/cargo:ro \
  -e CARGO_HOME=/cargo \
  ubuntu:18.04 \
  ./target/debug/cargo build

If that progresses to the point that it fails the build due to a missing cc then it means we got past the resolution phase and we've actually started executing rustc and everything is downloaded. I hit the two errors fixed here previously and afterwards it made more progress.

I'm not entirely certain if this matches up with Crater's use case, so I'm not 100% certain it will solve it.

@alexcrichton

This comment has been minimized.

Copy link
Member Author

commented May 14, 2019

Oh right there's these things called permission bits, let me add a test.

Re-enable compatibility with readonly CARGO_HOME
Previously Cargo would attempt to work as much as possible with a
previously filled out CARGO_HOME, even if it was mounted as read-only.
In #6880 this was regressed as a few global locks and files were always
attempted to be opened in writable mode.

This commit fixes these issues by correcting two locations:

* First the global package cache lock has error handling to allow
  acquiring the lock in read-only mode inaddition to read/write mode. If
  the read/write mode failed due to an error that looks like a readonly
  filesystem then we assume everything in the package cache is readonly
  and we switch to just acquiring any lock, this time a shared readonly
  one. We in theory aren't actually doing any synchronization at that
  point since it's all readonly anyway.

* Next when unpacking package we're careful to issue a `stat` call
  before opening a file in writable mode. This way our preexisting guard
  to return early if a package is unpacked will succeed before we open
  anything in writable mode.

Closes #6928

@alexcrichton alexcrichton force-pushed the alexcrichton:readonly-compat branch from f09940c to 5d9383e May 14, 2019

@alexcrichton

This comment has been minimized.

Copy link
Member Author

commented May 14, 2019

r? @ehuss

@rust-highfive rust-highfive assigned ehuss and unassigned nrc May 14, 2019

@ehuss

This comment has been minimized.

Copy link
Contributor

commented May 14, 2019

@bors r+

@bors

This comment has been minimized.

Copy link
Contributor

commented May 14, 2019

📌 Commit 5d9383e has been approved by ehuss

@bors

This comment has been minimized.

Copy link
Contributor

commented May 14, 2019

⌛️ Testing commit 5d9383e with merge 414c1eb...

bors added a commit that referenced this pull request May 14, 2019

Auto merge of #6940 - alexcrichton:readonly-compat, r=ehuss
Re-enable compatibility with readonly CARGO_HOME

Previously Cargo would attempt to work as much as possible with a
previously filled out CARGO_HOME, even if it was mounted as read-only.
In #6880 this was regressed as a few global locks and files were always
attempted to be opened in writable mode.

This commit fixes these issues by correcting two locations:

* First the global package cache lock has error handling to allow
  acquiring the lock in read-only mode inaddition to read/write mode. If
  the read/write mode failed due to an error that looks like a readonly
  filesystem then we assume everything in the package cache is readonly
  and we switch to just acquiring any lock, this time a shared readonly
  one. We in theory aren't actually doing any synchronization at that
  point since it's all readonly anyway.

* Next when unpacking package we're careful to issue a `stat` call
  before opening a file in writable mode. This way our preexisting guard
  to return early if a package is unpacked will succeed before we open
  anything in writable mode.

Closes #6928
@bors

This comment has been minimized.

Copy link
Contributor

commented May 14, 2019

☀️ Test successful - checks-travis, status-appveyor
Approved by: ehuss
Pushing 414c1eb to master...

@bors bors merged commit 5d9383e into rust-lang:master May 14, 2019

3 checks passed

Travis CI - Pull Request Build Passed
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
homu Test successful
Details

@ehuss ehuss referenced this pull request May 16, 2019

Merged

Update cargo #60874

bors added a commit to rust-lang/rust that referenced this pull request May 16, 2019

Auto merge of #60874 - ehuss:update-cargo, r=alexcrichton
Update cargo

17 commits in 759b6161a328db1d4863139e90875308ecd25a75..c4fcfb725b4be00c72eb9cf30c7d8b095577c280
2019-05-06 20:47:49 +0000 to 2019-05-15 19:48:47 +0000
- tests: registry: revert readonly permission after running tests. (rust-lang/cargo#6947)
- Remove Candidate (rust-lang/cargo#6946)
- Fix for "Running cargo update without a Cargo.lock ignores arguments" rust-lang/cargo#6872 (rust-lang/cargo#6904)
- Fix a minor mistake in the changelog. (rust-lang/cargo#6944)
- Give a better error message when crates.io requests time out (rust-lang/cargo#6936)
- Re-enable compatibility with readonly CARGO_HOME (rust-lang/cargo#6940)
- Fix version of `ignore`. (rust-lang/cargo#6938)
- Stabilize offline mode. (rust-lang/cargo#6934)
- zsh: Add doc options to include non-public items documentation (rust-lang/cargo#6929)
- zsh: Suggest --lib option as binary template now the default (rust-lang/cargo#6926)
- Migrate package include/exclude to gitignore patterns. (rust-lang/cargo#6924)
- Implement the Cargo half of pipelined compilation (take 2) (rust-lang/cargo#6883)
- Always include `Cargo.toml` when packaging. (rust-lang/cargo#6925)
- Remove unnecessary calls to masquerade_as_nightly_cargo. (rust-lang/cargo#6923)
- download: fix "Downloaded 1 crates" message (crates -> crate) (rust-lang/cargo#6920)
- Changed RUST_LOG usage to CARGO_LOG to avoid confusion. (rust-lang/cargo#6918)
- crate download: don't print that a crate was the largest download if it was the only download (rust-lang/cargo#6916)

@alexcrichton alexcrichton deleted the alexcrichton:readonly-compat branch Jun 5, 2019

@djahandarie

This comment has been minimized.

Copy link

commented Jul 11, 2019

Is this PR a full solution to the issue given NixOS/nixpkgs#61618?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.