Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switching cwd breaks usage of custom target specifications #4905

Open
robert-w-gries opened this issue Jan 5, 2018 · 11 comments

Comments

@robert-w-gries
Copy link

commented Jan 5, 2018

Per japaric/xargo#186 (comment), the following no longer works:

$ cargo new --lib foo

$ cd foo

$ echo '#![no_std]' > src/lib.rs

$ cargo add heapless

$ rustc -Z unstable-options --target x86_64-unknown-linux-gnu --print target-spec-json > foo.json

$ cargo build --target foo
   Compiling untagged-option v0.1.1
error: Error loading target specification: Could not find specification for target "foo"

The following conversation was taken from #4865:

@SergioBenitez

Shouldn't cargo be setting cwd to the workspace root (here just the crate root, foo) which is exactly where the target file foo.json is? If so, shouldn't rustc find the file and build properly? Or has something changed in rustc so that it no longer looks for target files in the cwd?

@alexcrichton

oh I believe the issue there is that you're compiling crates from crates.io and Cargo is now switching the cwd even for those crates (as their own workspace root is themselves in Cargo's $HOME/.cargo cache

Ah yeah so to clarify I believe everything is working correctly in cargo/rustc and this is an intentional change on Cargo's behalf. (breaking in the technical/pedantic sense)

Workaround

In order to fix the cargo build error, you need to manually set RUST_TARGET_PATH.

export RUST_TARGET_PATH=`pwd`

Remaining Discussion

I think the following question remains:

Why are users of custom target specifications required to set RUST_TARGET_PATH every time cargo build is invoked?

The error message does not make it clear why the build is failing. Also, it is unintuitive to set an environment variable pointing to the current directory for the build to succeed.

If the current behavior will remain in the long-term, there should probably be documentation somewhere that addresses the breaking change. Also, I think the Cargo error message should hint that RUST_TARGET_PATH is pointing to the wrong directory.

@phil-opp

This comment has been minimized.

Copy link
Contributor

commented Jan 8, 2018

We could use absolute paths for target triples, i.e. cargo build --target /home/xyz/foo/target.json. Then the target triple would be found independent of the current working directory. We could even convert all target file paths to absolute paths in cargo, so that cargo build --target target.json works too from the home/xyz/foo folder.

Unfortunately, absolute paths don't work today, since the target triple path is just pushed to the sysroot path:

// in src/librustc/session/filesearch.rs

pub fn relative_target_lib_path(sysroot: &Path, target_triple: &str) -> PathBuf {
    let mut p = PathBuf::from(find_libdir(sysroot).as_ref());
    assert!(p.is_relative());
    p.push(RUST_LIB_DIR);
    p.push(target_triple);        <-- problem if the path is absolute
    p.push("lib");
    p
}

If target_triple is absolute, the sysroot is completely eliminated from the path on push, which means that rustc looks for the sysroot in the project folder. So we should remove the root component from the path before pushing (i.e. convert /home/xyz/foo/target.json to home/xyz/foo/target.json). An alternative might be to store it under a hash of the target.json instead of depending on the passed path. This would also solve rust-lang/rust#24666 and i think this issue too: japaric/xargo#44 (comment).

hawkw added a commit to sos-os/kernel that referenced this issue Jan 24, 2018
hawkw added a commit to sos-os/kernel that referenced this issue Jan 24, 2018
hawkw added a commit to sos-os/kernel that referenced this issue Jan 24, 2018
@phil-opp

This comment has been minimized.

Copy link
Contributor

commented Jan 31, 2018

@alexcrichton @carols10cents @matklad

This issue is affecting lots of people (everyone who uses custom targets) and has not received any attention yet. So I thought I'd ping you as members of cargo team, maybe you could take a look.

@alexcrichton

This comment has been minimized.

Copy link
Member

commented Jan 31, 2018

@phil-opp I think it's unlikely we'll be reverting to the previous behavior, at this point I think it's best to consider how to change custom targets such that they work with Cargo "by default", although I'm not quite sure what that would mean.

@alexcrichton

This comment has been minimized.

Copy link
Member

commented Feb 1, 2018

One option for Cargo is to automatically pass an absolute path for anything it detects as JSON (aka ends with json and a file exists in the cwd), and then we could fix rustc's logic to work with absolute paths.

@SergioBenitez

This comment has been minimized.

Copy link
Contributor

commented Feb 1, 2018

@alexcrichton Why not do something similar, but limit it to a file named $target.json? I believe this would restore the previous functionality.

@alexcrichton

This comment has been minimized.

Copy link
Member

commented Feb 1, 2018

@SergioBenitez yep that'd also work!

@phil-opp

This comment has been minimized.

Copy link
Contributor

commented Feb 1, 2018

@alexcrichton Thanks for the quick reply and showing a path forward!

@phil-opp

This comment has been minimized.

Copy link
Contributor

commented Mar 22, 2018

Seems like the target triple is stored in the metadata of libcore rlib (created by xargo). If the targets don't match a "couldn't find crate core with expected target triple" error occurs.

This means that we already have to compile the sysroot for the absolute target and can't just convert the triple to an absolute path in cargo. So I try to adjust xargo now.

@phil-opp

This comment has been minimized.

Copy link
Contributor

commented Mar 22, 2018

The PR triple is ready: rust-lang/rust#49019, #5228, and japaric/xargo#205. The idea is to add special treatment for target arguments ending in .json. These weren't really supported before (e.g. xargo explicitely disallowed them, rustc didn't search RUST_TARGET_PATH for such targets), so this shouldn't cause breakage.

The three PRs introduce support for absolute target paths, which makes setting RUST_TARGET_PATH unnecessary. They also perform path canonalization for paths ending in *.json so that relative paths are automatically converted to absolute ones, with the same benefits.

In summary, these PRs allow xargo build --target foo.json without RUST_TARGET_PATH again, the only difference is the additional .json ending.

bors added a commit that referenced this issue Mar 26, 2018
Add support for absolute target.json paths

Builds upon rust-lang/rust#49019 with the goal to provide a solution to #4905.

This PR does two things:

~~1. It appends a hash of the target path to the target folder name if a `*.json` path is passed as `--target`, like it's done in rust-lang/rust#49019. This helps differentiating targets with the same JSON file name and avoids sysroot clashes in `xargo`.~~ See #5228 (comment)
2. It canonicalizes the passed target path (if it's a `*.json` path), so that the path stays valid when building dependencies and setting the `RUST_TARGET_PATH` environment variable is no longer necessary.
bors added a commit to rust-lang/rust that referenced this issue Mar 28, 2018
Introduce a TargetTriple enum to support absolute target paths

This PR replaces target triple strings with a `TargetTriple` enum, which represents either a target triple or a path to a JSON target file. The path variant is used if the `--target` argument has a `.json` extension, else the target triple variant is used.

The motivation of this PR is support for absolute target paths to avoid the need for setting the `RUST_TARGET_PATH` environment variable (see rust-lang/cargo#4905 for more information). For places where some kind of triple is needed (e.g. in the sysroot folder), we use the file name (without extension).

For compatibility, we keep the old behavior of searching for a file named `$(target_triple).json` in `RUST_TARGET_PATH` for non-official target triples.
@phil-opp

This comment has been minimized.

Copy link
Contributor

commented May 8, 2018

With rust-lang/rust#49019 and #5228 merged, the example from the issue description "works" now if --target foo.json is passed instead of --target foo. "Works" in the sense that it throws a “can't find crate for core” error as before. So I think this issue is solved from the Rust/Cargo side, unless we also want to fix target arguments with no .json extension.

It still doesn't work with xargo, because japaric/xargo#205 got no review. As a maintained alternative with --target *.json support I created cargo-xbuild. It is a fork of xargo with a minimized feature set, maybe it is useful to some people in this thread.

@alex

This comment has been minimized.

Copy link
Contributor

commented Jun 13, 2018

A natural extension of this (maybe belongs in a different bug), is to support distributing targets in cargo packages. This would enable re-usability, as well as bundling a target with a platform-specific library.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.