Handle target specific outputs such as .wasm or .dll.lib #4570

tatsuya6502 · Oct 3, 2017

Until now, Cargo does not have any knowledge about target-specific output files. It relies solely on rustc for this sort of information (rustc --print=file-names ...).

As a result, Cargo does not place some build artifacts (files) to target/{debug,release} directory. These files include *.wasm for wasm32-unknown-emscripten target and *.dll.lib for *-pc-windows-msvc cdylib target.

This commit will add such knowledge to Cargo so that *.wasm and *.dll.lib will be placed in target/{debug,release} directory.

EDIT: I added a summary of changes. Please read it for more details of changes.

IMPORTANT
Although I added test cases for both wasm32-unknown-emscripten and *-pc-windows-msvc cdylib targets, I did not do manual testing on wasm32-unknown-emscripten target as I do not have an environment with emscripten installed. It will be appreciated if anybody tests this change for wasm32-unknown-emscripten target. EDIT: Tested for both wasm32-unknown-emscripten and x86_64-pc-windows-msvc. Thanks @Herschel for the help.

rust-highfive · Oct 3, 2017

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @alexcrichton (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

tatsuya6502 · Oct 4, 2017

Although I added test cases for both wasm32-unknown-emscripten and *-pc-windows-msvc cdylib targets, I did not do manual testing on wasm32-unknown-emscripten target as I do not have an environment with emscripten installed.

Please do not merge this; we found a problem with wasm target. See this comment for details.

tatsuya6502 · Oct 5, 2017

Please do not merge this; we found a problem with wasm target. See this comment for details.

I added one more commit to address the problem.

Now this PR is ready for merge.

tatsuya6502 · Oct 5, 2017

Summary of Changes

For bin target of wasm32-unknown-emscripten

Copy .wasm file from dep to its parent directory.
- If crate name contains hyphens (e.g. "wasm-test"), rustc/emcc will generate js with hyphens ("wasm-test.js") and wasm with underscores ("wasm_test.wasm"). The updated codes is aware of this and will copy the correct wasm file.
Remove metadata suffix from the target js and wasm files in dep. This is required because wasm file name is embedded in generated js file. (See this comment for details.)

Before this PR:

[package]
name = "wasm-test"
# ...

% tree target/wasm32-unknown-emscripten/
target/wasm32-unknown-emscripten/
└── debug
    ├── build
    ├── deps
    │   ├── wasm_test-0f5ba9409d843569.js
    │   ├── wasm_test-0f5ba9409d843569.wasm
    │   ├── wasm_test-0f5ba9409d843569.wasm.map
    │   └── wasm_test-0f5ba9409d843569.wast
    ├── examples
    ├── incremental
    ├── native
    ├── wasm-test.d
    └── wasm-test.js

6 directories, 6 files

After this PR:

% tree target/wasm32-unknown-emscripten/
target/wasm32-unknown-emscripten/
└── debug
    ├── build
    ├── deps
    │   ├── wasm-test.js         <== no metadata, hyphen
    │   ├── wasm_test.wasm       <== no metadata
    │   ├── wasm_test.wasm.map
    │   └── wasm_test.wast
    ├── examples
    ├── incremental
    ├── native
    ├── wasm_test.d
    ├── wasm-test.d
    ├── wasm-test.js
    └── wasm_test.wasm           <== added

6 directories, 8 files

For cdylib/dylib targets of *-pc-windows-msvc

Copy .dll.lib file from dep to its parent directory.

Before this PR:

[package]
name = "cdylib-test"
# ...

[lib]
crate-type = ["cdylib"]

PS C:\Users\tatsuya\Documents\cdylib-test> dir .\target\debug\deps\

    Directory: C:\Users\tatsuya\Documents\cdylib-test\target\debug\deps

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----        10/5/2017  06:20 PM         115200 cdylib_test.dll
-a----        10/5/2017  06:20 PM           2141 cdylib_test.dll.exp
-a----        10/5/2017  06:20 PM           3892 cdylib_test.dll.lib
-a----        10/5/2017  06:20 PM        1085440 cdylib_test.pdb

PS C:\Users\tatsuya\Documents\cdylib-test> dir .\target\debug\

    Directory: C:\Users\tatsuya\Documents\cdylib-test\target\debug

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----        10/5/2017  06:20 PM                .fingerprint
d-----        10/5/2017  06:20 PM                build
d-----        10/5/2017  06:20 PM                deps
d-----        10/5/2017  06:20 PM                examples
d-----        10/5/2017  06:20 PM                incremental
d-----        10/5/2017  06:20 PM                native
-a----        10/5/2017  06:20 PM              0 .cargo-lock
-a----        10/5/2017  06:20 PM            119 cdylib_test.d
-a----        10/5/2017  06:20 PM         115200 cdylib_test.dll

After this PR:

PS C:\Users\tatsuya\Documents\cdylib-test> dir .\target\debug\deps\

    Directory: C:\Users\tatsuya\Documents\cdylib-test\target\debug\deps


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----        10/5/2017  06:28 PM         115200 cdylib_test.dll
-a----        10/5/2017  06:28 PM           2141 cdylib_test.dll.exp
-a----        10/5/2017  06:28 PM           3892 cdylib_test.dll.lib
-a----        10/5/2017  06:28 PM        1085440 cdylib_test.pdb


PS C:\Users\tatsuya\Documents\cdylib-test> dir .\target\debug\

    Directory: C:\Users\tatsuya\Documents\cdylib-test\target\debug


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----        10/5/2017  06:28 PM                .fingerprint
d-----        10/5/2017  06:28 PM                build
d-----        10/5/2017  06:28 PM                deps
d-----        10/5/2017  06:28 PM                examples
d-----        10/5/2017  06:28 PM                incremental
d-----        10/5/2017  06:28 PM                native
-a----        10/5/2017  06:28 PM              0 .cargo-lock
-a----        10/5/2017  06:28 PM            119 cdylib_test.d
-a----        10/5/2017  06:28 PM         115200 cdylib_test.dll
-a----        10/5/2017  06:28 PM            123 cdylib_test.dll.d
-a----        10/5/2017  06:28 PM           3892 cdylib_test.dll.lib  <== added

alexcrichton · Oct 5, 2017

alexcrichton reviewed Oct 5, 2017

View changes

Nice! I've got just one small nit but otherwise this looks great to me and should be good to go. If you're extra enterprising (this is optional) I think we could tackle #4056 with this infrastructure as well, but again, that's optional!

alexcrichton · Oct 5, 2017

src/cargo/ops/cargo_rustc/context.rs

@@ -478,9 +481,11 @@ impl<'a, 'cfg> Context<'a, 'cfg> {
        // doing this eventually.
        let __cargo_default_lib_metadata = env::var("__CARGO_DEFAULT_LIB_METADATA");
        if !unit.profile.test &&
-            (unit.target.is_dylib() || unit.target.is_cdylib()) &&
+            (unit.target.is_dylib() || unit.target.is_cdylib() ||
+                 (unit.target.is_bin() && self.target_triple().starts_with("wasm32-"))) &&


Could this be refactored into a dedicated method? Something like:

fn fails_to_work_with_mangled_filename(...) -> bool { // ... }

(just to make it a bit more clear as to the intent)

tatsuya6502 · Oct 5, 2017

Thank you for reviewing the codes.

If you're extra enterprising (this is optional) I think we could tackle #4056 with this infrastructure as well

Ah, alright, there is another. Sure. I will take that. I have macOS environment and will have some time to spare in coming weekend.

alexcrichton · Oct 5, 2017

Nice! My thinking is that we probably just need to lift up the .dSYM directory whenever debuginfo is enabled for a profile. Additionally we may need to disable mangling entirely, although I'm not sure if that's still required.

That actually reminds me, maybe we should lift up the pdb files produced with MSVC debug info as well?

Again though, both of these are optional, we can always tackle in future PRs :)

tatsuya6502 · Oct 9, 2017

I did not have enough time to update this PR last weekend. I was only able to reproduce the debugger problems on macOS and Windows MSVC; no time for updating the codes. I will try to work on them in tomorrow evening (It is Monday evening now in my time zone).

My impression is I can address macOS debugger problem in this PR in a couple of days, but not Windows one. I found the full-path to the PDB file is embedded in Windows binary, so I can't move the PDB to anywhere else. It seems there is a way to remove the full-path via a linker flag. But I guess we will need to change rustc's linker driver to achieve this(?)

alexcrichton · Oct 9, 2017

Ok no worries! Let's go ahead and get this in and iterate in-tree

@bors: r+

bors · Oct 9, 2017

📌 Commit f9a541b has been approved by alexcrichton

bors · Oct 9, 2017

⌛️ Testing commit f9a541b with merge 463e850...

bors · Oct 9, 2017

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing 463e850 to master...

rust-highfive assigned alexcrichton Oct 3, 2017

Revert the file mode on context.rs (100644 -> 100755)

Loading status checks…

84b3989

tatsuya6502 referenced this pull request Oct 3, 2017
Closed
Cargo does not provide a .wasm file in the correct location when building for wasm32-unknown-emscripten #4535

kennytm referenced this pull request Oct 7, 2017
Closed
Line numbers in backtrace still do not appear on macOS for Cargo projects #4490

bors merged commit f9a541b into rust-lang:master Oct 9, 2017
3 checks passed

3 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details

continuous-integration/travis-ci/pr The Travis CI build passed
Details

homu Test successful
Details

tatsuya6502 deleted the tatsuya6502:target-specific-artifacts branch Oct 10, 2017

kennytm referenced this pull request Oct 13, 2017
Merged
Uplift *.dSYM #4616

kennytm referenced this pull request Oct 24, 2017
Merged
Bump to 1.23 and update bootstrap #45285

kennytm referenced this pull request Oct 24, 2017
Merged
rustbuild: Fix `no output generated` error for next bootstrap cargo. #45496

alexcrichton referenced this pull request Oct 31, 2017
Open
assertion when using `cargo run` with `wasm32-unknown-emscripten` #4689

alexcrichton referenced this pull request Nov 25, 2017
Closed
Regression in filename handling on wasm with tests #4750

rust-lang/cargo

Join GitHub today

Handle target specific outputs such as .wasm or .dll.lib #4570

Conversation

tatsuya6502 commented Oct 3, 2017 • edited

rust-highfive assigned alexcrichton Oct 3, 2017

This comment has been minimized.

rust-highfive commented Oct 3, 2017

tatsuya6502 referenced this pull request Oct 3, 2017

This comment has been minimized.

tatsuya6502 commented Oct 4, 2017

This comment has been minimized.

tatsuya6502 commented Oct 5, 2017

This comment has been minimized.

tatsuya6502 commented Oct 5, 2017

Summary of Changes

For bin target of wasm32-unknown-emscripten

For cdylib/dylib targets of *-pc-windows-msvc

alexcrichton reviewed Oct 5, 2017 View changes

alexcrichton left a comment

This comment has been minimized.

This comment has been minimized.

tatsuya6502 commented Oct 5, 2017

This comment has been minimized.

alexcrichton commented Oct 5, 2017

kennytm referenced this pull request Oct 7, 2017

This comment has been minimized.

tatsuya6502 commented Oct 9, 2017

This comment has been minimized.

alexcrichton commented Oct 9, 2017

This comment has been minimized.

bors commented Oct 9, 2017

This comment has been minimized.

bors commented Oct 9, 2017

bors added a commit that referenced this pull request Oct 9, 2017

This comment has been minimized.

bors commented Oct 9, 2017

Hide details View details bors merged commit f9a541b into rust-lang:master Oct 9, 2017 3 checks passed

3 checks passed

tatsuya6502 deleted the tatsuya6502:target-specific-artifacts branch Oct 10, 2017

kennytm referenced this pull request Oct 13, 2017

bors added a commit that referenced this pull request Oct 14, 2017

kennytm referenced this pull request Oct 24, 2017

kennytm added a commit to kennytm/rust that referenced this pull request Oct 24, 2017

kennytm referenced this pull request Oct 24, 2017

bors added a commit to rust-lang/rust that referenced this pull request Oct 25, 2017

kennytm added a commit to kennytm/rust that referenced this pull request Oct 25, 2017

alexcrichton referenced this pull request Oct 31, 2017

alexcrichton referenced this pull request Nov 25, 2017

tatsuya6502 commented Oct 3, 2017 •

edited

alexcrichton reviewed Oct 5, 2017

View changes

bors merged commit `f9a541b` into rust-lang:master Oct 9, 2017
3 checks passed