Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rustc: Update wasm32 support for LLVM 9 #62809

Merged
merged 2 commits into from Jul 29, 2019

Conversation

@alexcrichton
Copy link
Member

commented Jul 19, 2019

This commit brings in a number of minor updates for rustc's support for
the wasm target which has changed in the LLVM 9 update. Notable updates
include:

  • The compiler now no longer manually inserts the producers section,
    instead relying on LLVM to do so. LLVM uses the llvm.ident metadata
    for the processed-by directive (which is now emitted on the wasm
    target in this PR) and it uses debuginfo to figure out what language
    to put in the producers section.

  • Threaded WebAssembly code now requires different flags to be passed
    with LLD. In LLD we now pass:

    • --shared-memory - required since objects are compiled with
      atomics. This also means that the generated memory will be marked as
      shared.
    • --max-memory=1GB - required with the --shared-memory argument
      since shared memories in WebAssembly must have a maximum size. The
      1GB number is intended to be a conservative estimate for rustc, but
      it should be overridable with -C link-arg if necessary.
    • --passive-segments - this has become the default for multithreaded
      memory, but when compiling a threaded module all data segments need
      to be marked as passive to ensure they don't re-initialize memory
      for each thread. This will also cause LLD to emit a synthetic
      function to initialize memory which users will have to arrange to
      call.
    • The __heap_base and __data_end globals are explicitly exported
      since they're now hidden by default due to the --export flags we
      pass to LLD.
@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented Jul 19, 2019

r? @matthewjasper

(rust_highfive has picked a reviewer for you, use r? to override)

@alexcrichton

This comment has been minimized.

Copy link
Member Author

commented Jul 19, 2019

cc @nikic

@alexcrichton

This comment has been minimized.

Copy link
Member Author

commented Jul 19, 2019

I've updated this as well with the necessary support to configure #[thread_local] on WebAssembly, meaning that the TLS implementation for wasm is much simplified and no longer needs custom external support to get something workable in libstd.

@rust-highfive

This comment was marked as outdated.

Copy link
Collaborator

commented Jul 20, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-07-19T22:53:07.9903859Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-07-19T22:53:08.0090103Z ##[command]git config gc.auto 0
2019-07-19T22:53:08.0160358Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-07-19T22:53:08.0207424Z ##[command]git config --get-all http.proxy
2019-07-19T22:53:08.0337878Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/62809/merge:refs/remotes/pull/62809/merge
---
2019-07-19T22:53:41.6658887Z do so (now or later) by using -b with the checkout command again. Example:
2019-07-19T22:53:41.6658934Z 
2019-07-19T22:53:41.6659121Z   git checkout -b <new-branch-name>
2019-07-19T22:53:41.6659146Z 
2019-07-19T22:53:41.6659186Z HEAD is now at 8dba87ec3 Merge 4c61a6673ad884d734289e060ce4aba03f76e5bf into e3cebcb3bd4ffaf86bb0cdfd2af5b7e698717b01
2019-07-19T22:53:41.6797268Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-07-19T22:53:41.6799655Z ==============================================================================
2019-07-19T22:53:41.6799703Z Task         : Bash
2019-07-19T22:53:41.6799759Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-07-19T22:59:47.1878903Z    Compiling serde_json v1.0.40
2019-07-19T22:59:50.6761493Z    Compiling tidy v0.1.0 (/checkout/src/tools/tidy)
2019-07-19T22:59:58.9380708Z     Finished release [optimized] target(s) in 1m 27s
2019-07-19T22:59:58.9448852Z tidy check
2019-07-19T22:59:59.8023614Z tidy error: /checkout/src/libstd/sys/wasm/fast_thread_local.rs:4: TODO is deprecated; use FIXME
2019-07-19T23:00:00.6984914Z some tidy checks failed
2019-07-19T23:00:00.6985758Z 
2019-07-19T23:00:00.6985758Z 
2019-07-19T23:00:00.6986718Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor"
2019-07-19T23:00:00.6993005Z 
2019-07-19T23:00:00.6993032Z 
2019-07-19T23:00:00.7005982Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
2019-07-19T23:00:00.7006351Z Build completed unsuccessfully in 0:01:30
2019-07-19T23:00:00.7006351Z Build completed unsuccessfully in 0:01:30
2019-07-19T23:00:02.0075862Z ##[error]Bash exited with code '1'.
2019-07-19T23:00:02.0125060Z ##[section]Starting: Checkout
2019-07-19T23:00:02.0126927Z ==============================================================================
2019-07-19T23:00:02.0127018Z Task         : Get sources
2019-07-19T23:00:02.0127065Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@alexcrichton alexcrichton force-pushed the alexcrichton:wasm-llvm-9 branch 2 times, most recently from b307ccc to c696d57 Jul 23, 2019

@rust-highfive

This comment was marked as outdated.

Copy link
Collaborator

commented Jul 23, 2019

The job LinuxTools of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-07-23T15:00:18.2852018Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-07-23T15:00:18.3015209Z ##[command]git config gc.auto 0
2019-07-23T15:00:18.3086564Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-07-23T15:00:18.3130258Z ##[command]git config --get-all http.proxy
2019-07-23T15:00:18.3262657Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/62809/merge:refs/remotes/pull/62809/merge
---
2019-07-23T15:00:54.7786475Z do so (now or later) by using -b with the checkout command again. Example:
2019-07-23T15:00:54.7786789Z 
2019-07-23T15:00:54.7787530Z   git checkout -b <new-branch-name>
2019-07-23T15:00:54.7787806Z 
2019-07-23T15:00:54.7788196Z HEAD is now at 0a4e77443 Merge c696d5701c10f3de4e25802649f05595dfcdcc23 into 3ebca72a11869f946b31f900faffb75c2bb2473a
2019-07-23T15:00:54.7921011Z ##[section]Finishing: Checkout
2019-07-23T15:00:54.8015310Z ##[section]Starting: Decide whether to run this job
2019-07-23T15:00:54.8018130Z Task         : Bash
2019-07-23T15:00:54.8018174Z Description  : Run a Bash script on macOS, Linux, or Windows
2019-07-23T15:00:54.8018224Z Version      : 3.151.3
2019-07-23T15:00:54.8018263Z Author       : Microsoft Corporation
2019-07-23T15:00:54.8018263Z Author       : Microsoft Corporation
2019-07-23T15:00:54.8018440Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash
2019-07-23T15:00:54.8018497Z ==============================================================================
2019-07-23T15:00:54.9302916Z Generating script.
2019-07-23T15:00:54.9336169Z ========================== Starting Command Output ===========================
2019-07-23T15:00:54.9355686Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/669c1dd9-3c95-4ebb-9791-ecdd4f1f4ae9.sh
2019-07-23T15:00:55.2814618Z Executing the job since submodules are updated
2019-07-23T15:00:55.2842504Z ##[section]Finishing: Decide whether to run this job
2019-07-23T15:00:55.2852168Z ==============================================================================
2019-07-23T15:00:55.2852261Z Task         : Bash
2019-07-23T15:00:55.2852303Z Description  : Run a Bash script on macOS, Linux, or Windows
2019-07-23T15:00:55.2852378Z Version      : 3.151.3
---
2019-07-23T16:27:06.2975349Z [ 69%] Building CXX object lib/Target/Mips/CMakeFiles/LLVMMipsCodeGen.dir/MipsISelLowering.cpp.o
2019-07-23T16:27:07.7337699Z [ 69%] Building CXX object lib/Target/AArch64/CMakeFiles/LLVMAArch64CodeGen.dir/AArch64SelectionDAGInfo.cpp.o
2019-07-23T16:27:14.1936169Z [ 69%] Building CXX object lib/Target/AArch64/CMakeFiles/LLVMAArch64CodeGen.dir/AArch64SpeculationHardening.cpp.o
2019-07-23T16:27:17.9869442Z [ 69%] Building CXX object lib/Target/Mips/CMakeFiles/LLVMMipsCodeGen.dir/MipsFrameLowering.cpp.o
2019-07-23T16:27:21.5271755Z [ 69%] Building CXX object lib/Target/AArch64/CMakeFiles/LLVMAArch64CodeGen.dir/AArch64StackTagging.cpp.o
2019-07-23T16:27:29.3455658Z [ 69%] Building CXX object lib/Target/AArch64/CMakeFiles/LLVMAArch64CodeGen.dir/AArch64StorePairSuppress.cpp.o
2019-07-23T16:27:31.6137002Z [ 69%] Building CXX object lib/Target/Mips/CMakeFiles/LLVMMipsCodeGen.dir/MipsBranchExpansion.cpp.o
2019-07-23T16:27:34.8029296Z [ 69%] Building CXX object lib/Target/AArch64/CMakeFiles/LLVMAArch64CodeGen.dir/AArch64Subtarget.cpp.o
2019-07-23T16:27:39.6413335Z [ 69%] Building CXX object lib/Target/Mips/CMakeFiles/LLVMMipsCodeGen.dir/MipsMCInstLower.cpp.o
---
2019-07-23T18:20:48.8454811Z Cloning into 'rust-toolstate'...
2019-07-23T18:20:49.4856614Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-23T18:20:49.5113306Z [master 4145817] (linux CI update)
2019-07-23T18:20:49.5113443Z  1 file changed, 1 insertion(+)
2019-07-23T18:20:50.2054252Z remote: Invalid username or password.
2019-07-23T18:20:50.2055049Z fatal: Authentication failed for 'https://github.com/rust-lang-nursery/rust-toolstate.git/'
2019-07-23T18:20:53.5650443Z  * branch            master     -> FETCH_HEAD
2019-07-23T18:20:53.5816549Z HEAD is now at 8007779 (linux CI update)
2019-07-23T18:20:53.5926094Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-23T18:20:53.6120982Z [master 3248da4] (linux CI update)
2019-07-23T18:20:53.6120982Z [master 3248da4] (linux CI update)
2019-07-23T18:20:53.6121687Z  1 file changed, 1 insertion(+)
2019-07-23T18:20:53.9347338Z fatal: could not read Username for 'https://github.com': No such device or address
2019-07-23T18:20:57.3021823Z  * branch            master     -> FETCH_HEAD
2019-07-23T18:20:57.3170039Z HEAD is now at 8007779 (linux CI update)
2019-07-23T18:20:57.3276717Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-23T18:20:57.3453298Z [master 99ccde3] (linux CI update)
2019-07-23T18:20:57.3453298Z [master 99ccde3] (linux CI update)
2019-07-23T18:20:57.3454078Z  1 file changed, 1 insertion(+)
2019-07-23T18:20:57.6566247Z fatal: could not read Username for 'https://github.com': No such device or address
2019-07-23T18:20:59.0370527Z  * branch            master     -> FETCH_HEAD
2019-07-23T18:20:59.0517907Z HEAD is now at 8007779 (linux CI update)
2019-07-23T18:20:59.0626785Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-23T18:20:59.0804604Z [master 5718258] (linux CI update)
2019-07-23T18:20:59.0804604Z [master 5718258] (linux CI update)
2019-07-23T18:20:59.0805578Z  1 file changed, 1 insertion(+)
2019-07-23T18:20:59.3957231Z fatal: could not read Username for 'https://github.com': No such device or address
2019-07-23T18:21:03.1844800Z  * branch            master     -> FETCH_HEAD
2019-07-23T18:21:03.1995751Z HEAD is now at 8007779 (linux CI update)
2019-07-23T18:21:03.2106913Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-23T18:21:03.2288538Z [master 8aec8a5] (linux CI update)
2019-07-23T18:21:03.2288538Z [master 8aec8a5] (linux CI update)
2019-07-23T18:21:03.2288627Z  1 file changed, 1 insertion(+)
2019-07-23T18:21:03.9514644Z fatal: could not read Username for 'https://github.com': No such device or address
2019-07-23T18:21:05.5661419Z  * branch            master     -> FETCH_HEAD
2019-07-23T18:21:05.5809460Z HEAD is now at 8007779 (linux CI update)
2019-07-23T18:21:05.5809460Z HEAD is now at 8007779 (linux CI update)
2019-07-23T18:21:06.4651410Z ##[error]Bash exited with code '1'.
2019-07-23T18:21:06.4705526Z ##[section]Starting: Checkout
2019-07-23T18:21:06.4707868Z ==============================================================================
2019-07-23T18:21:06.4707927Z Task         : Get sources
2019-07-23T18:21:06.4707974Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

} else {
// stubbed out because no functions actually access these intrinsics
// unless atomics are enabled
MY_ID

This comment has been minimized.

Copy link
@nikic

nikic Jul 23, 2019

Contributor

What is the reason for using a compare_exchange loop here, rather than an atomic increment (something like MY_ID = NEXT_ID.fetch_add(1, SeqCst))? Is this not supported by wasm?

This comment has been minimized.

Copy link
@alexcrichton

alexcrichton Jul 23, 2019

Author Member

While perhaps slightly overzealous, the thinking here is that if the counter everything should deterministically panic, as opposed to only one thread deterministically panicking but others continuing.

This comment has been minimized.

Copy link
@nikic

nikic Jul 25, 2019

Contributor

I see. Maybe this is a nice opportunity to use the unstable fetch_update API, something like:

MY_ID = match NEXT_ID.fetch_update(|cur| cur.overflowing_add(1), SeqCst, SeqCst) {
    Some(id) => id + 1,
    Err(_) => crate::arch::wasm32::unreachable(),
}

This comment has been minimized.

Copy link
@alexcrichton

alexcrichton Jul 25, 2019

Author Member

Oh good point! I think I'd prefer to avoid picking up an unstable feature for this since it's relatively minor, but I refactored a bit with your suggestion to use checked arithmetic which I think is better than what was there already!

@bors

This comment was marked as outdated.

Copy link
Contributor

commented Jul 25, 2019

☔️ The latest upstream changes (presumably #62961) made this pull request unmergeable. Please resolve the merge conflicts.

rustc: Update wasm32 support for LLVM 9
This commit brings in a number of minor updates for rustc's support for
the wasm target which has changed in the LLVM 9 update. Notable updates
include:

* The compiler now no longer manually inserts the `producers` section,
  instead relying on LLVM to do so. LLVM uses the `llvm.ident` metadata
  for the `processed-by` directive (which is now emitted on the wasm
  target in this PR) and it uses debuginfo to figure out what `language`
  to put in the `producers` section.

* Threaded WebAssembly code now requires different flags to be passed
  with LLD. In LLD we now pass:

  * `--shared-memory` - required since objects are compiled with
    atomics. This also means that the generated memory will be marked as
    `shared`.
  * `--max-memory=1GB` - required with the `--shared-memory` argument
    since shared memories in WebAssembly must have a maximum size. The
    1GB number is intended to be a conservative estimate for rustc, but
    it should be overridable with `-C link-arg` if necessary.
  * `--passive-segments` - this has become the default for multithreaded
    memory, but when compiling a threaded module all data segments need
    to be marked as passive to ensure they don't re-initialize memory
    for each thread. This will also cause LLD to emit a synthetic
    function to initialize memory which users will have to arrange to
    call.
  * The `__heap_base` and `__data_end` globals are explicitly exported
    since they're now hidden by default due to the `--export` flags we
    pass to LLD.

@alexcrichton alexcrichton force-pushed the alexcrichton:wasm-llvm-9 branch from c696d57 to c9529d8 Jul 25, 2019

@alexcrichton

This comment has been minimized.

Copy link
Member Author

commented Jul 25, 2019

Sorry to bother y'all again, but I'm not sure who's best to review this? @rust-lang/compiler would y'all be ok helping me find a reviewer for this?

@nikic

This comment has been minimized.

Copy link
Contributor

commented Jul 25, 2019

r? @nikic I'll review a bit later today...

@rust-highfive rust-highfive assigned nikic and unassigned matthewjasper Jul 25, 2019

@rust-highfive

This comment was marked as outdated.

Copy link
Collaborator

commented Jul 25, 2019

The job LinuxTools of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-07-25T14:09:36.5726176Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-07-25T14:09:36.5947699Z ##[command]git config gc.auto 0
2019-07-25T14:09:36.6012524Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-07-25T14:09:36.6077400Z ##[command]git config --get-all http.proxy
2019-07-25T14:09:36.6215606Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/62809/merge:refs/remotes/pull/62809/merge
---
2019-07-25T14:10:11.8590916Z do so (now or later) by using -b with the checkout command again. Example:
2019-07-25T14:10:11.8590949Z 
2019-07-25T14:10:11.8591178Z   git checkout -b <new-branch-name>
2019-07-25T14:10:11.8591207Z 
2019-07-25T14:10:11.8591270Z HEAD is now at 93928a558 Merge c9529d8c4b614afc19334753b5425c34fb0290fd into eedf6ce4ef54bb03818ab21d714f1b9f13a6b31c
2019-07-25T14:10:11.8750325Z ##[section]Finishing: Checkout
2019-07-25T14:10:11.8757714Z ##[section]Starting: Decide whether to run this job
2019-07-25T14:10:11.8761398Z Task         : Bash
2019-07-25T14:10:11.8761446Z Description  : Run a Bash script on macOS, Linux, or Windows
2019-07-25T14:10:11.8761494Z Version      : 3.151.3
2019-07-25T14:10:11.8761557Z Author       : Microsoft Corporation
2019-07-25T14:10:11.8761557Z Author       : Microsoft Corporation
2019-07-25T14:10:11.8761611Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash
2019-07-25T14:10:11.8761667Z ==============================================================================
2019-07-25T14:10:12.0115681Z Generating script.
2019-07-25T14:10:12.0148899Z ========================== Starting Command Output ===========================
2019-07-25T14:10:12.0167188Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/6cd29aef-c367-4b15-ba71-1cfa0661d968.sh
2019-07-25T14:10:12.3826865Z Executing the job since submodules are updated
2019-07-25T14:10:12.3921795Z ##[section]Finishing: Decide whether to run this job
2019-07-25T14:10:12.3931749Z ==============================================================================
2019-07-25T14:10:12.3931808Z Task         : Bash
2019-07-25T14:10:12.3931855Z Description  : Run a Bash script on macOS, Linux, or Windows
2019-07-25T14:10:12.3931940Z Version      : 3.151.3
---
2019-07-25T15:35:31.2704754Z [ 69%] Building CXX object lib/Target/Mips/CMakeFiles/LLVMMipsCodeGen.dir/MipsISelDAGToDAG.cpp.o
2019-07-25T15:35:32.6893994Z [ 69%] Building CXX object lib/Target/AArch64/CMakeFiles/LLVMAArch64CodeGen.dir/AArch64SelectionDAGInfo.cpp.o
2019-07-25T15:35:38.9473415Z [ 69%] Building CXX object lib/Target/Mips/CMakeFiles/LLVMMipsCodeGen.dir/MipsISelLowering.cpp.o
2019-07-25T15:35:39.1505693Z [ 69%] Building CXX object lib/Target/AArch64/CMakeFiles/LLVMAArch64CodeGen.dir/AArch64SpeculationHardening.cpp.o
2019-07-25T15:35:46.2742733Z [ 69%] Building CXX object lib/Target/AArch64/CMakeFiles/LLVMAArch64CodeGen.dir/AArch64StackTagging.cpp.o
2019-07-25T15:35:53.9503084Z [ 69%] Building CXX object lib/Target/AArch64/CMakeFiles/LLVMAArch64CodeGen.dir/AArch64StorePairSuppress.cpp.o
2019-07-25T15:35:57.8673491Z [ 69%] Building CXX object lib/Target/Mips/CMakeFiles/LLVMMipsCodeGen.dir/MipsLegalizerInfo.cpp.o
2019-07-25T15:35:59.3118880Z [ 69%] Building CXX object lib/Target/AArch64/CMakeFiles/LLVMAArch64CodeGen.dir/AArch64Subtarget.cpp.o
2019-07-25T15:36:03.9771189Z [ 69%] Building CXX object lib/Target/Mips/CMakeFiles/LLVMMipsCodeGen.dir/MipsBranchExpansion.cpp.o
---
2019-07-25T17:33:20.6412532Z Cloning into 'rust-toolstate'...
2019-07-25T17:33:21.2818483Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-25T17:33:21.3069389Z [master 1ff2519] (linux CI update)
2019-07-25T17:33:21.3069526Z  1 file changed, 1 insertion(+)
2019-07-25T17:33:21.9780238Z remote: Invalid username or password.
2019-07-25T17:33:21.9780993Z fatal: Authentication failed for 'https://github.com/rust-lang-nursery/rust-toolstate.git/'
2019-07-25T17:33:25.3077553Z  * branch            master     -> FETCH_HEAD
2019-07-25T17:33:25.3250437Z HEAD is now at a587986 (windows CI update)
2019-07-25T17:33:25.3364444Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-25T17:33:25.3548282Z [master 257d30b] (linux CI update)
2019-07-25T17:33:25.3548282Z [master 257d30b] (linux CI update)
2019-07-25T17:33:25.3549002Z  1 file changed, 1 insertion(+)
2019-07-25T17:33:25.6422950Z fatal: could not read Username for 'https://github.com': No such device or address
2019-07-25T17:33:26.9868164Z  * branch            master     -> FETCH_HEAD
2019-07-25T17:33:27.0016113Z HEAD is now at a587986 (windows CI update)
2019-07-25T17:33:27.0130139Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-25T17:33:27.0317047Z [master 6c62b31] (linux CI update)
2019-07-25T17:33:27.0317047Z [master 6c62b31] (linux CI update)
2019-07-25T17:33:27.0317346Z  1 file changed, 1 insertion(+)
2019-07-25T17:33:27.3149768Z fatal: could not read Username for 'https://github.com': No such device or address
2019-07-25T17:33:27.6386131Z  * branch            master     -> FETCH_HEAD
2019-07-25T17:33:27.6538402Z HEAD is now at a587986 (windows CI update)
2019-07-25T17:33:27.6651251Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-25T17:33:27.6840843Z [master 6c62b31] (linux CI update)
2019-07-25T17:33:27.6840843Z [master 6c62b31] (linux CI update)
2019-07-25T17:33:27.6841277Z  1 file changed, 1 insertion(+)
2019-07-25T17:33:27.9772865Z fatal: could not read Username for 'https://github.com': No such device or address
2019-07-25T17:33:28.2959469Z  * branch            master     -> FETCH_HEAD
2019-07-25T17:33:28.3107686Z HEAD is now at a587986 (windows CI update)
2019-07-25T17:33:28.3216445Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-25T17:33:28.3401236Z [master 01c3425] (linux CI update)
2019-07-25T17:33:28.3401236Z [master 01c3425] (linux CI update)
2019-07-25T17:33:28.3401369Z  1 file changed, 1 insertion(+)
2019-07-25T17:33:28.6577396Z fatal: could not read Username for 'https://github.com': No such device or address
2019-07-25T17:33:31.9969104Z  * branch            master     -> FETCH_HEAD
2019-07-25T17:33:32.0118621Z HEAD is now at a587986 (windows CI update)
2019-07-25T17:33:32.0118621Z HEAD is now at a587986 (windows CI update)
2019-07-25T17:33:32.5614929Z ##[error]Bash exited with code '1'.
2019-07-25T17:33:32.5657329Z ##[section]Starting: Checkout
2019-07-25T17:33:32.5659075Z ==============================================================================
2019-07-25T17:33:32.5659144Z Task         : Get sources
2019-07-25T17:33:32.5659191Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@nikic
Copy link
Contributor

left a comment

r=me with the wrapping issue fixed.

I'm not familiar with wasm linking details, but it looks reasonable :)

src/libstd/sys/wasm/thread.rs Outdated Show resolved Hide resolved
} else {
// stubbed out because no functions actually access these intrinsics
// unless atomics are enabled
MY_ID

This comment has been minimized.

Copy link
@nikic

nikic Jul 25, 2019

Contributor

I see. Maybe this is a nice opportunity to use the unstable fetch_update API, something like:

MY_ID = match NEXT_ID.fetch_update(|cur| cur.overflowing_add(1), SeqCst, SeqCst) {
    Some(id) => id + 1,
    Err(_) => crate::arch::wasm32::unreachable(),
}
src/librustc_target/spec/wasm32_base.rs Outdated Show resolved Hide resolved

@alexcrichton alexcrichton force-pushed the alexcrichton:wasm-llvm-9 branch from c9529d8 to eb9090f Jul 25, 2019

std: Use native `#[thread_local]` TLS on wasm
This commit moves `thread_local!` on WebAssembly targets to using the
`#[thread_local]` attribute in LLVM. This was recently implemented
upstream and is [in the process of being documented][dox]. This change
only takes affect if modules are compiled with `+atomics` which is
currently unstable and a pretty esoteric method of compiling wasm
artifacts.

This "new power" of the wasm toolchain means that the old
`wasm-bindgen-threads` feature of the standard library can be removed
since it should now be possible to create a fully functioning threaded
wasm module without intrusively dealing with libstd symbols or
intrinsics. Yay!

[dox]: WebAssembly/tool-conventions#116

@alexcrichton alexcrichton force-pushed the alexcrichton:wasm-llvm-9 branch from eb9090f to dc50a63 Jul 25, 2019

@alexcrichton

This comment has been minimized.

Copy link
Member Author

commented Jul 25, 2019

@bors: r=nikic

@bors

This comment has been minimized.

Copy link
Contributor

commented Jul 25, 2019

📌 Commit dc50a63 has been approved by nikic

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented Jul 25, 2019

The job LinuxTools of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-07-25T18:26:24.2431296Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-07-25T18:26:24.2633298Z ##[command]git config gc.auto 0
2019-07-25T18:26:24.2709658Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-07-25T18:26:24.2756043Z ##[command]git config --get-all http.proxy
2019-07-25T18:26:24.2894515Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/62809/merge:refs/remotes/pull/62809/merge
---
2019-07-25T18:26:57.7362235Z do so (now or later) by using -b with the checkout command again. Example:
2019-07-25T18:26:57.7362262Z 
2019-07-25T18:26:57.7362646Z   git checkout -b <new-branch-name>
2019-07-25T18:26:57.7362673Z 
2019-07-25T18:26:57.7362715Z HEAD is now at 593ba213f Merge dc50a633f3260a3aeb79a4ca9800587be7f732e7 into eedf6ce4ef54bb03818ab21d714f1b9f13a6b31c
2019-07-25T18:26:57.7510806Z ##[section]Finishing: Checkout
2019-07-25T18:26:57.7518402Z ##[section]Starting: Decide whether to run this job
2019-07-25T18:26:57.7520877Z Task         : Bash
2019-07-25T18:26:57.7520935Z Description  : Run a Bash script on macOS, Linux, or Windows
2019-07-25T18:26:57.7521102Z Version      : 3.151.3
2019-07-25T18:26:57.7521140Z Author       : Microsoft Corporation
2019-07-25T18:26:57.7521140Z Author       : Microsoft Corporation
2019-07-25T18:26:57.7521200Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash
2019-07-25T18:26:57.7521244Z ==============================================================================
2019-07-25T18:26:57.8849822Z Generating script.
2019-07-25T18:26:57.8881342Z ========================== Starting Command Output ===========================
2019-07-25T18:26:57.8903554Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/6dcdbbcd-ac3f-4bed-9fea-8c929086552c.sh
2019-07-25T18:26:58.2097137Z Executing the job since submodules are updated
2019-07-25T18:26:58.2197431Z ##[section]Finishing: Decide whether to run this job
2019-07-25T18:26:58.2208298Z ==============================================================================
2019-07-25T18:26:58.2208353Z Task         : Bash
2019-07-25T18:26:58.2208417Z Description  : Run a Bash script on macOS, Linux, or Windows
2019-07-25T18:26:58.2208592Z Version      : 3.151.3
---
2019-07-25T19:53:04.9763051Z [ 69%] Building CXX object lib/Target/Mips/CMakeFiles/LLVMMipsCodeGen.dir/MipsISelDAGToDAG.cpp.o
2019-07-25T19:53:10.2308378Z [ 69%] Building CXX object lib/Target/AArch64/CMakeFiles/LLVMAArch64CodeGen.dir/AArch64SelectionDAGInfo.cpp.o
2019-07-25T19:53:12.4585443Z [ 69%] Building CXX object lib/Target/Mips/CMakeFiles/LLVMMipsCodeGen.dir/MipsISelLowering.cpp.o
2019-07-25T19:53:16.5634735Z [ 69%] Building CXX object lib/Target/AArch64/CMakeFiles/LLVMAArch64CodeGen.dir/AArch64SpeculationHardening.cpp.o
2019-07-25T19:53:23.5003120Z [ 69%] Building CXX object lib/Target/AArch64/CMakeFiles/LLVMAArch64CodeGen.dir/AArch64StackTagging.cpp.o
2019-07-25T19:53:30.2891621Z [ 69%] Building CXX object lib/Target/Mips/CMakeFiles/LLVMMipsCodeGen.dir/MipsLegalizerInfo.cpp.o
2019-07-25T19:53:31.2317445Z [ 69%] Building CXX object lib/Target/AArch64/CMakeFiles/LLVMAArch64CodeGen.dir/AArch64StorePairSuppress.cpp.o
2019-07-25T19:53:36.3869472Z [ 69%] Building CXX object lib/Target/AArch64/CMakeFiles/LLVMAArch64CodeGen.dir/AArch64Subtarget.cpp.o
2019-07-25T19:53:36.6958225Z [ 69%] Building CXX object lib/Target/Mips/CMakeFiles/LLVMMipsCodeGen.dir/MipsBranchExpansion.cpp.o
---
2019-07-25T21:43:11.6581966Z Cloning into 'rust-toolstate'...
2019-07-25T21:43:12.3090009Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-25T21:43:12.3307706Z [master 6ffac73] (linux CI update)
2019-07-25T21:43:12.3309326Z  1 file changed, 1 insertion(+)
2019-07-25T21:43:12.9438737Z remote: Invalid username or password.
2019-07-25T21:43:12.9439619Z fatal: Authentication failed for 'https://github.com/rust-lang-nursery/rust-toolstate.git/'
2019-07-25T21:43:16.2460484Z  * branch            master     -> FETCH_HEAD
2019-07-25T21:43:16.2630045Z HEAD is now at 994f84a (windows CI update)
2019-07-25T21:43:16.2749892Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-25T21:43:16.2906775Z [master 4e0c1f5] (linux CI update)
2019-07-25T21:43:16.2906775Z [master 4e0c1f5] (linux CI update)
2019-07-25T21:43:16.2907769Z  1 file changed, 1 insertion(+)
2019-07-25T21:43:16.5796432Z fatal: could not read Username for 'https://github.com': No such device or address
2019-07-25T21:43:18.9191678Z  * branch            master     -> FETCH_HEAD
2019-07-25T21:43:18.9332104Z HEAD is now at 994f84a (windows CI update)
2019-07-25T21:43:18.9435096Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-25T21:43:18.9591663Z [master 666c8ef] (linux CI update)
2019-07-25T21:43:18.9591663Z [master 666c8ef] (linux CI update)
2019-07-25T21:43:18.9591741Z  1 file changed, 1 insertion(+)
2019-07-25T21:43:19.2355184Z fatal: could not read Username for 'https://github.com': No such device or address
2019-07-25T21:43:19.5427166Z  * branch            master     -> FETCH_HEAD
2019-07-25T21:43:19.5565835Z HEAD is now at 994f84a (windows CI update)
2019-07-25T21:43:19.5667530Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-25T21:43:19.5828562Z [master 12cfd22] (linux CI update)
2019-07-25T21:43:19.5828562Z [master 12cfd22] (linux CI update)
2019-07-25T21:43:19.5829090Z  1 file changed, 1 insertion(+)
2019-07-25T21:43:19.8607832Z fatal: could not read Username for 'https://github.com': No such device or address
2019-07-25T21:43:22.1643726Z  * branch            master     -> FETCH_HEAD
2019-07-25T21:43:22.1776511Z HEAD is now at 994f84a (windows CI update)
2019-07-25T21:43:22.1876844Z The state of "rustc-guide" has changed from "test-pass" to ""
2019-07-25T21:43:22.2040255Z [master e40a318] (linux CI update)
2019-07-25T21:43:22.2040255Z [master e40a318] (linux CI update)
2019-07-25T21:43:22.2040679Z  1 file changed, 1 insertion(+)
2019-07-25T21:43:22.4798321Z fatal: could not read Username for 'https://github.com': No such device or address
2019-07-25T21:43:24.7865019Z  * branch            master     -> FETCH_HEAD
2019-07-25T21:43:24.8041671Z HEAD is now at 994f84a (windows CI update)
2019-07-25T21:43:24.8041671Z HEAD is now at 994f84a (windows CI update)
2019-07-25T21:43:25.4853774Z ##[error]Bash exited with code '1'.
2019-07-25T21:43:25.4892815Z ##[section]Starting: Checkout
2019-07-25T21:43:25.4894881Z ==============================================================================
2019-07-25T21:43:25.4894943Z Task         : Get sources
2019-07-25T21:43:25.4894995Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

Centril added a commit to Centril/rust that referenced this pull request Jul 28, 2019

Rollup merge of rust-lang#62809 - alexcrichton:wasm-llvm-9, r=nikic
rustc: Update wasm32 support for LLVM 9

This commit brings in a number of minor updates for rustc's support for
the wasm target which has changed in the LLVM 9 update. Notable updates
include:

* The compiler now no longer manually inserts the `producers` section,
  instead relying on LLVM to do so. LLVM uses the `llvm.ident` metadata
  for the `processed-by` directive (which is now emitted on the wasm
  target in this PR) and it uses debuginfo to figure out what `language`
  to put in the `producers` section.

* Threaded WebAssembly code now requires different flags to be passed
  with LLD. In LLD we now pass:

  * `--shared-memory` - required since objects are compiled with
    atomics. This also means that the generated memory will be marked as
    `shared`.
  * `--max-memory=1GB` - required with the `--shared-memory` argument
    since shared memories in WebAssembly must have a maximum size. The
    1GB number is intended to be a conservative estimate for rustc, but
    it should be overridable with `-C link-arg` if necessary.
  * `--passive-segments` - this has become the default for multithreaded
    memory, but when compiling a threaded module all data segments need
    to be marked as passive to ensure they don't re-initialize memory
    for each thread. This will also cause LLD to emit a synthetic
    function to initialize memory which users will have to arrange to
    call.
  * The `__heap_base` and `__data_end` globals are explicitly exported
    since they're now hidden by default due to the `--export` flags we
    pass to LLD.

bors added a commit that referenced this pull request Jul 28, 2019

Auto merge of #63062 - Centril:rollup-1vjfw8s, r=Centril
Rollup of 4 pull requests

Successful merges:

 - #62550 (Implement RFC 2707 + Parser recovery for range patterns)
 - #62759 (Actually add rustc-guide to toolstate, don't fail builds for the guide)
 - #62809 (rustc: Update wasm32 support for LLVM 9)
 - #62974 (bump crossbeam-epoch dependency)

Failed merges:

r? @ghost
@bors

This comment has been minimized.

Copy link
Contributor

commented Jul 29, 2019

⌛️ Testing commit dc50a63 with merge e14bb10...

bors added a commit that referenced this pull request Jul 29, 2019

Auto merge of #62809 - alexcrichton:wasm-llvm-9, r=nikic
rustc: Update wasm32 support for LLVM 9

This commit brings in a number of minor updates for rustc's support for
the wasm target which has changed in the LLVM 9 update. Notable updates
include:

* The compiler now no longer manually inserts the `producers` section,
  instead relying on LLVM to do so. LLVM uses the `llvm.ident` metadata
  for the `processed-by` directive (which is now emitted on the wasm
  target in this PR) and it uses debuginfo to figure out what `language`
  to put in the `producers` section.

* Threaded WebAssembly code now requires different flags to be passed
  with LLD. In LLD we now pass:

  * `--shared-memory` - required since objects are compiled with
    atomics. This also means that the generated memory will be marked as
    `shared`.
  * `--max-memory=1GB` - required with the `--shared-memory` argument
    since shared memories in WebAssembly must have a maximum size. The
    1GB number is intended to be a conservative estimate for rustc, but
    it should be overridable with `-C link-arg` if necessary.
  * `--passive-segments` - this has become the default for multithreaded
    memory, but when compiling a threaded module all data segments need
    to be marked as passive to ensure they don't re-initialize memory
    for each thread. This will also cause LLD to emit a synthetic
    function to initialize memory which users will have to arrange to
    call.
  * The `__heap_base` and `__data_end` globals are explicitly exported
    since they're now hidden by default due to the `--export` flags we
    pass to LLD.

Centril added a commit to Centril/rust that referenced this pull request Jul 29, 2019

Rollup merge of rust-lang#62809 - alexcrichton:wasm-llvm-9, r=nikic
rustc: Update wasm32 support for LLVM 9

This commit brings in a number of minor updates for rustc's support for
the wasm target which has changed in the LLVM 9 update. Notable updates
include:

* The compiler now no longer manually inserts the `producers` section,
  instead relying on LLVM to do so. LLVM uses the `llvm.ident` metadata
  for the `processed-by` directive (which is now emitted on the wasm
  target in this PR) and it uses debuginfo to figure out what `language`
  to put in the `producers` section.

* Threaded WebAssembly code now requires different flags to be passed
  with LLD. In LLD we now pass:

  * `--shared-memory` - required since objects are compiled with
    atomics. This also means that the generated memory will be marked as
    `shared`.
  * `--max-memory=1GB` - required with the `--shared-memory` argument
    since shared memories in WebAssembly must have a maximum size. The
    1GB number is intended to be a conservative estimate for rustc, but
    it should be overridable with `-C link-arg` if necessary.
  * `--passive-segments` - this has become the default for multithreaded
    memory, but when compiling a threaded module all data segments need
    to be marked as passive to ensure they don't re-initialize memory
    for each thread. This will also cause LLD to emit a synthetic
    function to initialize memory which users will have to arrange to
    call.
  * The `__heap_base` and `__data_end` globals are explicitly exported
    since they're now hidden by default due to the `--export` flags we
    pass to LLD.
@Centril

This comment has been minimized.

Copy link
Member

commented Jul 29, 2019

@bors retry rolled up.

bors added a commit that referenced this pull request Jul 29, 2019

Auto merge of #63094 - Centril:rollup-lm7peuh, r=Centril
Rollup of 6 pull requests

Successful merges:

 - #62809 (rustc: Update wasm32 support for LLVM 9)
 - #63055 (Various cleanups to save analysis)
 - #63076 (Miri: fix determining size of an "extra function" allocation)
 - #63077 (cleanup: Remove some language features related to built-in macros)
 - #63086 (Ignore test cases that are not supported by vxWorks)
 - #63092 (Update `impl Trait` gate issues)

Failed merges:

r? @ghost
@bors

This comment has been minimized.

Copy link
Contributor

commented Jul 29, 2019

⌛️ Testing commit dc50a63 with merge 8b94e9e...

@bors bors merged commit dc50a63 into rust-lang:master Jul 29, 2019

2 of 5 checks passed

pr Build #20190725.47 failed
Details
pr (LinuxTools) LinuxTools failed
Details
homu Testing commit dc50a633f3260a3aeb79a4ca9800587be7f732e7 with merge 8b94e9e9188b65df38a5f1ae723617dc2dfb3155...
Details
pr (Linux mingw-check) Linux mingw-check succeeded
Details
pr (Linux x86_64-gnu-llvm-6.0) Linux x86_64-gnu-llvm-6.0 succeeded
Details

@alexcrichton alexcrichton deleted the alexcrichton:wasm-llvm-9 branch Jul 31, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.