Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use generator resume arguments in the async/await lowering #69033

Merged
merged 12 commits into from Mar 21, 2020

Conversation

@jonas-schievink
Copy link
Member

jonas-schievink commented Feb 10, 2020

This removes the TLS requirement from async/await and enables it in #![no_std] crates.

Closes #56974

I'm not confident the HIR lowering is completely correct, there seem to be quite a few undocumented invariants in there. The async-std and tokio test suites are passing with these changes though.

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

rust-highfive commented Feb 10, 2020

r? @davidtwco

(rust_highfive has picked a reviewer for you, use r? to override)

@jonas-schievink jonas-schievink force-pushed the jonas-schievink:resume-with-context branch from 8908f0d to 3292b32 Feb 10, 2020
src/librustc_ast_lowering/expr.rs Outdated Show resolved Hide resolved
let task_context_id = self.resolver.next_node_id();
let task_context_hid = self.lower_node_id(task_context_id);
let ast_decl = FnDecl {
inputs: vec![Param {
attrs: AttrVec::new(),
ty: AstP(Ty {
id: self.resolver.next_node_id(),
kind: TyKind::Infer,
span: DUMMY_SP,
}),
pat: AstP(Pat {
id: task_context_id,
kind: PatKind::Ident(
BindingMode::ByValue(Mutability::Mut),
Ident::with_dummy_span(sym::_task_context),
None,
),
span: DUMMY_SP,
}),
id: self.resolver.next_node_id(),
span: DUMMY_SP,
is_placeholder: false,
}],
output,
};
Comment on lines 488 to 511

This comment has been minimized.

Copy link
@Centril

Centril Feb 10, 2020

Member

I don't think we should be creating syntax::ast IR inside lowering just to immediately convert it to HIR. Instead, the HIR representation should be created directly. (cc @eddyb)

This comment has been minimized.

Copy link
@jonas-schievink

jonas-schievink Feb 11, 2020

Author Member

That makes sense to me, but there seems to be no existing code that directly creates an hir::FnDecl, and lower_fn_decl does quite a lot of work that I don't really want to hand-roll.

This comment has been minimized.

Copy link
@Centril

Centril Feb 11, 2020

Member

It's not that much code tbh which I think can be mostly shared by some refactorings of lower_fn_decl into the main parts (inputs, output, implicit_self, packaging those aforementioned up) as separate methods.

This comment has been minimized.

Copy link
@jonas-schievink

jonas-schievink Mar 4, 2020

Author Member

I've pushed a commit that should address this

src/libcore/future/mod.rs Outdated Show resolved Hide resolved
f.await;
}

fn main() {}

This comment has been minimized.

Copy link
@Centril

Centril Feb 10, 2020

Member

Could we test more like a whole program with more interactions?

This comment has been minimized.

Copy link
@vertexclique

vertexclique Mar 18, 2020

Member

Please add more run-pass with:

  • spawn onto the main thread with a basic waker vtable
  • a spinlock holds that task

This comment has been minimized.

Copy link
@jonas-schievink

jonas-schievink Mar 18, 2020

Author Member

There are already a lot of tests for async/await, most just don't turn on the #![no_std] flag. Since the underlying implementation is the same there is not really a difference though.

(I'm not entirely sure what the spinlock test would accomplish here though, you can store tasks however you want as long as you can get a Pin<&mut Self> to poll the future)

This comment has been minimized.

Copy link
@vertexclique

vertexclique Mar 18, 2020

Member

There are already a lot of tests for async/await, most just don't turn on the #![no_std] flag.

If we have a build matrix that enables no_std flag for async/await tests, it is ok to not add tests.(iirc we don't have) I assume compiler harness doesn't do that unless you explicitly ask for it. So when it is passing for std doesn't mean that it will pass by all cases of no_std.

(I'm not entirely sure what the spinlock test would accomplish here though,

To demonstrate also it is working in no_std env without any extra dependency brought in.

@Centril

This comment was marked as resolved.

Copy link
Member

Centril commented Feb 10, 2020

cc #69039 -- should be fixed before this can land?

@jonas-schievink

This comment was marked as resolved.

Copy link
Member Author

jonas-schievink commented Feb 10, 2020

cc #69039 -- should be fixed before this can land?

That seems to be an issue with drops – async/await only needs to be able to resume with Copy data.

@jonas-schievink

This comment was marked as resolved.

Copy link
Member Author

jonas-schievink commented Feb 10, 2020

Ah, no, I'll have to take a deeper look at that

@jonas-schievink

This comment was marked as resolved.

Copy link
Member Author

jonas-schievink commented Feb 10, 2020

#69039 should only be an issue if the resume argument is stored across yield points, and we don't do that here

@davidtwco

This comment has been minimized.

Copy link
Member

davidtwco commented Feb 11, 2020

@rust-highfive rust-highfive assigned tmandry and unassigned davidtwco Feb 11, 2020
@nikomatsakis

This comment has been minimized.

Copy link
Contributor

nikomatsakis commented Feb 14, 2020

I'm excited to see this, thanks @jonas-schievink

@CryZe

This comment has been minimized.

Copy link
Contributor

CryZe commented Feb 19, 2020

What's the status of this? Is this blocked on #69039 or could this be merged before that is resolved?

@jonas-schievink

This comment has been minimized.

Copy link
Member Author

jonas-schievink commented Feb 19, 2020

@CryZe I've started to write a test that should provide an answer to that, but I also plan to look into #69039 regardless. I hope to find some time to finish this off in the next couple of weeks.

src/libcore/future/mod.rs Show resolved Hide resolved
src/librustc_span/symbol.rs Show resolved Hide resolved
src/libcore/future/mod.rs Outdated Show resolved Hide resolved
src/libcore/future/mod.rs Show resolved Hide resolved
Copy link
Contributor

tmandry left a comment

Sorry for the latency on my end.

src/libcore/future/mod.rs Outdated Show resolved Hide resolved
src/libcore/future/mod.rs Show resolved Hide resolved
src/libcore/future/mod.rs Show resolved Hide resolved
@rust-highfive

This comment was marked as outdated.

Copy link
Collaborator

rust-highfive commented Mar 3, 2020

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2020-03-03T17:10:51.0311989Z ========================== Starting Command Output ===========================
2020-03-03T17:10:51.0313975Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/7de89db9-8299-475e-b9ee-b2780c770cfb.sh
2020-03-03T17:10:51.0314187Z 
2020-03-03T17:10:51.0317413Z ##[section]Finishing: Disable git automatic line ending conversion
2020-03-03T17:10:51.0333647Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/69033/merge to s
2020-03-03T17:10:51.0336368Z Task         : Get sources
2020-03-03T17:10:51.0336624Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-03-03T17:10:51.0336871Z Version      : 1.0.0
2020-03-03T17:10:51.0337049Z Author       : Microsoft
---
2020-03-03T17:10:52.0279250Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-03-03T17:10:52.0284445Z ##[command]git config gc.auto 0
2020-03-03T17:10:52.0287820Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-03-03T17:10:52.0291924Z ##[command]git config --get-all http.proxy
2020-03-03T17:10:52.0298377Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/69033/merge:refs/remotes/pull/69033/merge
---
2020-03-03T18:05:32.0659621Z .................................................................................................... 1700/9726
2020-03-03T18:05:36.3831552Z .................................................................................................... 1800/9726
2020-03-03T18:05:47.1017241Z .......................................................i............................................ 1900/9726
2020-03-03T18:05:54.2258968Z .................................................................................................... 2000/9726
2020-03-03T18:06:06.8510996Z .............................................iiiii.................................................. 2100/9726
2020-03-03T18:06:16.2192139Z .................................................................................................... 2300/9726
2020-03-03T18:06:18.3243579Z .................................................................................................... 2400/9726
2020-03-03T18:06:21.7376743Z .................................................................................................... 2500/9726
2020-03-03T18:06:41.5919382Z .................................................................................................... 2600/9726
---
2020-03-03T18:09:09.9295253Z ......i...............i............................................................................. 5000/9726
2020-03-03T18:09:18.6688339Z .................................................................................................... 5100/9726
2020-03-03T18:09:23.1270901Z .................................................i.................................................. 5200/9726
2020-03-03T18:09:30.9450849Z .................................................................................................... 5300/9726
2020-03-03T18:09:37.5085745Z ............................ii.ii........i...i...................................................... 5400/9726
2020-03-03T18:09:44.7817268Z .................................................................................................... 5600/9726
2020-03-03T18:09:53.1650345Z .................................................................................................... 5700/9726
2020-03-03T18:09:59.6688294Z ...................i................................................................................ 5800/9726
2020-03-03T18:10:04.6015627Z .................................................................................................... 5900/9726
2020-03-03T18:10:04.6015627Z .................................................................................................... 5900/9726
2020-03-03T18:10:14.2315895Z .................................................................................................... 6000/9726
2020-03-03T18:10:24.3901602Z ...........ii...i..ii...........i................................................................... 6100/9726
2020-03-03T18:10:38.9099699Z .................................................................................................... 6300/9726
2020-03-03T18:10:42.0338433Z .................................................................................................... 6400/9726
2020-03-03T18:10:42.0338433Z .................................................................................................... 6400/9726
2020-03-03T18:10:52.5771416Z ..........................................i..ii..................................................... 6500/9726
2020-03-03T18:11:12.0060854Z .................................................................................................... 6700/9726
2020-03-03T18:11:13.8161530Z ..................................i................................................................. 6800/9726
2020-03-03T18:11:15.6550892Z .................................................................................................... 6900/9726
2020-03-03T18:11:17.6549274Z ................................................................i................................... 7000/9726
---
2020-03-03T18:12:45.8693663Z .................................................................................................... 7700/9726
2020-03-03T18:12:50.4248087Z .................................................................................................... 7800/9726
2020-03-03T18:12:54.8669144Z .................................................................................................... 7900/9726
2020-03-03T18:13:02.3929603Z ..........i......................................................................................... 8000/9726
2020-03-03T18:13:10.4917967Z ...........................................................iiiiiii.i................................ 8100/9726
2020-03-03T18:13:22.9939800Z i......i............................................................................................ 8300/9726
2020-03-03T18:13:27.3540203Z .................................................................................................... 8400/9726
2020-03-03T18:13:40.3325098Z .................................................................................................... 8500/9726
2020-03-03T18:13:47.6403558Z .................................................................................................... 8600/9726
---
2020-03-03T18:15:30.4279392Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:348:22
2020-03-03T18:15:30.4279829Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2020-03-03T18:15:30.4300280Z 
2020-03-03T18:15:30.4300503Z 
2020-03-03T18:15:30.4304629Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "ui" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-7/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "7.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2020-03-03T18:15:30.4307404Z 
2020-03-03T18:15:30.4307508Z 
2020-03-03T18:15:30.4307904Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2020-03-03T18:15:30.4308341Z Build completed unsuccessfully in 0:57:31
2020-03-03T18:15:30.4308341Z Build completed unsuccessfully in 0:57:31
2020-03-03T18:15:30.4363317Z == clock drift check ==
2020-03-03T18:15:30.4387341Z   local time: Tue Mar  3 18:15:30 UTC 2020
2020-03-03T18:15:30.7318372Z   network time: Tue, 03 Mar 2020 18:15:30 GMT
2020-03-03T18:15:30.7325195Z == end clock drift check ==
2020-03-03T18:15:31.2574834Z 
2020-03-03T18:15:31.2638850Z ##[error]Bash exited with code '1'.
2020-03-03T18:15:31.2651650Z ##[section]Finishing: Run build
2020-03-03T18:15:31.2711019Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/69033/merge to s
2020-03-03T18:15:31.2715929Z Task         : Get sources
2020-03-03T18:15:31.2716275Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-03-03T18:15:31.2716704Z Version      : 1.0.0
2020-03-03T18:15:31.2716916Z Author       : Microsoft
2020-03-03T18:15:31.2716916Z Author       : Microsoft
2020-03-03T18:15:31.2717267Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2020-03-03T18:15:31.2717882Z ==============================================================================
2020-03-03T18:15:31.5848045Z Cleaning any cached credential from repository: rust-lang/rust (GitHub)
2020-03-03T18:15:31.5892012Z ##[section]Finishing: Checkout rust-lang/rust@refs/pull/69033/merge to s
2020-03-03T18:15:31.5974641Z Cleaning up task key
2020-03-03T18:15:31.5975882Z Start cleaning up orphan processes.
2020-03-03T18:15:31.6131819Z Terminate orphan process: pid (4218) (python)
2020-03-03T18:15:31.6383980Z ##[section]Finishing: Finalize Job

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @rust-lang/infra. (Feature Requests)

@rust-highfive

This comment was marked as outdated.

Copy link
Collaborator

rust-highfive commented Mar 4, 2020

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2020-03-04T13:58:32.7440062Z ========================== Starting Command Output ===========================
2020-03-04T13:58:32.7447402Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/4f15b2eb-6ed4-4580-9d63-35ca78d40c49.sh
2020-03-04T13:58:32.7448362Z 
2020-03-04T13:58:32.7454322Z ##[section]Finishing: Disable git automatic line ending conversion
2020-03-04T13:58:32.7476765Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/69033/merge to s
2020-03-04T13:58:32.7480567Z Task         : Get sources
2020-03-04T13:58:32.7480960Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-03-04T13:58:32.7481321Z Version      : 1.0.0
2020-03-04T13:58:32.7481567Z Author       : Microsoft
---
2020-03-04T13:58:33.7429603Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-03-04T13:58:33.7435688Z ##[command]git config gc.auto 0
2020-03-04T13:58:33.7439856Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-03-04T13:58:33.7443968Z ##[command]git config --get-all http.proxy
2020-03-04T13:58:33.7450859Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/69033/merge:refs/remotes/pull/69033/merge

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @rust-lang/infra. (Feature Requests)

@jonas-schievink jonas-schievink force-pushed the jonas-schievink:resume-with-context branch from 894f420 to 500ab9e Mar 4, 2020
@rust-highfive

This comment was marked as resolved.

Copy link
Collaborator

rust-highfive commented Mar 4, 2020

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2020-03-04T14:22:40.7145597Z ========================== Starting Command Output ===========================
2020-03-04T14:22:40.7152631Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/fd1f4e58-4a0b-4abf-9d2b-09f97b26b08f.sh
2020-03-04T14:22:40.7153220Z 
2020-03-04T14:22:40.7160543Z ##[section]Finishing: Disable git automatic line ending conversion
2020-03-04T14:22:40.7183024Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/69033/merge to s
2020-03-04T14:22:40.7187622Z Task         : Get sources
2020-03-04T14:22:40.7187962Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-03-04T14:22:40.7188279Z Version      : 1.0.0
2020-03-04T14:22:40.7188526Z Author       : Microsoft
---
2020-03-04T14:22:41.7006971Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-03-04T14:22:41.7012075Z ##[command]git config gc.auto 0
2020-03-04T14:22:41.7015486Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-03-04T14:22:41.7018404Z ##[command]git config --get-all http.proxy
2020-03-04T14:22:41.7023891Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/69033/merge:refs/remotes/pull/69033/merge
---
2020-03-04T15:25:20.0311715Z .................................................................................................... 1700/9728
2020-03-04T15:25:24.3703543Z .................................................................................................... 1800/9728
2020-03-04T15:25:35.2430529Z .......................................................i............................................ 1900/9728
2020-03-04T15:25:42.3437713Z .................................................................................................... 2000/9728
2020-03-04T15:25:55.6442976Z .............................................iiiii.................................................. 2100/9728
2020-03-04T15:26:05.0418012Z .................................................................................................... 2300/9728
2020-03-04T15:26:07.1408128Z .................................................................................................... 2400/9728
2020-03-04T15:26:10.4890771Z .................................................................................................... 2500/9728
2020-03-04T15:26:30.9999060Z .................................................................................................... 2600/9728
---
2020-03-04T15:29:01.6684682Z ......i...............i............................................................................. 5000/9728
2020-03-04T15:29:11.0766020Z .................................................................................................... 5100/9728
2020-03-04T15:29:15.5968968Z .................................................i.................................................. 5200/9728
2020-03-04T15:29:23.7625560Z .................................................................................................... 5300/9728
2020-03-04T15:29:30.7184851Z ............................ii.ii........i...i...................................................... 5400/9728
2020-03-04T15:29:38.5894065Z .................................................................................................... 5600/9728
2020-03-04T15:29:47.8966959Z .................................................................................................... 5700/9728
2020-03-04T15:29:55.0244017Z ...................i................................................................................ 5800/9728
2020-03-04T15:30:00.4190281Z .................................................................................................... 5900/9728
2020-03-04T15:30:00.4190281Z .................................................................................................... 5900/9728
2020-03-04T15:30:10.7737948Z .................................................................................................... 6000/9728
2020-03-04T15:30:21.2264505Z ...........ii...i..ii...........i................................................................... 6100/9728
2020-03-04T15:30:36.4239379Z .................................................................................................... 6300/9728
2020-03-04T15:30:43.1291861Z .................................................................................................... 6400/9728
2020-03-04T15:30:43.1291861Z .................................................................................................... 6400/9728
2020-03-04T15:30:56.1105794Z ..........................................i..ii..................................................... 6500/9728
2020-03-04T15:31:16.7377274Z .................................................................................................... 6700/9728
2020-03-04T15:31:18.7336911Z ..................................i................................................................. 6800/9728
2020-03-04T15:31:20.6595789Z .................................................................................................... 6900/9728
2020-03-04T15:31:22.7814130Z ................................................................i................................... 7000/9728
---
2020-03-04T15:32:57.7752442Z .................................................................................................... 7700/9728
2020-03-04T15:33:02.7602234Z .................................................................................................... 7800/9728
2020-03-04T15:33:07.6876660Z .................................................................................................... 7900/9728
2020-03-04T15:33:15.5539600Z ..........i......................................................................................... 8000/9728
2020-03-04T15:33:23.5057054Z ...........................................................iiiiiiiii.i.............................. 8100/9728
2020-03-04T15:33:37.1212879Z ..i......i.......................................................................................... 8300/9728
2020-03-04T15:33:42.1211242Z .................................................................................................... 8400/9728
2020-03-04T15:33:55.6748618Z .................................................................................................... 8500/9728
2020-03-04T15:34:03.4388155Z .................................................................................................... 8600/9728
---
2020-03-04T15:35:55.3390788Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:348:22
2020-03-04T15:35:55.3391262Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2020-03-04T15:35:55.3396656Z 
2020-03-04T15:35:55.3396837Z 
2020-03-04T15:35:55.3403565Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "ui" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-7/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "7.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2020-03-04T15:35:55.3406796Z 
2020-03-04T15:35:55.3406914Z 
2020-03-04T15:35:55.3407218Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2020-03-04T15:35:55.3407609Z Build completed unsuccessfully in 1:04:41
2020-03-04T15:35:55.3407609Z Build completed unsuccessfully in 1:04:41
2020-03-04T15:35:55.3455786Z == clock drift check ==
2020-03-04T15:35:55.3477912Z   local time: Wed Mar  4 15:35:55 UTC 2020
2020-03-04T15:35:55.6460939Z   network time: Wed, 04 Mar 2020 15:35:55 GMT
2020-03-04T15:35:55.6461638Z == end clock drift check ==
2020-03-04T15:35:56.2222710Z 
2020-03-04T15:35:56.2302006Z ##[error]Bash exited with code '1'.
2020-03-04T15:35:56.2315086Z ##[section]Finishing: Run build
2020-03-04T15:35:56.2364147Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/69033/merge to s
2020-03-04T15:35:56.2369185Z Task         : Get sources
2020-03-04T15:35:56.2369512Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-03-04T15:35:56.2369794Z Version      : 1.0.0
2020-03-04T15:35:56.2369991Z Author       : Microsoft
2020-03-04T15:35:56.2369991Z Author       : Microsoft
2020-03-04T15:35:56.2370343Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2020-03-04T15:35:56.2370703Z ==============================================================================
2020-03-04T15:35:56.5467912Z Cleaning any cached credential from repository: rust-lang/rust (GitHub)
2020-03-04T15:35:56.5512608Z ##[section]Finishing: Checkout rust-lang/rust@refs/pull/69033/merge to s
2020-03-04T15:35:56.5620155Z Cleaning up task key
2020-03-04T15:35:56.5621566Z Start cleaning up orphan processes.
2020-03-04T15:35:56.5849731Z Terminate orphan process: pid (3735) (python)
2020-03-04T15:35:56.6071164Z ##[section]Finishing: Finalize Job

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @rust-lang/infra. (Feature Requests)

@Centril Centril removed the I-nominated label Mar 19, 2020
bors bot added a commit to taiki-e/futures-async-stream that referenced this pull request Mar 20, 2020
Merge #34
34: Update for generator resume arguments r=taiki-e a=taiki-e

Refs: rust-lang/rust#69033

Co-authored-by: Taiki Endo <te316e89@gmail.com>
Centril added a commit to Centril/rust that referenced this pull request Mar 21, 2020
…, r=tmandry

Use generator resume arguments in the async/await lowering

This removes the TLS requirement from async/await and enables it in `#![no_std]` crates.

Closes rust-lang#56974

I'm not confident the HIR lowering is completely correct, there seem to be quite a few undocumented invariants in there. The `async-std` and tokio test suites are passing with these changes though.
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 21, 2020
Rollup of 17 pull requests

Successful merges:

 - rust-lang#65097 (Make std::sync::Arc compatible with ThreadSanitizer)
 - rust-lang#69033 (Use generator resume arguments in the async/await lowering)
 - rust-lang#69901 (add #[rustc_layout(debug)])
 - rust-lang#69997 (add `Option::{zip,zip_with}` methods under "option_zip" gate)
 - rust-lang#70038 (Remove the call that makes miri fail)
 - rust-lang#70058 (can_begin_literal_maybe_minus: `true` on `"-"? lit` NTs.)
 - rust-lang#70111 (BTreeMap: remove shared root)
 - rust-lang#70139 (add delay_span_bug to TransmuteSizeDiff, just to be sure)
 - rust-lang#70165 (Remove the erase regions MIR transform)
 - rust-lang#70166 (Derive PartialEq, Eq and Hash for RangeInclusive)
 - rust-lang#70176 (Add tests for rust-lang#58319 and rust-lang#65131)
 - rust-lang#70177 (Fix oudated comment for NamedRegionMap)
 - rust-lang#70184 (expand_include: set `.directory` to dir of included file.)
 - rust-lang#70187 (more clippy fixes)
 - rust-lang#70188 (Clean up E0439 explanation)
 - rust-lang#70189 (Abi::is_signed: assert that we are a Scalar)
 - rust-lang#70194 (#[must_use] on split_off())

Failed merges:

r? @ghost
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 21, 2020
Rollup of 16 pull requests

Successful merges:

 - rust-lang#65097 (Make std::sync::Arc compatible with ThreadSanitizer)
 - rust-lang#69033 (Use generator resume arguments in the async/await lowering)
 - rust-lang#69997 (add `Option::{zip,zip_with}` methods under "option_zip" gate)
 - rust-lang#70038 (Remove the call that makes miri fail)
 - rust-lang#70058 (can_begin_literal_maybe_minus: `true` on `"-"? lit` NTs.)
 - rust-lang#70111 (BTreeMap: remove shared root)
 - rust-lang#70139 (add delay_span_bug to TransmuteSizeDiff, just to be sure)
 - rust-lang#70165 (Remove the erase regions MIR transform)
 - rust-lang#70166 (Derive PartialEq, Eq and Hash for RangeInclusive)
 - rust-lang#70176 (Add tests for rust-lang#58319 and rust-lang#65131)
 - rust-lang#70177 (Fix oudated comment for NamedRegionMap)
 - rust-lang#70184 (expand_include: set `.directory` to dir of included file.)
 - rust-lang#70187 (more clippy fixes)
 - rust-lang#70188 (Clean up E0439 explanation)
 - rust-lang#70189 (Abi::is_signed: assert that we are a Scalar)
 - rust-lang#70194 (#[must_use] on split_off())

Failed merges:

r? @ghost
@bors bors merged commit ef7c8a1 into rust-lang:master Mar 21, 2020
4 checks passed
4 checks passed
pr Build #20200317.57 succeeded
Details
pr (Linux mingw-check) Linux mingw-check succeeded
Details
pr (Linux x86_64-gnu-llvm-7) Linux x86_64-gnu-llvm-7 succeeded
Details
pr (Linux x86_64-gnu-tools) Linux x86_64-gnu-tools succeeded
Details
@jonas-schievink jonas-schievink deleted the jonas-schievink:resume-with-context branch Mar 21, 2020
@jonas-schievink jonas-schievink added this to the 1.44 milestone Mar 21, 2020
@benma

This comment has been minimized.

Copy link

benma commented Mar 22, 2020

Tested in on nightly, works great. Awesome work!

A bit new to Rust release scheduling; will this be in stable with the next release, or do we need to wait until 1.44 is stable, which is not until June?

@jonas-schievink

This comment has been minimized.

Copy link
Member Author

jonas-schievink commented Mar 22, 2020

This will land in 1.44 (and will hit beta in 1.43)

@benma

This comment has been minimized.

Copy link

benma commented Mar 22, 2020

Thx. Any chance to put it into the next stable release? After all, it's an implementation detail, not a language change.

@jonas-schievink

This comment has been minimized.

Copy link
Member Author

jonas-schievink commented Mar 22, 2020

No, we only backport bug fixes

benma added a commit to benma/bitbox02-firmware that referenced this pull request Mar 25, 2020
This is to be able to use async/await syntax, which was merged a few
days ago into nightly for no_std:
rust-lang/rust#69033.

This feature will be in beta in April and stable in June, at which
point we can start to use those channels again.

rust-toolchain file is added as a sanity check or for devs/users not
using Docker.
benma added a commit to benma/bitbox02-firmware that referenced this pull request Mar 25, 2020
This is to be able to use async/await syntax, which was merged a few
days ago into nightly for no_std:
rust-lang/rust#69033.

This feature will be in beta in April and stable in June, at which
point we can start to use those channels again.

rust-toolchain file is added as a sanity check or for devs/users not
using Docker.
@kpp

This comment has been minimized.

Copy link
Contributor

kpp commented Mar 27, 2020

@jonas-schievink Hi. I got a 300% performance degradation on 1.44.0-nightly 2fbb075 2020-03-26 compared to stable 1.42 b8cedc0 2020-03-09 while benching futures.

Criterion output

future::ready/futures   time:   [1.5667 ns 1.5712 ns 1.5762 ns]                                   
                        change: [+223.24% +232.94% +242.84%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) low mild
  5 (5.00%) high mild
future::ready/async_combinators                                                                             
                        time:   [1.5699 ns 1.5743 ns 1.5793 ns]
                        change: [+279.32% +282.34% +285.20%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) low mild
  2 (2.00%) high mild
  3 (3.00%) high severe

future::poll_fn/futures time:   [260.26 ps 260.82 ps 261.40 ps]                                    
                        change: [-3.2265% -2.6712% -2.0788%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  5 (5.00%) high mild
  4 (4.00%) high severe
Benchmarking future::poll_fn/async_combinators: Collecting 100 samples in estimated 5.0000 s (19B iterations                                                                                                            future::poll_fn/async_combinators                        
                        time:   [260.18 ps 260.91 ps 261.77 ps]
                        change: [-1.3517% -0.6763% -0.0422%] (p = 0.05 < 0.05)
                        Change within noise threshold.
Found 10 outliers among 100 measurements (10.00%)
  3 (3.00%) low mild
  4 (4.00%) high mild
  3 (3.00%) high severe

future::map/futures     time:   [2.1487 ns 2.1544 ns 2.1601 ns]                                 
                        change: [+296.27% +298.58% +300.54%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) low severe
  1 (1.00%) low mild
  4 (4.00%) high mild
  1 (1.00%) high severe
future::map/async_combinators                                                                             
                        time:   [3.1330 ns 3.1393 ns 3.1458 ns]
                        change: [+233.70% +236.33% +238.68%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) low mild
  3 (3.00%) high mild

     Running target/release/deps/stream-ff2c5e87168d8d4c
stream::iter/futures    time:   [3.1216 ns 3.1275 ns 3.1341 ns]                                  
                        change: [+290.24% +293.62% +297.39%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
  1 (1.00%) low mild
  6 (6.00%) high mild
  4 (4.00%) high severe
stream::iter/async_combinators                                                                             
                        time:   [3.1331 ns 3.1423 ns 3.1532 ns]
                        change: [+280.94% +283.80% +286.95%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild
  3 (3.00%) high severe

stream::next/futures    time:   [3.3549 ns 3.4282 ns 3.5139 ns]                                  
                        change: [+319.16% +327.38% +336.10%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
stream::next/async_combinators                                                                             
                        time:   [3.1806 ns 3.2330 ns 3.2933 ns]
                        change: [+288.17% +292.97% +297.98%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 13 outliers among 100 measurements (13.00%)
  3 (3.00%) high mild
  10 (10.00%) high severe

stream::collect/futures time:   [6.3038 ns 6.3446 ns 6.3928 ns]                                     
                        change: [+298.95% +301.74% +305.36%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 17 outliers among 100 measurements (17.00%)
  4 (4.00%) low mild
  9 (9.00%) high mild
  4 (4.00%) high severe
Benchmarking stream::collect/async_combinators: Collecting 100 samples in estimated 5.0000 s (456M iteration                                                                                                            stream::collect/async_combinators                        
                        time:   [11.661 ns 12.051 ns 12.435 ns]
                        change: [+320.32% +329.63% +340.16%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

stream::map/futures     time:   [6.8157 ns 6.9007 ns 6.9814 ns]                                 
                        change: [+305.37% +309.91% +314.44%] (p = 0.00 < 0.05)
                        Performance has regressed.
stream::map/async_combinators                                                                             
                        time:   [7.0993 ns 7.2319 ns 7.3715 ns]
                        change: [+339.77% +348.71% +358.18%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

stream::fold/futures    time:   [4.3133 ns 4.3516 ns 4.4018 ns]                                  
                        change: [+277.61% +283.58% +291.19%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild
stream::fold/async_combinators                                                                             
                        time:   [9.3900 ns 9.4145 ns 9.4431 ns]
                        change: [+259.06% +261.65% +264.17%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 13 outliers among 100 measurements (13.00%)
  7 (7.00%) high mild
  6 (6.00%) high severe

I know my benchmarks are not perfect. Still 300% is a lot.

Would you please take a look?

===

Update: my benchmarks turned out to bench nothing.

@goffrie

This comment has been minimized.

Copy link
Contributor

goffrie commented Mar 28, 2020

I was able to reproduce the regression, but it's not from this PR: 5f13820 and 1057dc9 are both equally slow.

@goffrie

This comment has been minimized.

Copy link
Contributor

goffrie commented Mar 28, 2020

Bisecting shows that 57e1da5 introduced the regression.

@Mark-Simulacrum

This comment has been minimized.

Copy link
Member

Mark-Simulacrum commented Mar 28, 2020

Please file new issue(s) for problems arising as a result of PRs, otherwise it's highly likely to get lost.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

You can’t perform that action at this time.