Enable runtime-async for net11.0+#5195
Open
agocke wants to merge 1 commit into
Open
Conversation
Member
|
Just so I understand, this will make it such that the performance tests will be compiled with the new async machinery? Along those lines, if these tests were run on a runtime that has its libraries built with the new runtime-async would the current repo, without this change, use the new async behavior or the old. |
Member
|
We have the update in BDN now, dotnet/BenchmarkDotNet#3078. I will take a look at the impact and we will get this update out as soon as we can. |
Member
Author
|
Is this ready to merge? |
Member
|
Not yet. After the BDN update we are hitting a hang in CI, that I cannot reproduce locally. So work on it has been slow. I am still working through the issue, and will get this PR in ASAP. |
Member
Author
|
@DrewScoggins ping :) |
Member
|
Still working on this. I have hit about three different bugs in BDN that were introduced by the recent async refactor. I think I finally got the last one fixed, but now we are seeing random test failures. I will give an update later today. |
DrewScoggins
added a commit
that referenced
this pull request
May 19, 2026
* Enable runtime-async for net11.0+ * Update BenchmarkDotNet to 0.16.0-nightly.20260518.1249 Bumps BenchmarkDotNet to the latest master nightly, built from dotnet/BenchmarkDotNet@c7632225 ("Disassembly follow jump trampolines (#3136)") and pushed to the benchmark-dotnet-prerelease internal feed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Migrate BenchmarkDotNet.Extensions to BDN 0.16.0 APIs BDN 0.16.0 ('Async Refactor', PR #2958) made breaking API changes that the harness still consumed: - ExporterBase.ExportToLog(Summary, ILogger) was removed; the new abstract ExportAsync uses an internal CancelableStreamWriter that subclasses can't satisfy. PerfLabExporter now implements IExporter directly. - IValidator.Validate returning IEnumerable<ValidationError> was replaced with ValidateAsync returning IAsyncEnumerable<ValidationError>. The four validators (UniqueArguments, TooManyTestCases, NoWasm, MandatoryCategory) use .ToAsyncEnumerable() (transitively from BDN's System.Linq.AsyncEnumerable dep), not async + yield return - the latter produces an AsyncIteratorMethodBuilder state machine that deadlocks with BDN's BenchmarkSynchronizationContext. Switched BenchmarkDotNet.Extensions from netstandard2.0 to net8.0. The netstandard2.0 target was only used to enable an opt-in net472 path, but PERFLAB_TARGET_FRAMEWORKS=net472 is no longer exercised in CI and the new BDN APIs would otherwise require polyfill packages. Removed the net472 package conditional from MicroBenchmarks.csproj and the net472/.NET Framework references from docs/benchmarkdotnet.md and the micro README. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix nullability errors surfaced by net8.0 retarget Switching BenchmarkDotNet.Extensions from netstandard2.0 to net8.0 brought in stricter nullable annotations from the BCL and BDN package, which combined with TreatWarningsAsErrors=true (set in src/Directory.Build.props) turned previously-hidden nullability warnings into build errors. - ValuesGenerator.Dictionary<TKey, TValue>: add 'where TKey : notnull' constraint required by Dictionary<,>. - UniqueArgumentsValidator.BenchmarkArgumentsComparer.Equals: match the IEqualityComparer<T>.Equals(T?, T?) interface signature; handle nulls. - TooManyTestCasesValidator.SkipValidation: parameter must be MemberInfo? since MemberInfo.DeclaringType returns Type?. - DiffableDisassemblyExporter: add null-forgiving (!) on reflection lookups whose return values are intentionally trusted by this type (it's a copy of internal BDN code that operates on known types). - PerfLabExporter: BuildJson can return null; use string?. Verified by running the exact CI commands locally (dotnet restore + build with --framework net11.0) using the SDK from global.json. Build succeeds with 0 errors. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Switch micro entrypoint to RunAsync to avoid discovery-time deadlock BDN 0.16's sync entrypoints (BenchmarkSwitcher.Run / BenchmarkRunner.Run) install BenchmarkDotNetSynchronizationContext (a single-threaded message pump) before benchmark discovery. Discovery executes [ParamsSource] and [ArgumentsSource] callbacks; some perf-repo callbacks do sync-over-async (notably SslStreamTests.GetTls13Support, which calls HandshakeAsync(...) .GetAwaiter().GetResult()). Sync-over-async deadlocks on the single-threaded SyncCtx because the awaited continuation is queued back to a pump that the caller is blocking. Switch Program.Main to async Task<int> and await BenchmarkSwitcher.RunAsync. The async entrypoint never installs BenchmarkDotNetSynchronizationContext on the caller, so discovery runs on the default context and sync-over-async in source callbacks no longer deadlocks. Real benchmark execution still gets the SyncCtx semantics it needs because BDN installs it inside the per-benchmark execute path, not on the entrypoint thread. This is the supported BDN-recommended fix for the discovery-deadlock symptom; no BDN code change is required. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Gate runtime-async behind 'runtimeasync' experiment PR #5195 enabled the runtime-async language feature unconditionally for net11.0+ via `<Features>`. To avoid affecting baseline measurement runs, gate it behind the existing experiment infrastructure so it only takes effect in the dedicated experiment lane. - src/Directory.Build.targets: only set `runtime-async=on` when the `EnableRuntimeAsync` MSBuild property is `true` (in addition to the existing TFM check). - scripts/ci_setup.py: when `--experiment-name=runtimeasync` is passed, emit `EnableRuntimeAsync=true` as an env var so MSBuild picks it up as a property (matches the pattern used by the `jitoptrepeat` experiment). Verified locally: BDN dry runs against the BinaryDataPayload tests succeed both with and without the env var (18/18 benchmarks each). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Andy Gocke <andy@commentout.net> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.