Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync code #1

Merged
merged 144 commits into from
Oct 3, 2023
Merged

Sync code #1

merged 144 commits into from
Oct 3, 2023

Conversation

GeroL
Copy link
Owner

@GeroL GeroL commented Oct 3, 2023

No description provided.

stephentoub and others added 30 commits September 26, 2023 12:11
* Normalize some test naming

* Alphabetize tests

* Improve mistmatched length tests with all positions of the shorter tensor

* Alphabetize methods in TensorPrimitives.cs
* Move TargetsCurrent to net9 and add net8 workload

* Fix version references

* Update src/mono/nuget/Microsoft.NET.Workload.Mono.Toolchain.Current.Manifest/WorkloadManifest.targets.in

* [wasm] build net8 workload

* Update emsdk

* Update current template to reference net9

* Bump 8.0 version used for workloads

* Fix version for latest emscripten packages

* fix typo in 8.0 version used for the workload

* disamiguate templates

* WBT: explicitly use net8.0 projects for template projects

* Update emsdk dependency to get the workload fix

* fix

* Add some addtional workarounds for net8

* Remove extra character

* Fix test

* More wasi fixes

* Add net8 wasi-wasm runtime pack reference

* Add wasi-experimental-net8 workload

* [wasi] Fix use of workload

* [wasm] WBT: Fix test

* wasi: Allow wasi-wasm runtimepacks even when targeting net8

* fix test

---------

Co-authored-by: Ankit Jain <radical@gmail.com>
* implement absent generic ctor param check

* fix code style

* Improve nullability check for generic parameters in ctor

`NullabilityInfoContext.CheckParameterMetadataType` didn't have
code paths for parameters in constructors, leading to wrong
nullability results.

The PR adds a code path for constructor parameters.

Fix #92487

* add tests on nullability of ctors and methods with generic parameters

* fix test issues with AOT trimming
Add blazor hybrid globalization runs. This includes updating the Blazor and iOS test names to take into account hybridGlobalization and setting up a standard for scenario run configs going forward, at least for now. By having the hybridglobaliztion in both the runconfig and the name when different from the default, the names will only update for non-default settings auto-updating PowerBI while the runconfigs will be available whenever necessary.
* JitDump improvements and other cleanups

* More comment cleanups

* Be consistent in capitalization of `GenTree`
Since we store signature types now this bit is no longer necessary.
* Add net8 wasi workload tests

* Update eng/testing/tests.wasi.targets

Co-authored-by: Ankit Jain <radical@gmail.com>

* [wasm] CI: trigger WBT on changes to eng/testing/tests.{browser,wasm,wasi}.targets

* Update eng/testing/tests.wasi.targets

* Alias the net8 runtime pack correctly

---------

Co-authored-by: Ankit Jain <radical@gmail.com>
* Update the Windows ARM64 unwinder

This change updates the Windows ARM64 unwinder to match the current
state in Windows. It contains a fix for a bug that is needed as a basis
for a .NET issue fix.

* Reflect PR feedback
* Vectorize TensorPrimitives.Min/Max{Magnitude}

* Use AdvSimd.Max/Min

* Rename some parameters/locals for consistency

* Improve HorizontalAggregate

* Move a few helpers

* Avoid scalar path for returning found NaN
…ference-packages dotnet/emsdk dotnet/hotreload-utils dotnet/sdk (#92584)

[main] Update dependencies from dotnet/runtime dotnet/source-build-reference-packages dotnet/emsdk dotnet/hotreload-utils dotnet/sdk
- Coherency Updates:
  - runtime.linux-arm64.Microsoft.NETCore.Runtime.ObjWriter: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.linux-x64.Microsoft.NETCore.Runtime.ObjWriter: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.linux-musl-arm64.Microsoft.NETCore.Runtime.ObjWriter: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.linux-musl-x64.Microsoft.NETCore.Runtime.ObjWriter: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.win-arm64.Microsoft.NETCore.Runtime.ObjWriter: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.win-x64.Microsoft.NETCore.Runtime.ObjWriter: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.osx-arm64.Microsoft.NETCore.Runtime.ObjWriter: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.osx-x64.Microsoft.NETCore.Runtime.ObjWriter: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.linux-arm64.Microsoft.NETCore.Runtime.JIT.Tools: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.linux-x64.Microsoft.NETCore.Runtime.JIT.Tools: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.linux-musl-arm64.Microsoft.NETCore.Runtime.JIT.Tools: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.linux-musl-x64.Microsoft.NETCore.Runtime.JIT.Tools: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.win-arm64.Microsoft.NETCore.Runtime.JIT.Tools: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.win-x64.Microsoft.NETCore.Runtime.JIT.Tools: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.osx-arm64.Microsoft.NETCore.Runtime.JIT.Tools: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.osx-x64.Microsoft.NETCore.Runtime.JIT.Tools: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.linux-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.linux-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.linux-musl-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.linux-musl-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.linux-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.linux-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.linux-musl-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.linux-musl-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.win-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.win-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.osx-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.osx-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.osx-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)
  - runtime.osx-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools: from 16.0.5-alpha.1.23452.1 to 16.0.5-alpha.1.23472.1 (parent: Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport)

 - Merge branch 'main' into darc-main-be922536-a638-4652-9241-ddc0996cfe5a
…ive.<arch>.dll on Linux (#92492)

* 92278: add DFEATURE_ISYM_READER definition only for windows

---------

Co-authored-by: Andrey.Kudashkin <Andrey.Kudashkin@russianpost.ru>
Co-authored-by: Jan Kotas <jkotas@microsoft.com>
* Make config binding gen incremental

* Iterate on implementation

* Add incremental tests & driver

* Make incremental tests pass and revert functional regression

* Address failing tests

* Make tests pass

* Suppress diagnostic

* Address feedback on diag info creation

* Refactor member access expr parsing to indicate assumptions

* Address feedback & do misc clean up

* Adjust model to minimize baseline diff / misc clean up
Things that I added:

* Support for `typeof(T) == typeof(Bar)` (this will be useful later, we'll eventually be able to also freeze these).
* Support static interface method calls
* Constrained method calls on valuetypes
* More `ReadOnlySpan` construction patterns, `.Length`
* More indirect load/store support

Contributes to #78681. To full resolve this, we need to fix up things so we can answer `Sse2.IsSupported`.
* Move the IL rewriting for HW intrinsics `IsSuported` calls to `ILProvider` from `RyuJitCompilation`
* Also rewrite constant true/false
…a runtime exception (#91261)

* Enable tests.

* When AOTing, type checks do not fail compilation but create a runtime exception.

* Cleaned up type load error cleaning. TypeLoadException icall now has a message with type name.

* Removed another instance of indiscriminate exception clearing.

* Fixed build warning.

* Using class const instead of string const. Reverted some compile to runtime errors that were not necessary for the unit tests.

* White space.

* Fixed build warning.

* Trying to fix weird AOT errors, fixed type load throw function.

* Fixed build error.

* Special handling for classes that are NULL.

* Providing for a null klass when generating exception.

* Removed flow control directive from macro.

* Fixed stack corruption.

* Attempt to push the correct type onto the stack.

* Fixing uninitialized ins.

* Fixing ro_type.

* Initializing ins.

* Complex cases with type load failures replace method body with a throw.

* Cleaning up superfluous code changes.

* Restored sizeof cosntant on failed types.
…iveness (#92651)

The liveness pass in physical promotion will currently handle any struct
LCL_FLD access of a physically promoted struct as accessing the
remainder. However, if the LCL_FLD only touches promoted fields then the
remainder is not actually used. There was a TODO around this which this
PR fixes as I stumbled upon a case this would improve.
* Add tests for field access on generics

The tests are currently disabled.

* Fix ILC to compile UnsafeAccessorsTests

UnsafeAccessorsTests passes on NAOT.
* initial work with hacks to switch to EtwProvider python script

* move to generated scripts

* Fixes for some link issues

* fix link issue

* adding private etw callback to enable GC events

* Fix x86 build break

* fixing Linux build break

* fixing gcpriv.h

* making minimal typedefs

* FB

* Fix for posix break
* Removed special encoding that was rendering the test logs near
impossible to read properly.

* Adjusted the offending test to print the invalid character's hex code
instead, and fixed it alongside its sibling test because they didn't
handle all correct/incorrect cases properly.

* Added special handling for illegal XML characters in the test results'
XML logs.

* Simplified the sanitizing algorithm to one pass, as per Dan's feedback.
…named runtime packs (#92712)

In 75ee623 the condition in `src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.props` got changed from checking `MonoBundleLLVMOptimizer` to `MonoAOTEnableLLVM` but we weren't setting that property in runtime-official.yml so both jobs produced runtime packs with the same suffix, resulting in the artifact uploads randomly overwriting each other.
davmason and others added 28 commits October 2, 2023 14:22
Tracing heavily increases the time it takes to run Native AOT tests.
We already run these tests in the outerloop and the probability that
any given PR will break these tests is low. Outerloop coverage should
be good enough right now.
#92826)

* Inline some of the options for the new crossgen2_publish project.

* Resolve TODOs in targetingpacks.targets

* Crossgen1 is long gone. Don't try to discover it in our override targets.

* Move ReadyToRun.targets infra into the shared repo infrastructure and have projects automatically opt-in to it instead of the LKG crossgen2 when they are targeting the live build.

* Remove extraneous property set (the same value is calculated automatically already)

* Remove outdated comment.

* Fix NativeAOT and installer legs

* Condition turning off pack downloads based on opt-in to local pack usage.

* Use the LKG host instead of the 7.0 host as the fallback for NativeExports.

* Apply suggestions from code review

Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>

* Remove AdditionalProperties as they aren't needed (global properties on the command line are already transitive)

* PR feedback

* Hook into the targets pipeline to avoid overridding targets for R2Ring projects that reference the live framework packs. Move the "target override" logic back to where we build the runtime pack as that's the only place where we need crossgen2 and can't reference the runtime pack (as we're building it).

* Don't set CoreCLRArtifactsPath manually.

* PR feedback

---------

Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
This weekend's runs hit a bunch of timeouts due to auto-injected CodeQL.
Currently we have multiple separate mechanisms to report information
back from superpmi.exe:
1. -baseMetricsSummary/metricsSummary, which outputs a .csv file with
   aggregated statics for all contexts from the perspective of the base
   JIT
2. -diffMetricsSummary, which is the corresponding for the diff JIT
   when diffing
3. -diffsInfo, which during diffing will output a .csv with individual
   rows for every context that had diffs in it

This PR replaces these three mechanisms with a -details argument. When
passed, superpmi.exe will write a .csv file to the specified path that
contains a row for every context.

The arg is supported in both replay and diff mode but creates .csv files
with slightly different formats for these. For replays the header output
is:
```
Context,Context size,Result,MinOpts,Size,Instructions
```

For diffs the output is:
```
Context,Context size,Base result,Diff result,MinOpts,Has diff,Base size,Diff size,Base instructions,Diff instructions
```

superpmi.py is changed to utilize this new output instead, which
involves computing some of the same details we were getting from the
metrics summaries before.

Prerequisite for #85755
…92882)

* SIMD regs are now zeroed with movi instead of eor.

* Simplified vector length selection.
Co-authored-by: Egor <egorbo@Egors-MacBook-Pro.local>
Co-authored-by: Jakob Botsch Nielsen <Jakob.botsch.nielsen@gmail.com>
* Improve throughput / allocations of JsonNode.GetPath

The current implementation is creating a `List<string>` and appending each segment to it, which in most of the cases is allocating a `string`. Then it iterates through that list in reverse order appending to a newly-created `StringBuilder`, which it then `ToString`s. In this change, it instead just uses `ValueStringBuilder`, appending to it as it goes.

In doing so, it does reverse the order of enumeration. Previously each node would effectively do:
```C#
void GetPath()
{
    AddNode(this);
    parent?.GetPath();
}
```
and now it's doing:
```C#
void GetPath()
{
    parent?.GetPath();
    AddNode(this);
}
```
While C# doesn't emit tail calls, with optimizations enabled, it's feasible the JIT might emit the recursive call as a jmp rather than a call, in which case it would avoid possible stack dives. However, that's not guaranteed, and doesn't happen today in tier 0 and other unoptimized code. On top of that, to get such a deep nesting in a JsonNode, you need to either go out of your way to create one manually using the JsonNode/Object/Array/Value constructors, or you need to use JsonSerializer.Deserializer, overriding its default MaxDepth, and in the case of a really deep input, it's also recursive and will stack overflow in smaller situations. I have a different version of this change that keeps the same ordering, passing around a span and a length separately, and prepending to the end of the span, but it results in more complicated code, so I'd prefer this variation that just uses ValueStringBuilder unless we have real concerns.

* Address PR feedback
…nals build 20231002.3 (#92936)

Microsoft.SourceBuild.Intermediate.source-build-externals
 From Version 9.0.0-alpha.1.23475.2 -> To Version 9.0.0-alpha.1.23502.3

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
* [wasm] Ignore empty `$ASPNETCORE_URLS`

* [wasm] DevServer: honor urls specified in the options

* [wasm] CI: Don't trigger non-wbt jobs on wasm-app-host changes

* CI: don't trigger wasm runtime tests on wasm-app-host changes
* Add a way to support operations that can't be vectorized on netstandard

* Updating TensorPrimitives.Log2 to be vectorized on .NET Core

* Update src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.netstandard.cs

Co-authored-by: Stephen Toub <stoub@microsoft.com>

* Ensure we do an arithmetic right shift in the Log2 vectorization

* Ensure the code can compile on .NET 7

* Ensure that edge cases are properly handled and don't resolve to `x`

* Ensure that Log2 special results are explicitly handled.

---------

Co-authored-by: Stephen Toub <stoub@microsoft.com>
This allows analysis of inline array access operations, by
treating them similarly to array access. However, like
ILLink/ILCompiler it doesn't understand inline array creation, so
doesn't track them as arrays. The result is that values read out
of an inline array are unknown, so this produces dataflow
warnings when such a value is passed to a location with dataflow
requirements, matching the ILLink/ILCompiler behavior.

Using `InlineArray` required referencing a more recent of the
.NET 8 reference assemblies.

Fixes #88684
Microsoft.Dotnet.Sdk.Internal
 From Version 9.0.100-alpha.1.23479.5 -> To Version 9.0.100-alpha.1.23502.7

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
* Add testing for #92539.

* Remove netfx test skips.
* Fix illink task lock during live build

Fixes #92126

* Update illink.targets
…2903)

* CI: Don't trigger runtime pipelines on perf pipeline only changes
* address review feedback from @ cincuranet
* Condition the use of NetCoreAppPrevious TFM

NuGet doesn't support duplicate TFMs in the TargetFrameworks string.
Condition the use of NetCoreAppPrevious TFMs until NuGet supports that
(which is planned afaik).

* Fix ODBC project TFM
…icate some SIMD code (#90764)

* Adding an internal ISimdVector`2 interface

* Move LastIndexOfValueType to use ISimdVector`2

* Fix a couple minor whitespace nits and remove an unnecessary local
…92953)

Failing test: `System.Numerics.Tensors.Tests.TensorPrimitivesTests.ConvertToHalf_SpecialValues`

Issue: #92885
The JIT has some backwards compatibility for accessing unaligned float
fields on ARM32. With physical promotion, we can end up with some new
patterns that we didn't handle. Expand the pattern matching to handle a
constant address unaligned address.

Fix #92382
* CI: runtime-wasm-perf: add triggers for running on PRs

This is useful to prevent perf pipeline from breaking when changes are
made in `dotnet/runtime`.

* CI: Add run-scenarios-job.yml to list of perf pipeline specific files
* Adding a vectorized implementation of TensorPrimitives.Log

* Make sure to hit Ctrl+S
@GeroL GeroL self-assigned this Oct 3, 2023
@GeroL GeroL merged commit 2f1a106 into GeroL:main Oct 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment