Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dotnet runtime master ingestion into Utf8String experiment #17

Merged
merged 182 commits into from
Jul 17, 2020

Conversation

joperezr
Copy link
Member

No description provided.

BrzVlad and others added 30 commits July 13, 2020 12:04
* [interp] Add intrinsic for Marvin.Block

* [interp] Intrinsify ConvertAllAsciiCharsInUInt32ToUppercase

* [interp] Intrinisfy UInt32OrdinalIgnoreCaseAscii

* [interp] Intrinsify UInt64OrdinalIgnoreCaseAscii

* [interp] Add intrinsic for UInt32ToDecStr

* [interp] Add intrinsic opcode for WidenAsciiToUtf16

* [interp] Rename intrinsics to interp-intrins

There seem to be some build failures on windows. Probably because of conflict with intrinsics source files from mini.

* [interp] Add comments

To prevent bcl and interp intrinsics going out of sync.
New property EnableMultipleHttp2Connections on WinHttpHandler enables multiple HTTP/2 connection to the same server.

Contributes to #35088
Added EncodeString method that accounts for codepoint boundaries to
Base64Stream and QEncodedStream. Used EncodeString instead of
EncodeBytes where strings were encoded. Refactored EncodeBytes while
its logic remained the same. Extracted encoding logic into ByteEncoder
class.

Fix dotnet#1485
* [browser][wasm][tests] JavaScript Interop Marshal tests

* Add primitive and string marshal tests

* Add tests for object identity across marshaling calls for JS object and managed objects.

- Tests to make sure the objects stay the same and are not created new.

* Add tests and code cleanup

* Add marshal of js function tests

* Add delegate marshaling test

* Fix License text

* More tests

* Fix an error where `mono_method_resolve` is called before BINDING object was initialized

* Add more marshal tests

* Add more tests

* Address review comments
* enable GS Cookie on OSX

* Actual fix
* Add always trimmed attributes list for browser config

Example of size reduction on SPC with the most minimalistic Hello
World sample.

| System.Private.CoreLib | Size (kB) | Delta |
|-|-|-|
| Original | 1358 kB | - |
| Trimmed | 1271 kB | - 87 kB |

* Review feedback

* Combine and embed the ILLink.LinkAttributes.xml file into the assembly.

* Update linker command line to ignore attributes file.
Add ExcludeFromCodeCoverageAttribute.

* PR feedback.
Add note about Obsolete.
Only use assembly fullname="*" for compiler generated attributes.

* Update ILLink.LinkAttributes.wasm.xml

Co-authored-by: Eric Erhardt <eric.erhardt@microsoft.com>
…38957)

* DotProduct

* DotProductBySelectedQuadruplet

* MultiplyRoundedDoublingAndAddSaturateHigh

* MultiplyRoundedDoublingAndAddSaturateHighScalar

* MultiplyRoundedDoublingAndSubtractSaturateHigh

* MultiplyRoundedDoublingAndSubtractSaturateHighScalar

* MultiplyRoundedDoublingBySelectedScalarAndAddSaturateHigh

* MultiplyRoundedDoublingBySelectedScalarAndSubtractSaturateHigh

* MultiplyRoundedDoublingScalarBySelectedScalarAndAddSaturateHigh

* MultiplyRoundedDoublingScalarBySelectedScalarAndSubtractSaturateHigh
Co-authored-by: Mitchell Hwang <mitchell.hwang@microsoft.com>
Co-authored-by: Steve Pfister <steve.pfister@microsoft.com>
Co-authored-by: Alexander Köplinger <alex.koeplinger@outlook.com>
* Add documentation about libraries features switches

* Review feedback

* Casing tweaks
…te (#39111)

* Inlined GC Polls for call to methods with SuppressGCTransitionAttribute.

* Emit inlined GC Polls for methods with SuppressGCTransitionAttribute
  when possible and when optimizing.

* Emit only one GC poll per basic block.

* Move insertion of GC polls to a new phase `fgInsertGCPolls` that runs after
  most optimizations so that we don't insert unnecessary GC polls.

* I plan to delete `fgCreateGCPolls` phase that was previously used to insert
  GC polls for platforms that don't support hijacking in a subsequent PR.
  We currently don't support such platforms.

* Fix `fgCreateGCPoll` to be able to insert inlined GC polls for `BBJ_NONE` and
`BBJ_THROW` basic blocks.
* Added field support to JSON serializer

* Addressed issues

* Added tests

* Addressed issues
…(#39199)

Once we release the semaphore, we no longer have ownership over _sendBuffer, so we have to release the latter before not after releasing the semaphore.
* add write queue to Http2Connection to optimize write handling

* address feedback

* remove AsyncMutex

* remove AsyncMutex for real

* use Channel<T> instead of custom queue

* make channel options static

Co-authored-by: Geoffrey Kizer <geoffrek@windows.microsoft.com>
- coreclr/build-test.sh now has a new subcommand: 'mono_aot', which builds a
new target, named 'MonoAotCompileTests', added to
coreclr/tests/src/runtest.proj. This target compiles the runtime tests using
Mono LLVM AOT in a simple configuration where the host platform is identical to
the target platform. Parallel compilation happens via a hack: actual
compilation happens in mono/msbuild/aot-compile.proj, a single-target msbuild
file, and runtest.proj invokes this single-purpose project and target using
batching to create multiple parallelizable instances of this project. Future
work: use the MonoAOTCompiler custom task currently used to build the iOS
sample program.

- Avoid using the runtimeVariant string when defining
coreClrProductArtifactName in mono/templates/xplat-pipeline-job.yml. There are
no "runtime variants" of CoreCLR configured with this parameter; instead,
depend on a shared non-runtime-variant build of CoreCLR.

- Mark function DISubprograms as local definitions--this is an LLVM 9
compatibility fix.

- Use --tag=CXX when linking libmonosgen-2.0.so via libtool when LLVM is linked
into Mono.

  This makes libtool use the C++ compiler driver when linking Mono--which uses
whatever platform-specific flags are necessary to link against the C++ stdlib.
Previously, libtool would use the C compiler driver, which didn't do this and
would produce shared objects with no explicit dependency on libstdc++.

  This problem is normally masked because of the very lax dynamic linking
semantics on ELF, but Mono on our CI setup is built in a CentOS 7 image (which
does not contain a C++11 libstdc++) that has a GCC 7 compatibility package
installed, along with a clang 9 installation that detects headers from the GCC
7 compatibility package. This compatibility package includes a libstdc++ linker
script that links C++11 libstdc++ components statically into the target while
dynamically linking against components present in pre-C++11 libstdc++. The end
result of all of this is that Mono built with this configuration will
dynamically depend on C++11 libstdc++ symbols that should have been statically
linked into the library, and will outright fail to run on machines without a
newer version of libstdc++ available.

- Add tests that fail after LLVM AOT compilation to issues.targets.
Minimum viable support for portable pdb generation from ilasm.

Supporting the following features of the portable pdb format
- Portable PDB CodeView debug directory entry
- Document table
- MethodDebugInformation table
- Local scope table
- Local variable table

This change extends the metadata writer to be able to write the portable pdb format.
- All changes related to portable PDB metadata generation should be controlled with preprocessor directive FEATURE_METADATA_EMIT_PORT_PDB
- New metadata code is not included in the runtime, as the runtime uses a managed parser when it works with portable pdbs

Authored-by: Ivan Povazan <ivan@raincode.com>
This library was added to .NETStandard2.1 and thus can no-longer be
changed.
They try to test various conditions that only work if you have multiple threads which is not the case on WebAssembly.
… globalization (#39228)

Allows the test suite to pass on WebAssembly: `Tests run: 692, Errors: 0, Failures: 0, Skipped: 15. Time: 1.237011s`
* WASM: Fix System.Diagnostics.TraceSource tests

One test was using `Assembly.GetEntryAssembly()` which returns null on WebAssembly.
Others are testing that the output contains the stacktrace but it is empty on WASM right now.

* PR feedback

Co-authored-by: Stephen Toub <stoub@microsoft.com>

Co-authored-by: Stephen Toub <stoub@microsoft.com>
We only support `Dns.GetHostName()` since that didn't throw in the older mono WebAssembly release and can be redirected to `Environment.MachineName`.
vargaz and others added 6 commits July 17, 2020 12:28
…on wasm. (#38991)

* Add a --enable-minimal=eventpipe configure option. Disable eventpipe on wasm.

* Address review comments.

* Disable FeaturePerfTracing for wasm.

* Disable calls to ep_init()/ep_cleanup() as well.
* Small performance improvment

* Update Matrix3x2 as well
* Add telemetry to System.Net.NameResolution

* Rename hostName to hostNameOrAddress

* Use constants for event IDs

* Address PR feedback

* Reduce overhead when only the counter is needed

* Make NameResolution Telemetry an activity

* Remove redundant check

* Fix indentation

* Check IsEnabled before AfterResolution to help linker

* Add Telemetry tests

* Throws => ThrowsAny
@joperezr joperezr merged commit 82c3b4c into dotnet:Utf8String Jul 17, 2020
@joperezr joperezr deleted the Utf8String-ingestion branch July 17, 2020 22:03
scalablecory pushed a commit that referenced this pull request Sep 22, 2020
Syncs a lot of Linux code refactoring and fixes.
runtimelab-bot pushed a commit that referenced this pull request May 18, 2021
…2769)

Transition to GC Unsafe mode on every MONO_RT_EXTERNAL_ONLY function in
reflection.c

In particular, fix mono_reflection_type_from_name which is used in
https://github.com/xamarin/xamarin-android/blob/681887ebdbd192ce7ce1cd02221d4939599ba762/src/monodroid/jni/embedded-assemblies.cc#L350

Fixes stack traces like

```
05-14 08:06:12.848 31274 31274 F DEBUG   :       #00 pc 00000b99  [vdso] (__kernel_vsyscall+9)
05-14 08:06:12.848 31274 31274 F DEBUG   :       #1 pc 0005ad68  /apex/com.android.runtime/lib/bionic/libc.so (syscall+40) (BuildId: 6e3a0180fa6637b68c0d181c343e6806)
05-14 08:06:12.848 31274 31274 F DEBUG   :       #2 pc 00076511  /apex/com.android.runtime/lib/bionic/libc.so (abort+209) (BuildId: 6e3a0180fa6637b68c0d181c343e6806)
05-14 08:06:12.848 31274 31274 F DEBUG   :       #3 pc 0002afcd  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonodroid.so (xamarin::android::internal::MonodroidRuntime::mono_log_handler(char const*, char const*, char const*, int, void*)+141) (BuildId: 9726f32ad5f8fa5e7c5762baf2f6e3294da41cc1)
05-14 08:06:12.848 31274 31274 F DEBUG   :       #4 pc 00112c5d  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (eglib_log_adapter+141) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #5 pc 00020fdf  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (monoeg_g_logv+175) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #6 pc 0002113a  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (monoeg_g_log+42) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #7 pc 00128892  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_threads_transition_do_blocking+258) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #8 pc 0012a406  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_threads_enter_gc_safe_region_unbalanced_with_info+134) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #9 pc 0012a27e  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_threads_enter_gc_safe_region_internal+46) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #10 pc 000799a7  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_loader_lock+71) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #11 pc 000447a1  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_class_create_from_typedef+129) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #12 pc 0003c073  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_class_get_checked+99) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #13 pc 0003cc0f  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_class_from_name_checked_aux+735) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #14 pc 00037989  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_class_from_name_checked+73) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #15 pc 000cc5f4  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_reflection_get_type_internal+132) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #16 pc 000c9bce  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_reflection_get_type_with_rootimage+126) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #17 pc 000ca204  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (_mono_reflection_get_type_from_info+292) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #18 pc 000ca06e  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_reflection_type_from_name_checked+334) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #19 pc 000c9f01  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_reflection_type_from_name+49) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #20 pc 0001b40b  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonodroid.so (xamarin::android::internal::EmbeddedAssemblies::typemap_java_to_managed(char const*)+427) (BuildId: 9726f32ad5f8fa5e7c5762baf2f6e3294da41cc1)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #21 pc 0001b551  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonodroid.so (xamarin::android::internal::EmbeddedAssemblies::typemap_java_to_managed(_MonoString*)+113) (BuildId: 9726f32ad5f8fa5e7c5762baf2f6e3294da41cc1)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #22 pc 000211a7  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonodroid.so (xamarin::android::internal::MonodroidRuntime::typemap_java_to_managed(_MonoString*)+39) (BuildId: 9726f32ad5f8fa5e7c5762baf2f6e3294da41cc1)
```
runtimelab-bot pushed a commit that referenced this pull request Jun 22, 2022
* Initial implementation for contract customization

fix build errors

Move converter rooting to DefaultJsonTypeInfoResolver so that it can be used standalone

Fix ConfigurationList.IsReadOnly

Minor refactorings (#1)

* Makes the following changes:

* Move singleton initialization for DefaultTypeInfoResolver behind a static property.
* Consolidate JsonSerializerContext & IJsonTypeInfoResolver values to a single field.
* Move reflection fallback logic away from JsonSerializerContext and into JsonSerializerOptions

* Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializerOptions.cs

* remove testing of removed field

Simplify the JsonTypeInfo.CreateObject implemenetation (#2)

* Simplify the JsonTypeInfo.CreateObject implemenetation

* Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.cs

* Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.cs

Co-authored-by: Krzysztof Wicher <mordotymoja@gmail.com>

Co-authored-by: Krzysztof Wicher <mordotymoja@gmail.com>

Tests and fixes for JsonTypeInfoKind.None

TypeInfo type mismatch tests

Allow setting NumberHandling on JsonTypeInfoKind.None

test resolver returning wrong type of options

JsonTypeInfo/JsonPropertyInfo mutability tests

rename test file

Move default converter rooting responsibility behind DefaultJsonTypeInfoResolver (#3)

* Move default converter rooting responsibility behind DefaultJsonTypeInfoResolver

* address feedback

Add simple test for using JsonTypeInfo<T> with APIs directly taking it

fix and tests for untyped/typed CreateObject

uncomment test cases, remove todo

More tests and tiny fixes

Add a JsonTypeInfoResolver.Combine test for JsonSerializerContext (#4)

* Fix JsonTypeInfoResolver.Combine for JsonSerializerContext

* Break up failing test

Fix simple scenarios for combining contexts (#6)

* Fix simple scenarios for combining contexts

* feedback

JsonSerializerContext combine test with different camel casing

Remove unneeded virtual calls & branching when accessing Get & Set delegates (#7)

JsonPropertyInfo tests everything minus ShouldSerialize & NumberHandling

Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs

Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs

throw InvalidOperationException rather than ArgumentNullException for source gen when PropertyInfo.Name is assigned through JsonPropertyInfoValues

tests for duplicated property names and JsonPropertyInfo.NumberHandling

Add tests for NumberHandling and failing tests for ShouldSerialize

disable the failing test and add extra checks

disable remainder of the failing ShouldSerialize tests, fix working one

Fix ShouldSerialize and IgnoreCondition interop

Add failing tests for CreateObject + parametrized constructors

Fix CreateObject support for JsonConstructor types (#10)

* Fix CreateObject support for JsonConstructor types

* address feedback

Make contexts more combinator friendly (#9)

* Make contexts more combinator friendly

* remove converter cache

* redesign test to account for JsonConstructorAttribute

* Combine unit tests

* address feedback

* Add acceptance tests for DataContract attributes & Specified pattern (#11)

* Add private field serialization acceptance test (#13)

* tests, PR feedback (#14)

* PR feedback and extra tests

* Shorten class name, remove incorrect check (not true for polimorphic cases)

* Make parameter matching for custom properties map property Name with parameter (#16)

* Test static initialization with JsonTypeInfo (#17)

* Fix test failures and proper fix this time (#18)

* Fix test failures and proper fix this time

* reinstate ActiveIssueAttribute

* PR feedback and adjust couple of tests which don't set TypeInfoResolver

* fix IAsyncEnumerable tests

* Lock JsonSerializerOptions in JsonTypeInfo.EnsureConfigured()

Co-authored-by: Eirik Tsarpalis <eirik.tsarpalis@gmail.com>

Co-authored-by: Eirik Tsarpalis <eirik.tsarpalis@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet