Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix SslStreamStreamToStreamTest to exercise correct overloads #36065

Merged
merged 1 commit into from Mar 15, 2019

Conversation

Projects
None yet
2 participants
@stephentoub
Copy link
Member

stephentoub commented Mar 15, 2019

The SslStreamStreamToStreamTest is set up as a base class from which three test classes derive, one for each of Async, Begin/End, and Sync. But the base class isn't actually deferring to the derived types to customize most of the functionality being executed, namely read/write methods. This PR fixes that, so that the base class properly exercises the relevant methods, customized to the base type.

cc: @davidsh

@davidsh davidsh added this to the 3.0 milestone Mar 15, 2019

Fix SslStreamStreamToStreamTest to exercise correct overloads
The SslStreamStreamToStreamTest is set up as a base class from which three test classes derive, one for each of Async, Begin/End, and Sync.  But the base class isn't actually deferring to the derived types to customize most of the functionality being executed, namely read/write methods.  This PR fixes that, so that the base class properly exercises the relevant methods, customized to the base type.

@stephentoub stephentoub force-pushed the stephentoub:sslstreamtests branch from f3d923b to e3f2882 Mar 15, 2019

@stephentoub stephentoub merged commit 6a9fe42 into dotnet:master Mar 15, 2019

1 check passed

license/cla All CLA requirements met.
Details

@stephentoub stephentoub deleted the stephentoub:sslstreamtests branch Mar 15, 2019

leeohalloran added a commit to leeohalloran/corefx that referenced this pull request Mar 18, 2019

let the latest code (#1)
* Refine help messages (#35688)

* Refine help messages

* Revisit back pressure implementation (#35711)

- Today when using a PipeReader from a Pipe, when the pause threshold is hit and the reader does not consume enough data to unblock the writer (enough to reduce the resume threshold) an exception is thrown. This exception is extremely hard to understand and work around since it's based on a limit set by the owner of the Pipe. It leads to people setting an extremely big limit (bigger than the biggest message) or disabling the limit completely.
- The changes the back pressure mechanic to be based on examined instead of consumed. This pauses the writer if the pause threshold is hit, but lets the reader continue to buffer until a logical payload is reached (whatever matters to the protocol).
- This breaks software that relied on these limits to enforce a maximum buffer size and those pieces of code need to now handle their own maximum buffer size.

* Fix CoreCLR build break

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Update dependencies from https://github.com/dotnet/coreclr build 20190302.72 (#35733)

This change updates the following dependencies
- Microsoft.NET.Sdk.IL - 3.0.0-preview4-27502-72
- Microsoft.NETCore.ILAsm - 3.0.0-preview4-27502-72
- Microsoft.NETCore.Runtime.CoreCLR - 3.0.0-preview4-27502-72

* Update dependencies from https://github.com/dotnet/corefx build 20190303.2 (#35734)

This change updates the following dependencies
- Microsoft.NETCore.Platforms - 3.0.0-preview4.19153.2

* Update dependencies from https://dnceng@dev.azure.com/dnceng/internal/_git/dotnet-optimization build 20190303.1 (#35735)

This change updates the following dependencies
- optimization.windows_nt-x64.IBC.CoreFx - 99.99.99-master-20190303.1

* Move fully managed TimerQueue implementation to shared partition (#7111)

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Update developer-guide to address workflow changes

* OSX to macOS rename in dev guide

* Project msbuild syntax cleanup (#35686)

* Remove license header from non-shipping files

* Remove obsolete Project attributes

* Remove xml header and remove DefaultTargets

* Enable -? help parameter (#35739)

* Avoid NullReferenceException in BinaryObjectInfo (#35740)

A corrupted BinaryFormatter stream can result in a BinaryObjectInfo getting constructed before the ObjectManager is constructed, resulting in the BinaryObjectInfo not having a reference to the ObjectManager.  Attempts to use it then hit a null reference exception.

* Update dependencies from https://github.com/dotnet/corefx build 20190303.8 (#35750)

This change updates the following dependencies
- Microsoft.NETCore.Platforms - 3.0.0-preview4.19153.8

* Update dependencies from https://dnceng@dev.azure.com/dnceng/internal/_git/dotnet-optimization build 20190304.1 (#35752)

This change updates the following dependencies
- optimization.windows_nt-x64.IBC.CoreFx - 99.99.99-master-20190304.1

* Add tests for dotnet/coreclr#20815 - new Math and MathF rounding modes (#33297)

* add tests for new midpointrounding modes

new tests added for Math.cs double Round methods

* add tests for float, decimal

convert to theory

* Update tests

tests now reflect correct rounding behavior

* add tests for existing modes

* code review feedback

- refactor tests to use member data
- move MathF tests to MathF.netcoreapp.cs
- add tests for new modes with digits and special cases

* move math tests to mathtests.netcoreapp.cs

netcoreapp specific tests moved

* Remove SharedFrameworkValidation (#35747)

* Remove code coverage badge till it's working again

* SBCSCodePageEncoding: Don't call DecoderFallback.CreateFallbackBuffer until needed (#35199)

* Don't call DecoderFallback.CreateFallbackBuffer until needed

* Use null instead of fallbackBuffer

* Use C# async iterator to implement ChannelReader.ReadAllAsync (#35745)

While the hand-written implementation is a bit faster than what the compiler currently generates, it's also a lot more code, and complex, error-prone code.  Better to go with the simpler implementation, and work to improve the compiler for the future.

* Adding support for custom ValidationResult (derived class) in CustomValidationAttribute

* Removed extra leading space

* Ignored tests for custom ValidationResults on .NET Framework, since it is not supported there

* removed test from full .net framework

* Update brotli to v1.0.7 (#35172)

* update to brotli v1.0.7

* update WithState perf test to use compressLevel arg

* update brotli-version.txt

* Update adding-api-guidelines.md

* Disable alpine39 (#35770)

* disable alpine3.9

* Adhere to desktop implementation of [Can]ConvertFrom (#35760)

* Adhere to desktop implementation of [Can]ConvertFrom

* Add tests for InstanceDescriptor in TypeConverter

* Update dependencies from https://github.com/dotnet/coreclr build 20190303.72 (#35749)

This change updates the following dependencies
- Microsoft.NET.Sdk.IL - 3.0.0-preview4-27503-72
- Microsoft.NETCore.ILAsm - 3.0.0-preview4-27503-72
- Microsoft.NETCore.Runtime.CoreCLR - 3.0.0-preview4-27503-72

* Update API for new RCW license impl in coreclr (#35767)

* disable AllHelperPortsAreInGetPortNames on ARM Windows (#35773)

* Update dependencies from https://github.com/dotnet/coreclr build 20190304.74 (#35787)

This change updates the following dependencies
- Microsoft.NET.Sdk.IL - 3.0.0-preview4-27504-74
- Microsoft.NETCore.ILAsm - 3.0.0-preview4-27504-74
- Microsoft.NETCore.Runtime.CoreCLR - 3.0.0-preview4-27504-74

* Update dependencies from https://github.com/dotnet/corefx build 20190304.9 (#35788)

This change updates the following dependencies
- Microsoft.NETCore.Platforms - 3.0.0-preview4.19154.9

* Update dependencies from https://dnceng@dev.azure.com/dnceng/internal/_git/dotnet-optimization build 20190305.1 (#35789)

This change updates the following dependencies
- optimization.windows_nt-x64.IBC.CoreFx - 99.99.99-master-20190305.1

* Allow CreateRelationship to work with FileAccess.Write (#35763)

When adding a relationship InternalRelationshipCollection was creating a part
but not writing anything to it.  This caused an empty entry to be created in
the zip archive which later needed to be rewritten.  That's disallowed with
FileAccess.Write.  Instead, avoid creating the entry until it's content is
written.

When writing a relationship file InternalRelationshipCollection was attempting
to truncate any existing content.  This throws when the package is open  with
FileAccess.Write since the part stream is not seekable.  Don't do the seek
when package is open for FileAccess.Write since no pre-existing content is
possible.

In Package.PartExists don't attempt to get the part.  PartExists on .NETCore
only needs to query its local list of parts.  Since it isn't returning a
mutable part, it doesn't need to throw for packages opened with FileAccess.Write.

Avoid out of bounds exception when enumerating part keys. The fix that delays
creating the relationship part until its content exposed an issue in
DoOperationOnEachPart.  This was creating a copy of the part keys, but then
enumerating over the length of original list instead of the copy.  Since I
delayed the creating of a part until it was flushed this made the length change
while iterating the loop.  Since this code was using its copied list of keys
it should have never been using the original bounds.

* Fix Linux chain.ChainStatus when multiple layers have errors

The recent chain rewrite caused only the last layer with errors to be reported.

* Fix issue with BufferSegment reuse (#35779)

- We stored the last examined segment to compute how many bytes we should decrement on the writer side but if that segment is returned to the segment pool, we end up in a bad state. This properly clears the state so that doesn't happen.

* [master] Update dependencies from dotnet/core-setup (#35713)

* Update dependencies from https://github.com/dotnet/core-setup build 20190301.13

This change updates the following dependencies
- Microsoft.NETCore.App - 3.0.0-preview4-27501-13
- Microsoft.NETCore.DotNetHost - 3.0.0-preview4-27501-13
- Microsoft.NETCore.DotNetHostPolicy - 3.0.0-preview4-27501-13

* Remove workaround for System.Runtime.CompilerServices.Unsafe

* Update dependencies from https://github.com/dotnet/core-setup build 20190302.8

This change updates the following dependencies
- Microsoft.NETCore.App - 3.0.0-preview4-27502-8
- Microsoft.NETCore.DotNetHost - 3.0.0-preview4-27502-8
- Microsoft.NETCore.DotNetHostPolicy - 3.0.0-preview4-27502-8

* Update dependencies from https://github.com/dotnet/core-setup build 20190303.9

This change updates the following dependencies
- Microsoft.NETCore.App - 3.0.0-preview4-27503-9
- Microsoft.NETCore.DotNetHost - 3.0.0-preview4-27503-9
- Microsoft.NETCore.DotNetHostPolicy - 3.0.0-preview4-27503-9

* Update dependencies from https://github.com/dotnet/core-setup build 20190304.10

This change updates the following dependencies
- Microsoft.NETCore.App - 3.0.0-preview4-27504-10
- Microsoft.NETCore.DotNetHost - 3.0.0-preview4-27504-10
- Microsoft.NETCore.DotNetHostPolicy - 3.0.0-preview4-27504-10

* Don't include EventRegistrationToken on Unix

This type was accidentally duplicated between CoreLib and contracts.

Suppress that for now since we're detecting the duplication in the shared framework.

Make a fix in the shared items to restore the condition on this type.

* Reduce inner and outer loop test time for S.T.Json (#35776)

* Reduce inner and outer loop test time for S.T.Json

* Remove redundant sequence slicing, only compare the SequencePositions.

* Add more Syndication tests (#35107)

* Add more Syndication tests

* Tweak ImmutableDictionary.TryGetValue to improve throughput (#35759)

* Avoid recursive calls in SortedInt32KeyNode.Search

* Compare first key directly in HashBucket

* Seal a few internal types that have virtual overrides or interface implementations

* Avoid extra level of indirection with Search

Now that it's no longer recursive, we can just inline it and build the caller's decisions right into it.

* Add Alpine arm64 official build (#35778)

* Add Alpine arm64 official build

* Include musl_arm64 in PRs as well

* Fix ArgIterator condition (#35741)

* Cleanup some Syndication code (#35175)

* Fix publish.proj when pushing corefx-test-assets.xml file (#35802)

* Update dependencies from https://github.com/dotnet/coreclr build 20190305.73 (#35803)

This change updates the following dependencies
- Microsoft.NET.Sdk.IL - 3.0.0-preview4-27505-73
- Microsoft.NETCore.ILAsm - 3.0.0-preview4-27505-73
- Microsoft.NETCore.Runtime.CoreCLR - 3.0.0-preview4-27505-73

* replace WaitForPipeDrain with Flush in Kill_EntireProcessTree_True_EntireTreeTerminated test (#35796)

* replace WaitForPipeDrain with FLush

* feedback from review

* Remove dead code from GuidConverter and add tests (#35790)

* Improve X509Chain handling of NotSignatureValid on Linux (#35801)

* Native changes for better NotSignatureValid support

* clang-format -i the modified files

* Tests and managed changes for better supporting NotSignatureValid chains

* Chain the WorkingChain ctors

* Fix tests on macOS (Classic) Sierra

* Update benchmarking.md doc (#35786)

* update doc

* address PR feedback

Co-Authored-By: MarcoRossignoli <marco.rossignoli@gmail.com>

* Update dependencies from https://github.com/dotnet/coreclr build 20190305.75 (#35816)

This change updates the following dependencies
- Microsoft.NET.Sdk.IL - 3.0.0-preview4-27505-75
- Microsoft.NETCore.ILAsm - 3.0.0-preview4-27505-75
- Microsoft.NETCore.Runtime.CoreCLR - 3.0.0-preview4-27505-75

* Fix ValueStringBuilder.EnsureCapacity to call Grow correctly (#35810)

* Fix ValueStringBuilder.EnsureCapacity to call Grow correctly (#35746)

Call to EnsureCapacity may result in invalid buffer size being
allocated.

* Update ValueStringBuilder.Grow to prevent misuse (#35746)

Updated name of the argument to make it clearer on the call-site also
added summary co clearly state the behavior of Grow.

* re-enable MulticastInterface test on RH6, disable on RH7 and OSX (#35798)

* re-enable MulticastInterface test on RH6, disable on RH7 and OSX

* add missing using

* feedback from review

* Reduce overhead of ReadOnlySequence<T>.CopyTo (#35819)

`ReadOnlySequence<T>.Length` isn't as small or fast as `ReadOnlySpan<T>.Length`, which is just a field access.  Since we need the span anyway for the single element case, just get the span first and then compare with its length.

* Clean up lots of string.Format / SR.Format usage (#35777)

* Clean up lots of string.Format usage

This PR does a few things:
- Adds new SR.Format overloads, in particular to support accepting an IFormatProvider as the first argument.
- Fixes a bunch of calls to string.Format that should have been using SR.Format.  This not only helps in AOT builds where we want to be able to more cleanly remove resource strings, it enables easier auditing of the remaining string.Format uses to determine whether there are more appropriate implementations.
- Replaces some string.Format uses, in particular in asserts, with easier to read and maintain string interpolation
- Replaces some string.Format uses with string concatenation where it's simple and cheaper
- Fixes a bunch of SR.Format(SR.Something) calls where the SR.Format part is extraneous and should just be removed.
- Replaces a bunch of Debug.Assert(false, ...) calls with Debug.Fail(...).  This isn't directly related to string.Format, but I started this cleanup as part of cleaning up asserts that were using string.Format, and then ended up accidentally merging the chain into the same commit, and decided it was simple enough that I should just leave it rather than trying to separate it out again.

* Address PR feedback

* Fix breaks due to lower netstandard builds

* Fix ProjectN build breaks (dotnet/corert#7115)

Signed-off-by: dotnet-bot <anirudhagnihotry098@gmail.com>

* Fix issue 23009

Signed-off-by: dotnet-bot <anirudhagnihotry098@gmail.com>

* Move more Assembly methods to shared partition (dotnet/coreclr#22817)

Signed-off-by: dotnet-bot <anirudhagnihotry098@gmail.com>

* Add Type Forward (dotnet/coreclr#23036)

Signed-off-by: dotnet-bot <anirudhagnihotry098@gmail.com>

* Clean up some string.Format usage (dotnet/coreclr#23025)

* Clean up some string.Format usage

Mainly changes some string.Format usage to be SR.Format when working with resource strings.  Also cleans up a few asserts that were using string.Format.

* Address PR feedback and clean up src

* Address more feedback

* Update src/System.Private.CoreLib/shared/System/Type.Enum.cs

Thanks.

Co-Authored-By: stephentoub <stoub@microsoft.com>
Signed-off-by: dotnet-bot <anirudhagnihotry098@gmail.com>

* Fixing the float/double parsing logic to handle exponents that are too large/small (#23043)

* Fixing the float/double parsing logic to handle exponents that are too large/small

* Fixing the debug assert to use >= and <=

* Fixing the max exponents to be +1 to match what scale uses.

Signed-off-by: dotnet-bot <anirudhagnihotry098@gmail.com>

* ignore verification failures on 10.14 (#35829)

* skip tests failing on Windows ARM64 (#35825)

* skip tests failing on Windows ARM64

* fix CanWriteToPerfCounters condition to properly exclude run on arm

* Reduce fix to just S.L.P (#35728)

* disable  tests failing on Windows ARM (#35849)

* Improve JsonSerializer.ReadAsync(Stream) throughput (#35823)

* Improve JsonSerializer.ReadAsync(Stream) throughput

Some low hanging fruit:
- The method was using the Stream.ReadAsync overload that returns a `Task<int>`.  Changed it to take a `Memory<byte>` so that it instead returns a `ValueTask<int>`.
- The method was clearing the full rented buffer upon returning it to the pool, even when only using a small portion of it.  Changed it to only clear what was used.
- The method was resizing the buffer unnecessarily due to faulty logic around how much data remained.  Fixed it to only resize when more than half the buffer is full, which was the original intention.
- The ReadCore method is a bit chunky to call, initializing a new Utf8JsonReader each time, copying large structs, etc.  Since we need to read all of the data from the stream anyway, I changed it to try to fill the buffer, which then minimizes the number of times we need to call ReadCore, in particular avoiding the extra empty call at the end.  We can tweak this further in the future as needed, e.g. only making a second attempt to fill the buffer rather than doing so aggressively.
- Also fixed the exception to contain the full amount of data read from the stream, not just from the most recent call.

* Address PR feedback

* Updating the Microsoft.Private.Intellisense package to 3.0.0-preview3-190305-0 (#35842)

* Disable used-but-marked-unused warnings in System.Security.Cryptography.Native

When building in non-portable mode, some OpenSSL 1.1 function defnitions
that are marked as unused can be picked up by our build. When those
functions are called, clang reports a warning and fails the build:

    src/Native/Unix/System.Security.Cryptography.Native/openssl.c:432:12:
    error: 'sk_ASN1_OBJECT_num' was marked unused but was used [-Werror,-Wused-but-marked-unused]
          return sk_ASN1_OBJECT_num(eku);
                 ^

This 'unused' attribute was recently added to sk_* methods in OpenSSL
1.1: https://github.com/openssl/openssl/pull/8246

Fixes #35807

* Seed the AsnReader/AsnWriter changes from corefxlab

These files are identical to the files at

https://github.com/dotnet/corefxlab/tree/42ab2422b190521dd21c5c9e816810cf23f6a828/src/System.Security.Cryptography.Asn1.Experimental/System/Security/Cryptography/Asn1

though PointerMemoryManager is not included (already present in corefx)

* Update dependencies from https://github.com/dotnet/coreclr build 20190306.73

This change updates the following dependencies
- Microsoft.NET.Sdk.IL - 3.0.0-preview4-27506-73
- Microsoft.NETCore.ILAsm - 3.0.0-preview4-27506-73
- Microsoft.NETCore.Runtime.CoreCLR - 3.0.0-preview4-27506-73

* Add FileStream.CopyToAsync missing override to ref (#35850)

* Rename APIs based on API review (#35848)

See https://github.com/dotnet/corefx/issues/34828

* Temporarily disabling a RealParser test.

* Cleanup Globalization info code (dotnet/coreclr#22591)

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Expose AmbiguousImplementationException and add tests (#35632)

* Expose AmbiguousImplementationException and add tests

Fixes: #34124

* Replace AsnReader/AsnWriter with updated versions, remove unused AsnSerializer

* Copy AsnReader/AsnWriter unit tests from dotnet/corefxlabs.
Remove original AsnReader/AsnWriter/AsnSerializer.

Original commit message from dotnet/corefxlab:

* Tests
  * AssertExtensions.Throws => Assert.Throws
  * Rename private overloads of public methods to *Core to avoid xunit discovery bug
  * Add tests for WriteUtcTime(maxYear)
  * Add tests for the ObjectDisposedException priority

* Change classes in the System.Security.Cryptography.Asn1 namespace to be "internal" instead of "public".
Change comment for AsnReader.ReadNamedBitListValue to allow compilation in project with warnings-as-errors.

* Update unit test names to match API names

* Add shared project for AsnReader/AsnWriter files

* Update AsnXml template to use new API names and `WriteEncodedValue` overload with `Span`

* S.S.C.Asn1: Regenerate AsnXml files

* Use new `AsnReader` API names in common S.S.C code

* S.S.C.Pkcs: Regenerate AsnXml files

* S.S.C.X509Certificates: Regenerate AsnXml files

* Update code to use `AsnWriter.WriteEncodedValue` with `Span`

* S.S.C.Algorithms: Use shared csproj for AsnReader/AsnWriter, add Argument_EncodeDestinationTooSmall resource

* S.S.C.Cng: Use shared csproj for AsnReader/AsnWriter, add Argument_EncodeDestinationTooSmall resource

* S.S.C.Encoding, S.S.C.Encoding.Tests: Use shared csproj for AsnReader/AsnWriter, add Argument_EncodeDestinationTooSmall resource

* S.S.C.OpenSSL: Use shared csproj for AsnReader/AsnWriter, add Argument_EncodeDestinationTooSmall resource

* S.S.C.Pkcs: Use shared csproj for AsnReader/AsnWriter, add Argument_EncodeDestinationTooSmall resource

* S.S.C.Pkcs: Update usage of `AsnWriter.WriteUtcTime` to use the maximum year instead of minimum year

* S.S.C.Pkcs: Use new `AsnReader` API names

* S.S.C.Pkcs: Use `AsnReader.WriteEncodedValue` with `Span`

* S.S.C.X509Certificates: Use shared csproj for AsnReader/AsnWriter, add Argument_EncodeDestinationTooSmall resource

* S.S.C.X509Certificates: Use new `AsnReader` API names

* Add missing CopyToAsync override in FileStream.Unix.cs (#23101)

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Add unit tests for Rune.Decode and friends (#35469)

Also forwards OperationStatus from System.Memory to System.Runtime

* Shrink ReadOnlySequence by 8 bytes (#35860)

* Shrink ReadOnlySequence by 8 bytes

* nit

* Unwrap Start, End when used interally

* Utf8JsonReader.cs - Add support for single line comments ending on \r, \r\n (#33393)

* Add support for single line comments ending on \r, \r\n

* Single Segment done

* Move single segment tests over. Add 2/3 multisegment tests

* Multi-segment tests

* Fix formatting in tests file

* Fix formatting feedback. Fix failing test post-rebase

* Modified logic and added new test cases

* Changes related to review comments

* Fixes to tests

* Remove Verify Info, fix issues

* Fix NETFX_x86 test failure

* Added UserFlags field, unblocks #34560 (#35851)

* Added UserFlags field

* Added UserFlags

* Improve process test fail output (#35853)

* improve test

* cleanup

* add current pid

* address PR feedback

* nit

* Update ProcessTests.cs

* Fix ReflectPropertyDescriptor.IsReadOnly (#35872)

* Fix ReflectPropertyDescriptor.IsReadOnly

* PR Feedback and fix one more place we were fetching nonPublic members when shouldn't

* Add regression test for overriden readonly virtual property

* enable SendToRecvFrom_Datagram_UDP again (#35846)

* enable SendToRecvFrom_Datagram_UDP again and add some basic instrumentation

* increase timeout + feedback from review

* Upgrade Test Sdk to 16.0.1 (#35876)

* Upgrade Test Sdk to 16.0.1

* Remove static dependency

* Remove VB exclusion

* Cleanup StrongNameKeyPair tests and increase coverage (#35856)

* Add dummy object field to TypedReference in ref assembly (#35883)

* Add target to remove binding redirect (#35828)

* Add target to remove binding redirect

* Update name of target to not collide with WindowsRuntime target

* use rented buffers in stream writes
use cached lazy allocated default text stream encoding

* Use platform specific implementation of GetCurrentProcessId (#23084)

* Use platform specific implementation of GetCurrentProcessId

directly in shared location instead of relying on runtime export

* Move GetCurrentProcessId to Interop class

* Update ES_BUILD_STANDALONE build

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Update System.Activity API to be more readonly (#35863)

* Add back in Read-only-ness for ActivitySpanId and ActivityTraceId

I was able to determine a way of casting away readonly-ness (using Unsafe.AsRef),
so I was able to make ActivitySpanId and ActivityTraceId read-only (and cast away
the read-only when updating the cache).   This works better becasuse it allows
refs to these IDs to be readonly and thus prevent update to Activity.TraceId and Activity.SpanId (which we don't want).

Note that for some reason on NetStandard1.3 ref readonly returns fail (says it can't find the 'InAttribute').
This error seems to be bogus, but the upshot is that older standards gives the C# compiler a problem so
we work around it by simply droping the read-only attribute for those (you still have all the functionality).

* Resolved conflicts

* Fix for NetStandard1.3

* Update to make the cache ref-equality friendly.

* Fix a race and inefficiency in Microsoft-Diagnostics-DiagnosticSource (#35764)

* Fix a race and inefficiency in Microsoft-Diagnostics-DiagnosticSource

There is a bridge from DiagnosticSource to EventSoruce called Microsoft-Diagnostics-DiagnosticSource.
This bridge take the data object payload and derializes it int EventSource events.
There was a cache that kept  the type and it cooresponding serialization transform of the last object, which were not atomic and thus could be confused if code was executed concurrently.

In this cache is per EXPLICITLY filtered events, and works well for that case.  If however the transform is implicit, many events can share the same cache slot, and the cache becomes very ineffective.

To fix this we keep the existing one element cache, but we only set it once (which keeps things simple but avoids races and covers the explicitly filtered case), and add a lazily created ConcurrentDictionary for the 'many envent' case.

Also fixes some unrelate test issues (avoided using the same DiagnosticListner names so that tests don't interact when run concurrently),   Fixed issue were a test fails under the debugger (because the debugger turns on Microsoft-Diagnostics-DiagnosticSource which interfers with the test).

I did walk new code to insure that we at least had code coverage for the new code over our set of tests.

* Review feedback.

Basically rather than tolerate races, change to code to remove them.
This makes the code less error prone.

* Fix more nits.

* fix whitespace

* comment and use lambda so c# caches it.

* fix typo

* add in deleted line

* JsonDocument should allow the UTF-8 content-BOM from Stream inputs

* [master] Update dependencies from dotnet/coreclr (#35891)

* Update dependencies from https://github.com/dotnet/coreclr build 20190307.75

This change updates the following dependencies
- Microsoft.NET.Sdk.IL - 3.0.0-preview4-27507-75
- Microsoft.NETCore.ILAsm - 3.0.0-preview4-27507-75
- Microsoft.NETCore.Runtime.CoreCLR - 3.0.0-preview4-27507-75

* Update tests for globalization cleanup

* Fix NegativeSign_SetNull_ThrowsArgumentNullException test with updated arg name

* Update dependencies from https://github.com/dotnet/coreclr build 20190308.76 (#35908)

This change updates the following dependencies
- Microsoft.NET.Sdk.IL - 3.0.0-preview4-27508-76
- Microsoft.NETCore.ILAsm - 3.0.0-preview4-27508-76
- Microsoft.NETCore.Runtime.CoreCLR - 3.0.0-preview4-27508-76

* Update dependencies from https://github.com/dotnet/corefx build 20190308.7 (#35911)

This change updates the following dependencies
- Microsoft.NETCore.Platforms - 3.0.0-preview4.19158.7

* Update dependencies from https://github.com/dotnet/standard build 20190308.1 (#35912)

This change updates the following dependencies
- NETStandard.Library - 2.1.0-prerelease.19158.1

* Update dependencies from https://dnceng@dev.azure.com/dnceng/internal/_git/dotnet-optimization build 20190308.5 (#35914)

This change updates the following dependencies
- optimization.windows_nt-x64.IBC.CoreFx - 99.99.99-master-20190308.5

* Update dependencies from https://github.com/dotnet/core-setup build 20190308.16 (#35910)

This change updates the following dependencies
- Microsoft.NETCore.App - 3.0.0-preview4-27508-16
- Microsoft.NETCore.DotNetHost - 3.0.0-preview4-27508-16
- Microsoft.NETCore.DotNetHostPolicy - 3.0.0-preview4-27508-16

* Allow deserialization for ResourceManager created with non-default resource set type (#23051)

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Remove 2 suffix step 1 (#35904)

* Bug fix: await SmtpClient.SendMailAsync never ends (#35915)

Call InvokeCallback() in case of CompletedSynchronously.

Fixes #35738

* Added new Range Manipulation APIs for Collection<T> which propogate up to ObservableCollection<T>. AddRange, InsertRange, RemoveRange and Replace Range

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Updated index out of range exception check to be a single comparison

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Updated foreach to use T instead of var

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Adding braces on for loop

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Added ReplaceItemsRange API and updated ReplaceRange to invoke the protected method

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Remove ternary operator from invocation as it is the same as inputing items.Count

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Updated RemoveItemsRange to prevent int.MaxValue overflow errors

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Updated RemoveItemsRange to properly remove each item and not just the index items

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Added ArgumentOutOfRangeException exception in RemoveRange if the count is less than 0

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Updated RemoveRange invocation of RemoveAt to use index instead of i, because the array changes with each iteration of the for loop and will cause side-effects which may include index out of range exceptions. This needs to be index because as the array shrinks the index is always at the correct position.

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Updated RemoveAt->RemoveItem. This change removes redundent validation checks that happen using the RemoveAt API. RemoveItem gives us direct access to invoke the command

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Updated InsertItemsRange to simplify the expression and added performance improvements. If the underlying `items` is using List<T> we should use it's InsertRange method since it is optimized, othersie we use InsertItem

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Added new RemoveRange validation check to see if the resulting range (index + count) > items.Count and if it is true throw ArgumentException

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Removed (Int64) cast statements from RemoveItemsRange when checking for overflow errors

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* InsertItemsRange API now checks if the underlying items.IsReadOnly first then collection == null

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Simplified the RemoveItem logic since we have more robust validation

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Optimized RemoveItemsRange to call the list.RemoveRange if the items is of type List<T>

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Move part of RuntimeHelpers to shared partition (dotnet/coreclr#23130)

* Move part of RuntimeHelpers to shared partition

* Remove FormatterServices.cs

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Add stack depth check to all Task continuations (dotnet/coreclr#23152)

Currently Task has a stack depth check that avoids stack overflows on very deep stack continuation chains, but it only applies to Task.ContinueWith, not to other kinds of continuations.  This changes that to have it apply to all.

As part of this, this also deletes the current StackGuard type used to achieve the check.  The type was meant to avoid expensive calls to check where we are on the stack, but now that we're using TryEnsureSufficientExecutionStack, it's actually faster to just call that rather than access the current StackGuard from a ThreadLocal.  This then also cleans up the call sites nicely, as they no longer need finally blocks to undo the increment performed on the StackGuard.

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Move Buffer to shared (dotnet/coreclr#23157)

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Fix NetEventSource logging for Windows auth on Linux (#35918)

While debugging some Linux enterprise auth (Negotiate/NTLM) scenarios, I discovered that the
logging I added in PR #35383 wasn't working. This was due to not passing in a 'null' value
for the context object for NetEventSource.Info. It was using the string object as the
context object and not logging the string data.

As part of fixing that I optimized some code for remembering what auth package (NTLM
or SPNEGO) is used.

* disable test on netfx (#35921)

* disable SendToRecvFrom_Datagram_UDP aagin. (#35920)

* Update dependencies from https://github.com/dotnet/coreclr build 20190309.73 (#35923)

This change updates the following dependencies
- Microsoft.NET.Sdk.IL - 3.0.0-preview4-27509-73
- Microsoft.NETCore.ILAsm - 3.0.0-preview4-27509-73
- Microsoft.NETCore.Runtime.CoreCLR - 3.0.0-preview4-27509-73

* Update dependencies from https://github.com/dotnet/core-setup build 20190309.14 (#35925)

This change updates the following dependencies
- Microsoft.NETCore.App - 3.0.0-preview4-27509-14
- Microsoft.NETCore.DotNetHost - 3.0.0-preview4-27509-14
- Microsoft.NETCore.DotNetHostPolicy - 3.0.0-preview4-27509-14

* Update dependencies from https://github.com/dotnet/corefx build 20190310.1 (#35926)

This change updates the following dependencies
- Microsoft.NETCore.Platforms - 3.0.0-preview4.19160.1

* Fix CompositionContract.Equals for null (#35132)

* Fix CompositionContract.Equals for null

* Move tests to right folder

* Update dependencies from https://github.com/dotnet/corefx build 20190310.5 (#35931)

This change updates the following dependencies
- Microsoft.NETCore.Platforms - 3.0.0-preview4.19160.5

* Update dependencies from https://github.com/dotnet/core-setup build 20190310.07 (#35930)

This change updates the following dependencies
- Microsoft.NETCore.App - 3.0.0-preview4-27510-07
- Microsoft.NETCore.DotNetHost - 3.0.0-preview4-27510-07
- Microsoft.NETCore.DotNetHostPolicy - 3.0.0-preview4-27510-07

* Update dependencies from https://github.com/dotnet/coreclr build 20190310.71 (#35932)

This change updates the following dependencies
- Microsoft.NET.Sdk.IL - 3.0.0-preview4-27510-71
- Microsoft.NETCore.ILAsm - 3.0.0-preview4-27510-71
- Microsoft.NETCore.Runtime.CoreCLR - 3.0.0-preview4-27510-71

* Add await stack guard test (#35905)

* Add await stack guard test

* Fix typo

* Skip new test on netfx

* Refactor Encoding to split fast-path and fallback logic (#23098)

This refactoring is limited to ASCIIEncoding at the moment, but it can easily be applied to UTF-8 / UTF-16 / UTF-32.

High-level changes:
- Fallback logic has been split from the fast-path, improving performance of GetBytes and similar routines.
- All of the plumbing of when to invoke the fallback logic and how to manage leftover data has been moved into the base class.
- Almost all of the logic except for the fast-path is now written in terms of verifiable code (Span and ReadOnlySpan).
- Minor bug fixes in EncoderNLS.Convert (see https://github.com/dotnet/coreclr/issues/23020).

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Update dependencies from https://github.com/dotnet/corefx build 20190310.8 (#35945)

This change updates the following dependencies
- Microsoft.NETCore.Platforms - 3.0.0-preview4.19160.8

* Update dependencies from https://github.com/dotnet/coreclr build 20190310.74 (#35947)

This change updates the following dependencies
- Microsoft.NET.Sdk.IL - 3.0.0-preview4-27510-74
- Microsoft.NETCore.ILAsm - 3.0.0-preview4-27510-74
- Microsoft.NETCore.Runtime.CoreCLR - 3.0.0-preview4-27510-74

* Disable TestCheckChildProcessUserAndGroupIds test (#35949)

Sporadically failing in CI

* Better defaults for the Pipe (#35939)

- Use a 4K buffer instead of 2K (4K is a very common buffer size and usually maps to system page size)
- Use a stack for the buffer segment pool and allow pooling more than 16 segments for large writes

* Update dependencies from https://github.com/dotnet/core-setup build 20190311.01 (#35946)

This change updates the following dependencies
- Microsoft.NETCore.App - 3.0.0-preview4-27511-01
- Microsoft.NETCore.DotNetHost - 3.0.0-preview4-27511-01
- Microsoft.NETCore.DotNetHostPolicy - 3.0.0-preview4-27511-01

* Cleanup and enhance misc reflection/attribute tests (#35858)

* Cleanup and enhance misc reflection tests

* Address PR feedback

* Fix timezone-related failures in System.Runtime.Tests (#35955)

In timezones with a positive offset, several tests were failing due to going below the minimum supported values.

* Add missing deserialization ctor to ChannelClosedException (#35952)

* Use TryAddWithoutValidation in WebSocketHandle.ConnectAsyncCore (#35954)

Don't validate the headers the developer has explicitly added to ClientWebSocketOptions.RequestHeaders; just pass them through.

* Add GNU compiler support to build src/Native (#35737)

* Add GNU compiler support to build src/Native
Tested with gcc v5, v6, v7 and v8 on CentOS 7, Ubuntu 18, Debian 8 and
FreeBSD 11.
`corefx/src/Native/build-native.sh x64 checked gcc`

* Remove unnecessary cast

* Sync gen-buildsys-gcc.sh with CoreCLR

* Throw an exception if examined < consumed and reworked lastExamined (#35879)

- Store a single absolute position instead of the segment
- Fixed some typos and added tests for null segments
- Added more tests

* Inject W3C headers in HTTP diagnostics handler (#35882)

* Inject W3C headers in corefx Htpp Diagnostics HAndler

* Inject W3C headers if activity format is W3C

* fix style and indentation

* review: fix issue

* datastream bugfix first commit

* fix catch (#35970)

* Replace Delegate.CreateDelegate (dotnet/corert#7147)

Replacing this with a pattern that is easier to statically analyze. We use the GetType/GetMethod/CreateDelegate pattern in several places already.

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Fix NRE in EqualStringArrays (dotnet/coreclr#23174)

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Switch from asserts on POSIX_FADV_* to conversions (#35971)

* Don't allocate a stack (#35958)

- Avoid an extra object allocation by using a struct based stack

* Update dependencies from https://github.com/dotnet/corefx build 20190311.10 (#35983)

This change updates the following dependencies
- Microsoft.NETCore.Platforms - 3.0.0-preview4.19161.10

* Update dependencies from https://github.com/dotnet/core-setup build 20190311.16 (#35982)

This change updates the following dependencies
- Microsoft.NETCore.App - 3.0.0-preview4-27511-16
- Microsoft.NETCore.DotNetHost - 3.0.0-preview4-27511-16
- Microsoft.NETCore.DotNetHostPolicy - 3.0.0-preview4-27511-16

* Remove stale TODO

We have no plans to address this further.

* Made some tweaks to the BufferSegmentStack (#35980)

- Remove array covariance checks
- Added another test for pooled segments

* Include correct error code in WebSocket.ThrowOnInvalidState (#35960)

* Include correct error code in WebSocket.ThrowOnInvalidState

* Address PR feedback

* Fix UWP leg

* Fix ImageCharacteristics of PEHeaderBuilder when initialized by CreateLibraryHeader

Fixes: #35758

* Add unit tests for PEHeaderBuilder factory methods

See: #35758

* Update Windows.Compatibility package with references to newer WCF versions.

* Fix SendWithPingUtilityAsync to catch InvalidOperationException (#35992)

* ignore InvalidOperationException from Kill

* ignore InvalidOperationException from Kill

* Pin buffer for NtQuerySystemInformation (#35991)

Fixes #35987

* DataSteam Test change

* address feedback

* Fixing a couple of issues in the Utf8Parsing/Formatting code (#23134)

* Fixing NumberToSingle and NumberToDouble to handle DigitsCount == 0

* Fixing Utf8Formatter.Float to transcode non ASCII bytes

* Responding to PR feedback

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* React to changes through ASCIIEncoding and EncoderNLS
Also create some additional ASCII tests

* Update dependencies from https://github.com/dotnet/coreclr build 20190311.77

This change updates the following dependencies
- Microsoft.NET.Sdk.IL - 3.0.0-preview4-27511-77
- Microsoft.NETCore.ILAsm - 3.0.0-preview4-27511-77
- Microsoft.NETCore.Runtime.CoreCLR - 3.0.0-preview4-27511-77

* make TextDataFeed.DefaultEncoding internal

* DataStream Test change

* Fix JsonReader bug in multi-segment string processing (#35978)

* Update dependencies from https://dnceng@dev.azure.com/dnceng/internal/_git/dotnet-optimization build 20190312.5 (#36006)

This change updates the following dependencies
- optimization.windows_nt-x64.IBC.CoreFx - 99.99.99-master-20190312.5

* Updating to latest WCF version.

* small fixes for ping to properly set state in case of failure (#33621)

* unix ping fixes

* feedback from review and fixes for failing tests

* feedback from review

* more rework on cleanup logic

* add missing check for disposed object in Async path

* remove freebsd part of this change to make it cleaner

* await for all calls of SendPingAsyncCore()

* make arg checking synchronous and stabilize SendPingAsync_InvalidArgs failing on some platforms

* feedback from review

* Formatting change

* Delete XEvent before method exits.

* Update optimization repo url

* Update dependencies from https://github.com/dotnet/core-setup build 20190312.18 (#36008)

This change updates the following dependencies
- Microsoft.NETCore.App - 3.0.0-preview4-27512-18
- Microsoft.NETCore.DotNetHost - 3.0.0-preview4-27512-18
- Microsoft.NETCore.DotNetHostPolicy - 3.0.0-preview4-27512-18

* Add (Try)GetDateTime(Offset) to Utf8JsonReader (#35903)

Add (Try)GetDateTime(Offset) to Utf8JsonReader

This change partially addresses https://github.com/dotnet/corefx/issues/34690.

These methods parse JSON strings to DateTime(Offset) objects according to format `YYYY-MM-DD[Thh:mm[:ss[.s]][TZD]]`

* let ArrayPool clear returned arrays

* Disable tests failing on WSL (#36017)

This is on Windows 10 Version 10.0.17763 Build 17763.

* Add new counter APIs (dotnet/coreclr#23077)

* Refactor EventCounter and add PollingCounter

* Add PollingCounter

* Add IncrementingEventCounter and IncrementingPollingCounter

* Add MetaData API to Counters

* Some more refactoring

* removing commented out code

* some more cleanup

* build fix and addressing some PR comments

* Addressing PR feedback

* use StringBuilder to generate metadata string

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Update dependencies from https://github.com/dotnet/corefx build 20190312.12 (#36007)

This change updates the following dependencies
- Microsoft.NETCore.Platforms - 3.0.0-preview4.19162.12

* Update dependencies from https://github.com/dotnet/coreclr build 20190312.77 (#36009)

This change updates the following dependencies
- Microsoft.NET.Sdk.IL - 3.0.0-preview4-27512-77
- Microsoft.NETCore.ILAsm - 3.0.0-preview4-27512-77
- Microsoft.NETCore.Runtime.CoreCLR - 3.0.0-preview4-27512-77

* Fix missed RelocatedType annotation

[tfs-changeset: 1735855]

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Add OperationStatus-based UTF8 transcoding APIs (dotnet/coreclr#23219)

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Clean up some tests and move to new Azure endpoint (#36018)

This PR changes the Azure test endpoint for HTTP/1.1 and WebSocket tests to use
Azure App Service instead of the classic Azure Cloud Service endpoint. This now
matches the HTTP/2.0 endpoint architecture.

We are deprecating the use of Azure Cloud Service endpoints because they are hard
to deploy and maintain. Azure App Service, on the other hand, provides a lot of benefits
including built in production/staging slots, TLS certificate handling and easier
integration with Azure DevOps deployment models.

There are a few downsides to Azure App Service which are known feature limitations.
Since it uses ARR (reverse proxy), it causes websocket connections to be proxied.
This results in some behavior changes for some edge condition tests we have. For example,
when a websocket handshake fails (due to subprotocol mismatch for example), the client
side doesn't see a TCP disconnect. Instead, due to the reverse proxy, we end up getting
an HTTP status code (like 500). Either way, it is a websocket handshake failure. So, I've
updated a few tests to be less brittle for that. I also opened another issue #36016 to
track moving a few websocket tests to the loopback websocket server which doesn't yet
have full capability.

I also converted an HTTP statusline test to use the loopback server.

* Fix local zip entry header for large files

When opened for Write ZipArchiveEntry needs to write the local header
information to the archive stream upon the first call to write since the
compressed content comes after that and we write directly to the backing stream
rather than buffering in memory.

By default header information only accommodates 32-bit sizes since this is much
more likely and saves space in archives with lots of files and at the first
write we don’t know how big the entry is going to be.

Once the entry is disposed we know the final length and may find it cannot be
represented in 32-bit fields.  It can’t insert the extra bytes it would need
in the header region since it’s already written the compressed data and
streams/files don’t have insertion operations, so instead it sets a bit that
says the entry's size information is in a descriptor at the end of the entry.

The entry was setting the bit that indicated the descriptor existed but didn't
set the version in the header to ZIP64 to indicate wether that descriptor had
64-bit fields.  Additionally we were writing the central-directory header for
the file with a version of ZIP64 so our local header and central header didn't
match.

May ZIP implementations were OK with this, but not all.  Specifically
WindowsBase.dll's implementation of the System.IO.Packaging APIs has a
ZipArchive that insists the local header version must be ZIP64 to have a
64-bit descriptor.  Moreover it also insists that the local-header version must
match the central header version.

To ensure zips created with .NETCore's System.IO.Packaging APIs work with
desktop we should set this version field in the local header.

* Tweak SslApplicationProtocol (#36021)

* Tweak SslApplicationProtocol

- It currently stores a `ReadOnlyMemory<byte>`.  That just adds unnecessary expense: we can instead just store the provided `byte[]`.
- The most common values are those exposed statically: Http2 and Http11, but ToString on those results in creating a new string each time.  Special-case them.
- Constructing an SslApplicationProtocol with a null string results in an ArgumentNullException being thrown with the wrong parameter name.  Fix it.
- Miscellaneous cleanup on the file.

* Address PR feedback

* Update dependencies from https://github.com/dotnet/corefx build 20190313.10 (#36042)

This change updates the following dependencies
- Microsoft.NETCore.Platforms - 3.0.0-preview4.19163.10

* Update dependencies from https://github.com/dotnet/core-setup build 20190314.01 (#36043)

This change updates the following dependencies
- Microsoft.NETCore.App - 3.0.0-preview4-27514-01
- Microsoft.NETCore.DotNetHost - 3.0.0-preview4-27514-01
- Microsoft.NETCore.DotNetHostPolicy - 3.0.0-preview4-27514-01

* Update dependencies from https://dnceng@dev.azure.com/dnceng/internal/_git/dotnet-optimization build 20190314.1 (#36044)

This change updates the following dependencies
- optimization.windows_nt-x64.IBC.CoreFx - 99.99.99-master-20190314.1

* Update dependencies from https://github.com/dotnet/coreclr build 20190313.77 (#36045)

This change updates the following dependencies
- Microsoft.NET.Sdk.IL - 3.0.0-preview4-27513-77
- Microsoft.NETCore.ILAsm - 3.0.0-preview4-27513-77
- Microsoft.NETCore.Runtime.CoreCLR - 3.0.0-preview4-27513-77

* Remove remaining usage of Contract.* (#36002)

Only remaining use of System.Diagnostics.Contracts in corefx is tests for the library itself, and usage of the [Pure] attribute.

* Disable some proxy tests (#36037)

Disable some problematic proxy related tests.

Contributes to issue #32809

* Update ResourceManager tests to account for #35114. (#35811)

* Use ILResourceReference (#36038)

* Fix: System.IO.Ports runtime package signing (#36022)

* temporarily disable publishing

* temporarily disable another publish step

* Move signing to publish step

* re-enable publish steps

* Enable default interfaces unconditionally (#36012)

Corresponds to dotnet/coreclr#23225.

* Remove the workaround to put the IBC files in the right directory

* Rename optimzation package version

* Fix

* Make StackTrace Symbol Caching Aware of Collectible Types (Phase 2) (#35827)

* Update GetSourceLineInfo to consume an assembly:
* Use Assembly as cache key for MetadataReader
* Dynamic and regular assemblies now have the same cache key
* Prevents stale cache entries in the case where a PEFile is unloaded
and replaced by a new one in the same location

dotnet/coreclr#20179

* Remove redundant overload of GetSourceLineInfo
* update comments to reflect new behavior

* Change TryGetReader to use ConditionalWeakTable's atomic Add with callback method.
* cleaning up var names based on feedback

* Fix single character typo. (#36047)

* Optimize recursion of System.Net.Http.HttpRuleParser.GetExpressionLength (#35959)

Optimize recursion of System.Net.Http.HttpRuleParser.GetExpressionLength

* Avoid three expensive allocations in UriHelper (#36056)

In this repro:
```C#
using System;
using System.Diagnostics;
using System.Text;

class Program
{
    static void Main()
    {
        string input = $"param1={GenerateUrlEncoded(40)}&param2={GenerateUrlEncoded(220)}";
        Console.WriteLine("Input length: " + input.Length);
        var sw = Stopwatch.StartNew();
        string result = Uri.UnescapeDataString(input);
        Console.WriteLine("Result length: " + result.Length);
        Console.WriteLine(sw.Elapsed);
    }

    private  static string GenerateUrlEncoded(int rowsCount)
    {
        var sb = new StringBuilder();
        for (int i = 0x100; i < 0x999; i++)
        {
            sb.Append((char)i);
            if (i % 10 == 0) sb.Append('<');
            if (i % 20 == 0) sb.Append('>');
            if (i % 15 == 0) sb.Append('\"');
        }

        string escaped = Uri.EscapeDataString(sb.ToString());
        sb.Clear();
        for (int i = 0; i < rowsCount; i++)
        {
            sb.AppendLine(escaped);
        }

        return sb.ToString();
    }
}
```
on my machine it ends up allocating ~630GB of memory and takes ~14 seconds.

Almost all of that ~14 seconds is spent in gc_heap::allocate_large, and most of that inside memset_repmovs.  This ends up being due to some large allocations being done in a tight loop.

This PR contains three simple fixes that address the majority of the problem.  There's still more that can be done here, but this is the lowest of the low-hanging fruit and makes the biggest impact:
1. In UnescapeString, the previous code was allocating a new char[bytes.Length] for each iteration.  Stop doing that.  Instead, just reuse the same array over and over and only grow it if it's smaller than is needed.
2. In MatchUTF8Sequence, the previous code was allocating a byte[] for each character or surrogate pair.  Stop doing that.  Instead, just use a reusable four-byte segment of stack.
3. In UnescapeString, the previous code was allocating a new UTF8Encoding for each iteration of the loop.  Stop doing that.  The object is thread-safe and can be used for all requests, so we just make it a static.

These changes drop that ~630GB to ~22MB and that ~14s to ~0.05s.

Subsequently, there's more memory-related changes that could be done in this code, e.g. using pooling, addressing some of the other allocation, but I've left that for the future.

* Improve test logging: add process name to log (#35996)

* add process name to log

* updates

* update

* Fix HttpWebRequest to use WebRequest.DefaultWebProxy credentials (#36059)

If the HttpWebRequest is using the default system proxy, we need to pass any proxy
credentials that the developer might have set via the WebRequest.DefaultWebProxy.Credentials
property. This matches .NET Framework behavior.

I tested this manually using Fiddler as the authenticating proxy.

Fixes #36058

* Fix CryptoStream.Dispose to only transform final block when writing (#36048)

* Fix CryptoStream.Dispose to only transform final block when writing

When reading, the final transform isn't required, and using it can result in exceptions when the stream is only partially read.

* Address PR feedback

* Add test for HttpWebRequest default proxy credentials (#36061)

It's easier to simulate default system proxy settings on Linux since it uses environment
variables. Add a test to verify that the proxy credentials are passed from
WebRequest.DefaultWebProxy.Credentials to the system proxy.

Follow-up to PR #36059

* Move TypeLoadException to shared (dotnet/coreclr#23238)

* fix coding-style

* shorter form for TypeName property

* move coreclr-specific ctor to TypeLoadException

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Add TextEquals helper methods to Utf8JsonReader (#35979)

* Add initial impl of ValueEquals with basic tests.

* Rename to TextEquals based on API review feedback.

* Move to separate test file, increase coverage, and fill in missing impl
details.

* Add xml comments and more test cases.

* Re-enable all the tests.

* Update tests to be netstandard compliant.

* Rename some existing tests to be clearer.

* Address PR feedback.

* Return early if we know unescaping won't produce a match.

* More eagerly check the lengths to return mismatch sooner.

* Add tests to improve code coverage and re-write to avoid unreachable
code.

* Fix resource string merge issue.

* Fix test build failures on netstandard (missing implicit string->span
cast)

* Collection<T>: Validate parameters in the public methods (#23166)

I was looking over the new Range methods on `Collection<T>` and it occurred to me that for all existing methods, parameter validation is done in the *public* methods before calling the *protected virtual* methods. This changes the new Range methods to do the same.

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Add test for using SslStream for concurrent read/writes (#36064)

* Fix SslStreamStreamToStreamTest to exercise correct overloads (#36065)

The SslStreamStreamToStreamTest is set up as a base class from which three test classes derive, one for each of Async, Begin/End, and Sync.  But the base class isn't actually deferring to the derived types to customize most of the functionality being executed, namely read/write methods.  This PR fixes that, so that the base class properly exercises the relevant methods, customized to the base type.

* Inject W3C  headers in netfx HTTP diagnostics hook (#35880)

* Inject W3C headers in Http Desktop hook

* do not inject headers if they were injected before

* check that w3c headers are empty when format is Hierarchical

* review comments

* Fix and re-enable some HttpListener authentication tests (#36084)

The Negotiate and NTLM HttpListener tests were disabled due to a behavior of running them
on loopback using an HttpClient with WinHttpHandler. But now the default handler for HttpClient
is SocketsHttpHandler. And it doesn't have this problem with loopback authentication.

Closes #20604

* Update dependencies from https://github.com/dotnet/coreclr build 20190315.74 (#36090)

This change updates the following dependencies
- Microsoft.NET.Sdk.IL - 3.0.0-preview4-27515-74
- Microsoft.NETCore.ILAsm - 3.0.0-preview4-27515-74
- Microsoft.NETCore.Runtime.CoreCLR - 3.0.0-preview4-27515-74

* Update dependencies from https://github.com/dotnet/corefx build 20190314.7 (#36092)

This change updates the following dependencies
- Microsoft.NETCore.Platforms - 3.0.0-preview4.19164.7

* Fix typo (dotnet/coreclr#23300)

.Net -> .NET

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Add SslStream test for "unlocking" after failure (#36094)

* Add SslStream test for "unlocking" after failure

* Address PR feedback

* Http trailer support for HTTP/1.x (#35337)

* http trailer for http/1.1

* address feedback

* address feedback

* pr feedback

* stop blocking context

* fix framework test failiure

* address feedback

* seperate tests

* address some feedback

* more feedback from review

* correct ReadAsync

* feedback from review

* handle short ReadAsync()

* more feedback

* Update dependencies from https://github.com/dotnet/core-setup build 20190315.05 (#36091)

This change updates the following dependencies
- Microsoft.NETCore.App - 3.0.0-preview4-27515-05
- Microsoft.NETCore.DotNetHost - 3.0.0-preview4-27515-05
- Microsoft.NETCore.DotNetHostPolicy - 3.0.0-preview4-27515-05

* Flatten SslStream by removing SslStreamInternal SslState (#35109)

* Still moving sslState

* Initial SslState removed, should build

* Fix tests

* Moved first methods from SslStreamInternal

* moved a few more messages

* Moved more SslStreamInternal into SslStream

* Move half of SslStreamInternal into SslStream now

* Almost all moved other than shutdown

* Removed SslStreamInternal completely

* Final cleanup

* Clean Fakes

* Formatting

* Remove useless properties, remove dispose internal

* Remove HandshakeComplete and IsShutdown, removed methods from the fakes that match

* Added comment on Fake around removing build warnings

* Disable System.Text.Json outerloop tests failing in CI (#36096)

* Make ReallyLargeLookupUTF8 test outer loop

It allocates a huge amount of memory currently.

* Fixing some payload serialization issue on new EventCounters (#23295)

* Fixing some payload serialization issue on new EventCounters

* Fix build error

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Fix casing of SslStream.Implementation.cs file name (#36099)

* Move Enum and CorElementType to shared (dotnet/coreclr#23177)

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Add SslStream test to validate synchronous invocation on underlying Stream (#36097)

* Change SslStreamStreamToStreamTest_Sync to actually be synchronous

* Add SslStream test to validate synchronous invocation on underlying Stream

SslStream doesn't introduce its own asynchrony.  The only reason an asynchronous operation on SslStream should complete asynchronously is because of an asynchronous call made on its underlying stream completes asynchronously.  This adds tests to validate that, e.g. the tests would fail if we started using Task.Run in the SslStream implementation to invoke operations on the underlying stream.

* Update dependencies from https://github.com/dotnet/coreclr build 20190316.72 (#36104)

This change updates the following dependencies
- Microsoft.NET.Sdk.IL - 3.0.0-preview4-27516-72
- Microsoft.NETCore.ILAsm - 3.0.0-preview4-27516-72
- Microsoft.NETCore.Runtime.CoreCLR - 3.0.0-preview4-27516-72

* Update dependencies from https://github.com/dotnet/core-setup build 20190316.06 (#36105)

This change updates the following dependencies
- Microsoft.NETCore.App - 3.0.0-preview4-27516-06
- Microsoft.NETCore.DotNetHost - 3.0.0-preview4-27516-06
- Microsoft.NETCore.DotNetHostPolicy - 3.0.0-preview4-27516-06

* Add (Try)GetDateTime(Offset) to JsonElement

* Fix typo (#36107)

.Net -> .NET

* Fix WebSocketHttpListenerDuplexStream error handling (#36110)

An unobserved websocket task exception was happening when attempting to read/write on
non-existent connections. This code originally came from .NET Framework. It is specific
to Windows only and uses HttpListener as the server for the WebSocket. I was able to
reproduce the problem on .NET Framework as well.

The problem was due to the logic in WebSocketHttpListenerDuplexStream where it was always
creating a TaskCompletionSource and doing a TrySetException even if the error occured on
a synchronous codepath. The ReadAsyncFast (and WriteAsyncFast) methods are always calling
a callback to finish the operation and thus always setting an exception into the task. But
the task was never awaited because the exception was also rethrown from the ReadAsyncFast
method and skips the await.

The fix is to change the logic slightly so that ReadAsyncFast won't rethrow the exception.
It doesn't need to since the exception is already stored in the task. So, awaiting that task
will be fast since it is already completed (faulted with the exception).

In addition to the CI tests, I also verified the fix with the original repro from the issue.

Fixes #29005

* Default Ut8fJsonReader.Read should return false. (#36112)

* Default Ut8fJsonReader.Read should return false.

* Address PR feedback.

* Reduce test string size for TextEquals and mover overflow test to (#36115)

outerloop.

* Make more types eligible for vectorized code paths in MemoryExtensions (#20855)

Types like sbyte, ushort, and other 8-bit / 16-bit elemental types are now directed down the fast vectorized code paths that previously only byte and char were eligible for.

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>

* Unlock SslStream for writes after failed write (#36106)

* Update dependencies from https://github.com/dotnet/coreclr build 20190317.72 (#36120)

This change updates the following dependencies
- Microsoft.NET.Sdk.IL - 3.0.0-preview4-27517-72
- Microsoft.NETCore.ILAsm - 3.0.0-preview4-27517-72
- Microsoft.NETCore.Runtime.CoreCLR - 3.0.0-preview4-27517-72

* Update dependencies from https://github.com/dotnet/core-setup build 20190317.03 (#36121)

This change updates the following dependencies
- Microsoft.NETCore.App - 3.0.0-preview4-27517-03
- Microsoft.NETCore.DotNetHost - 3.0.0-preview4-27517-03
- Microsoft.NETCore.DotNetHostPolicy - 3.0.0-preview4-27517-03

* Disable BuildInvalidSignatureTwice in X509Certificates.Tests on Windows (#36127)

* Utf8JsonReader should be consistent when dealing with more than one JSON payload. (#36114)

* Utf8JsonReader should be consistent when dealing with more than one JSON
payload.

* Resolve PR feedback - update tests (Assert.Equals, move out of loop,
AssertThrows)

* Remove unnecessary assert since AssertThrows validates that anyway.

* Add ref APIs and unit tests for System.Text.Unicode.Utf8 (#34538)

Also provides the implementation of BoundedMemory for checking for buffer overruns during unit testing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.