Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sfh fix tests #8

Merged
merged 339 commits into from Jun 23, 2022
Merged

Sfh fix tests #8

merged 339 commits into from Jun 23, 2022

Conversation

deeprobin
Copy link
Owner

No description provided.

SingleAccretion and others added 30 commits June 13, 2022 21:01
* Use signature types when building the ABI info

This will allow us to let "mismatched" struct types as call arguments,
which in turn is expected to simplify and de-pessimize much of the code
working around the need to keep precise handles on struct nodes.

Credit to @jakobbotsch for being able to make this change.

* Work around the "InferOpSizeAlign" problem

Handling it generally requires solving a larger problem
of "eeGetArgSizeAlignment" not working well on ARM.

(See also the use of "eeGetArgSizeAlignment" in "lvaInitUserArgs")
* [wasm] Build WasmAppHost with UseAppHost=false

- WasmAppHost is included in the WebAssembly.Sdk pack, and doesn't have a
platform specific package.
- Since, this is being built for x64, the build defaults to using the
app host, which means that we get a single binary file that can be run
directly.
    - But this also means that the binary included in the
    WebAssembly.Sdk platform-agnostic package will target the platform
    where it was built.

- Instead, build with UseAppHost=false, and update the wasm host to use
the managed assembly instead.

And update the RunCommand to use `dotnet exec WasmAppHost.dll`.

* fix
On ARM64 it is expensive to materialize the address of an indirection
cell since that requires multiple instructions. This is particular a
problem when CFG is enabled where validation + call uses the indir cell
twice. This changes the CFG logic to create a local in this case.

We also were forcing the indir cell argument node into the register.
This is not ideal for this particular case, so remove this logic and let
LSRA deal with it. ARM still needs this logic.

Fix dotnet#65076
* Handle "mis-sized" structs in XARCH PUTARG_STK codegen

Previously, for structs which were not a multiple of TARGET_POINTER_SIZE
in size:

1) On x86, we copied them to a local in morph. This was a small CQ issue.
2) On Unix x64, we didn't handle them at all and could read out of bounds
   in "genStructPutArgUnroll".

This change fixes both issues by properly detecting cases where we need to
be careful and updating codegen to use correct load sizes.

* Add a test
The wcscpy_s in ClrDataAccess::GetRegisterName was failing with an invalid parameter exception because
the prefixLen and regLen didn't include the terminating null wchar.
…t#70595)

* [wasm] Enable some previously failing WBT/AOT tests on windows

Fixes dotnet#61725 .

* [wasm] WBT: Fix a failing satellite assembly test

The test fails because it is trying to find the string `got: こんにちは`
in the app output, but it sees `?????`. This is due to xharness not
setting the console output to UTF8, and then the tests not doing the
same.

Instead of that, we can move the check to be in the app itself, thus
removing the need for any of this.

Fixes dotnet#58709 .
A variable initialized to false and never modified means we can remove the
variable and any code that depends on that variable not being false.

This is a consequence of 15ab9f9, where code
that assigned the variable was removed.
* Avoid throwing Win32Exception from HTTP authentication

When server sends malformed NTLM challenge the NT authentication processing
would throw an unexpected Win32Exception from HttpClientHandler.Send[Async]
calls. This aligns the behavior to WinHTTP handler where the Unauthorized
reply with challenge token is returned back to the client.

Similarly, failure to validate the last MIC token in Negotiate scheme could
result in Win32Exception. Handle it by throwing HttpRequestException instead.

* Make the unit test more resilient

* Add trace to Negotiate authentication

* Dispose connection instead of draining the response

* Remove outdated ActiveIssue
* Fix build detection of GSS_KRB5_CRED_NO_CI_FLAGS_X

* Don't use GSS_KRB5_CRED_NO_CI_FLAGS_X on NTLM

* Handle GSS_S_UNAVAILABLE error from gss_set_cred_option (NTLM wrapped in Negotiate)

* Make the GSSAPI shim work with krb5 1.13

* Preserve the gss_acquire_cred minor status
The IndexOfHtmlDecodingChars method was iterating character by character looking for either a `&` or a surrogate, but then the slow path if one of those is found doesn't special-case surrogates.  So, we can just collapse this to a vectorized `IndexOf('&')`, which makes the fast path of detecting whether there's anything to decode much faster if there's any meaningful amount of input prior to a `&`. (I experimented with also using `IndexOf('&')` in the main routine, but it made cases with lots of entities slower, and so I'm not including that here.)
* first working version

* Skip check of VN hoisting

* Account for duplicate blocks

* clean up

* wip

* isCommaTree && hasExcep

* revert lsra changes

* Update hoisting condition

- Only update if node to be hoisted has side-effects and the sibling that is before that throws exception

* Change to BasicBlockList

* organize preheaders

* update hoistedInCurLoop and hoistedInSiblingLoop

* Reverse the loop order

* cleanup and jit-format

* Revert "Minor fix to display IG01 weight correctly"

This reverts commit 757120e863b2da188db2593da1b7142fd1ecf191.

* simplify code

* Remove m_hoistedVNInSiblingLoop

* Add back ResetHoistedInCurLoop

Fix igWeight

* Remove reversal of loop processing order

* jit format

* Experimental: Also generate PerfScore:

* review feedback

* fix the superpmi script

* Revert superpmi asmdiffs change

* Rename method

* Add a comment
* Move help link parsing to managed

* Cleanup wtoi and nativeIsDigit

* Fix metasig declaration

* Guard GetHelpContext under FEATURE_COMINTEROP

* Remove definition of GetHelpLink and guard more methods under cominterop

* DWORD should be uint

* Add help context marshaling test

* Adjust marshaling test

* Fix #ifdef with ILLink

* Fix method signature mismatch

* Specify string marshaling

* Throwing test should not test successful HRESULT

* Implement ISupportErrorInfo

* Test interface in InterfaceSupportsErrorInfo
* Use regex matching for Xsltc test baseline files

* Delete xsltc.exe tests entirely
* Reenable 4244 warning for Mono.

* Enable compiler errors for gcc/clang.

* Fix issues enabling -Werror=implicit-int-conversion

* Revert turning on implicit-int-conversion as error.
  This check was triggering failures on parts of the PAL that
  require additional scrutiny beyond the scope of this work.
…otnet#64128)

* align memory load calculation in GC with Docker and Kubernetes

* pass inactive file field name as a parameter
* Fix cmake error

   ```
   Manually-specified variables were not used by the project:
   CONFIGURATION_WASM_OPT_FLAGS
   ```

* Build the interpreter with -O1 on Wasm in Debug configs

   Otherwise `interp_exec_method` and `generate_code` can easily overflow the stack in some browsers with even a few recursive calls (for example during .cctor initializaiton)
…#70694)

* [wasm] Make runtime_is_initialized promise callbacks one-shot

Throw if runtime_is_initialized_resolve or runtime_is_initialized_reject is called more than once

* Add a slightly generalized GuardedPromise<T> object

   Protects against multiple-resolve, multiple-reject, reject after resolve and resolve after reject.

  Does not protect against the executor throwing.
… protocol. (dotnet#69072)

* Implement get bytes from loaded_files using debugger protocol.

* fix pdb size == nul

* Adressing @radical comments.

* Fix build.

* fix compilation

* Addressing @radical comments.

Co-authored-by: Ankit Jain <radical@gmail.com>
* Add doc on Unix temporary file security practice

* Update unix-tmp.md

* Update unix-tmp.md

* Add example permissions encoding

* Update docs/design/security/unix-tmp.md

Co-authored-by: Dan Moseley <danmose@microsoft.com>

* Update unix-tmp.md

Co-authored-by: Dan Moseley <danmose@microsoft.com>
…otnet#70592)

* ConfigurationBinder.Bind on virtual properties duplicates elements

* simplify GetAllProperties

* return array instead of list
According to my comparative perf measurements composite framework
spends about 5M more instructions in the method

coreclr.dll!CMiniMdTemplate<CMiniMd>::SearchTableForMultipleRows(CMiniColDef sColumn)

I tracked this down to the R2R table AttributePresence which is
used to speed up attribute queries against the ECMA metadata model.
In composite mode we were putting the table in the image header,
not component header, and so the runtime was unable to locate it.
This change fixes generation of the table in Crossgen2; I have
verified that with this change I no longer see the 5M outlier.

Thanks

Tomas
panguye and others added 26 commits June 22, 2022 13:39
* Fix ILGenerator maxstack computation

When generating a dynamic method both with many unconditional jumps, the
ILGenerator's computation of maxstack is way to large - it adds the max
stack of each basic block ending with an unconditional transfer
(br, ret, throw, etc). If the generated code has many such bbs
(eg, from the "then" clauses of if-then-else constructs) this sum can
overflow 2^16. When it does, the maxstack recorded/used is the computed
value mod 2^16. When that is less than the correct value, the
JIT throws an InvalidProgramException.

Keep track of the stack depth at various positions and use it to
calculate an adjustment to the depth.

Fix dotnet#63805

* Fix System.Linq.Expressions tests

* Remove stack handling in MethodBuilder
This commit contains fixes for various bugs exposed by enabling fake
hot/cold splitting on ARM64:
- Branches between hot/cold sections are now always long.
- The pseudoinstruction for loading a constant from the cold section
did not support loading 16-byte data into vector registers, as it
temporarily loaded the constant into an 8-byte integer register. Now,
16-byte constants are loaded directly into vector registers via an
`ld1` instruction.
- Asserts/NYIs blocking hot/cold splitting on ARM64 have been removed.

Fake hot/cold splitting requires we fake unwind info by treating each
split function as one hot section. A more architecture-agnostic
approach for this has been applied. To facilitate this approach, the
fake-splitting implementation has been revised to place the hot
and cold sections contiguously in memory (immediately followed
by the read-only data section on ARM64).
The previous code was iterating character by character through the name, looking for the last occurrence of a slash and the last occurrence of a period.  For the slash, we can just LasIndexOfAny.  For the period, it was only doing this in order to trim off ".exe" if a period was found and the text at that point was ".exe"... so we can just compare the end with ".exe".
… ITrigonometricFunctions (dotnet#71033)

* Exposing AcosPi, AsinPi, Atan2Pi, AtanPi, CosPi, SinPi, and TanPi for ITrigonometricFunctions

* Adding tests for AcosPi, AsinPi, Atan2Pi, AtanPi, CosPi, SinPi, and TanPi

* Fixing the handling of AcosPi, AsinPi, and AtanPi tests on WASM/Unix
* Add tests for IMallocSpy
* Make RateLimitPartition.Factory public

* Fix tests

* Fix comment
…net#71141)

Bumps [Newtonsoft.Json](https://github.com/JamesNK/Newtonsoft.Json) from 9.0.1 to 13.0.1.
- [Release notes](https://github.com/JamesNK/Newtonsoft.Json/releases)
- [Commits](JamesNK/Newtonsoft.Json@9.0.1...13.0.1)

---
updated-dependencies:
- dependency-name: Newtonsoft.Json
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
)

There is a crash in `Exception.CaptureDispatchState` when called from one thread at the same time another calls into `Exception.RestoreDispatchState`. The reason for the crash is due to the way we do not update `foreignExceptionFrames` in a thread-safe way.  `foreignExceptionFrames` is used in both methods and can crash when the size changes before the array is copied.

The fix copies `foreignExceptionFrame` into a local variable in `CaptureDispatchState`

Fixes dotnet#70081
* Get cacheLevel for unix

* Get cacheLevel for windows

* Get cacheLevel for os

* add the macro CHECK_CACHE_SIZE

* fix multiplier

* Review comments

* Fix linux arm64 build

* Change from std:string to char[]

* minor nit

* opps should be outside the loop

* Replace assert with _ASSERTE

* CHECK_CACHE_SIZE -> UPDATE_CACHE_SIZE_AND_LEVEL

* Use GetTotalProcessorCount for windows
…tnet#71116)

- Separate out previous custom marshalling support into _V1 classes
- Add basic support for new stateless marshaller shape for values

This only adds really basic support for stateless value marshalling. It doesn't deal with support for caller-allocated buffer, pinning, guaranteed unmarshal, or linear collection marshalling.
this is to fix dotnet#70231.

for regions we could run into this situation -

object is the last object before heap_segment_allocated (hs)
T0 calls NextObj, gets next obj which starts at heap_segment_allocated (hs)
T1 changes ephemeral_heap_segment to hs
T0 does these comparisons
        (nextobj >= heap_segment_allocated(hs) && hs != hp->ephemeral_heap_segment) ||
        (nextobj >= hp->alloc_allocated))
both still false because alloc_allocated hasn't been changed just yet (and the old alloc_allocated is larger than nextobj)

T0 validates next obj, concludes its m_alignpad is not 0, asserts

T1 forms an allocation context starting at heap_segment_allocated, clears memory so by the time the dump is taken, m_alignpad is already cleared (actually we clear it in a_fit_segment_end)

I'm fixing this by saving the ephemeral_heap_segment and alloc_allocated and bail if nextobj is not on the saved eph seg or if those 2 saved values are no long in sync.
… validation) (dotnet#71135)

* Regression test for recursive generics causing runtime stack overflow

* Adjust load levels in SVM validation to prevent infinite recursion

* Fix C# source code per hez2010's PR feedback

* Address David's PR feedback

* Fix TryResolveVirtualStaticMethodOnThisType per David's feedback
…t#71038)

* Change double to decimal in timestamp conversions to preserve precision

* Re-enable disabled test

* Adjust assert message

* Reuse FileSystemInfo Last*TimeUtc fields, use implicit cast operator to store as DateTimeOffset.

* Fix using wrong fieldname for ctime.

* Using 'G' for decimal to string conversion. Adjust test to not require a dot. The DateTimeOffset comparison done afterwards should suffice.

* Ensure timestamps are converted to long, not int.

* Add Epochalypse and Past-octal-limit timestamp tests

* TarHelpers methods can be private

* Remove unreachable code for adding ctime and atime before writing Pax entry. Add tests to ensure we always add those entries to the dictionary on construction.

* Fix typo in test when retrieving ctime.

* Make sure CTime adds nanoseconds on Unix when retrieving info from disk.

Co-authored-by: carlossanlop <carlossanlop@users.noreply.github.com>
* Use IndexOfAny in HttpListener.HandleAuthentication

Just cleaning up a manual loop that can instead be IndexOfAny.

* Fix missing check
…dotnet#71101)

* Change SString::GetUTF8NoConvert to GetUTF8 that converts the SString

This enables SString to get out of the UTF16 state and helps move us away from "SString's natural encoding is UTF16"

* Remove some stack scratch buffers that are unneeded now (as we can now convert the SString itself and in some cases it was already in UTF8) and implement PR feedback.

* Add SetAndConvertToUTF8 method and remove the GetUTF8 method variants with a scratch buffer.

* Remove unneeded variables.

Co-authored-by: Jan Kotas <jkotas@microsoft.com>

Co-authored-by: Aaron Robinson <arobins@microsoft.com>
Co-authored-by: Jan Kotas <jkotas@microsoft.com>
We currently need to pad OSR funclet frames with the full size of the
Tier0 frame, and this paddign sits above the register save area. For
large Tier0 frames we thus need to probe the stack in the OSR prolog
for type 1, 3, and 5 frames, before doing the initial SP adjustment and
save.

Longer term the plan is to revise arm64 OSR so we will not need to
do this padding; at that point we can remove this extra probing.

Fixes dotnet#70263.
Microsoft.Diagnostics.NETCore.Client change to allow future branches to build correctly in runtimelab
…feedback (dotnet#71017)

Co-authored-by: Aaron Robinson <arobins@microsoft.com>
Co-authored-by: Elinor Fung <elfung@microsoft.com>
* Implement UnixFileMode APIs on Unix.

* Throw PNSE on Windows, add UnsupportedOSPlatform.

* Fix API compat issue.

* Borrow a few things from SafeFileHandle API PR to this compiles.

* Fix System.IO.FileSystem.AccessControl compilation.

* Add xml docs.

* Replace Interop.Sys.Permissions to System.IO.UnixFileMode.

* Throw PNSE immediately on Windows.

* Add ODE to xml docs of methods that accept a handle.

* Don't throw (PNSE) from FileSystemInfo.UnixFileMode getter on Windows.

* Minor style fix.

* Get rid of some casts.

Co-authored-by: Eric Erhardt <eric.erhardt@microsoft.com>

* Add tests for creating a file/directory with UnixFileMode.

* Some CI envs don't have a umask exe, try retrieving via a shell builtin.

* Update expected test mode values.

* Fix OSX

* Fix Windows build.

* Add ArgumentException tests.

* Fix Windows build.

* Add get/set tests.

* Update test for Windows.

* Make setters target link instead of link target.

* Linux: fix SetUnixFileMode

* Fix OSX compilation.

* Try make all tests pass in CI.

* For link, operate on target permissions.

* Skip tests on Browser.

* Add tests for 'Get' that doesn't use a 'Set' first.

* Don't perform exist check for handles.

* Fix Get test for wasm.

* Review xml comments.

* Add comment to test.

* GetUnixFileMode for handle won't throw UnauthorizedAccessException.

* Apply suggestions from code review

Co-authored-by: Eric Erhardt <eric.erhardt@microsoft.com>

* PR feedback.

* Update enum doc to say 'owner' instead of 'user'.

* Use UnixFileMode in library.

* Use UnixFileMode in library tests.

* Fix Windows build.

* Fix missing FileAccess when changing to FileStreamOptions API.

* PR feedback.

* Fix Argument_InvalidUnixCreateMode message.

Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>

Co-authored-by: Eric Erhardt <eric.erhardt@microsoft.com>
Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>
…t#71182)

This is enabled for release runtime only, same as how we do it for regular CoreCLR builds.
@deeprobin deeprobin merged commit 5776aad into deeprobin:issue-20234 Jun 23, 2022
deeprobin added a commit that referenced this pull request Jun 23, 2022
This reverts commit 5776aad.
@deeprobin deeprobin mentioned this pull request Jun 23, 2022
deeprobin added a commit that referenced this pull request Jun 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment