Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wasm] Fix WebSocket failures on wasm test runs #102366

Merged
merged 6 commits into from
May 24, 2024

Conversation

mkhamoyan
Copy link
Contributor

@mkhamoyan mkhamoyan commented May 17, 2024

OuterLoop tests that are disabled still are running when /p:TestScope=outerloop.
Xharness ignores -notrait category=failing and runs tests cases marked with SkipOnPlatform,ActiveIssue

Fixes #102157

@mkhamoyan
Copy link
Contributor Author

/azp run runtime-libraries-mono outerloop

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@build-analysis build-analysis bot mentioned this pull request May 17, 2024
@mkhamoyan mkhamoyan marked this pull request as ready for review May 17, 2024 11:17
@mkhamoyan
Copy link
Contributor Author

/azp run runtime-libraries-mono outerloop

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@pavelsavara
Copy link
Member

Could you please also fix all the PlatformNotSupportedException in the outer loop ?

@mkhamoyan
Copy link
Contributor Author

/azp run runtime-libraries-mono outerloop

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@mkhamoyan mkhamoyan marked this pull request as draft May 21, 2024 07:33
@mkhamoyan
Copy link
Contributor Author

/azp run runtime-libraries-mono outerloop

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@mkhamoyan
Copy link
Contributor Author

/azp run runtime-libraries-mono outerloop

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@mkhamoyan
Copy link
Contributor Author

mkhamoyan commented May 21, 2024

I find out that -trait and -notrait are respected when passing with outerloop attribute

[OuterLoop("Resource-intensive due to parallel processing", ~TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on browser")]

but ignored when passed like

[OuterLoop]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on browser")]

There are 2 ways to go

  1. One solution is to update these test cases to skip on wasm from outerloop attribute. (This is implemented in this PR).
  2. Fix it in dotnet/arcade repo, I opened a ticket for it Tests marked with OuterLoop attribute ignore -notrait category=failing arcade#14779

@lewing @pavelsavara what do you think ?

@mkhamoyan
Copy link
Contributor Author

/azp run runtime-libraries-mono outerloop

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@mkhamoyan
Copy link
Contributor Author

All test cases throwing PlatformNotSupportedException are now disabled.
System.Linq.Expressions.Tests, System.Linq.Parallel.Tests, System.Threading.Tasks.Parallel.Tests, System.Threading.Tasks.Tests are failing because of timeout

Tests timed out after 1800secs
XHarness exit code: 70 (TIMED_OUT)

@mkhamoyan mkhamoyan marked this pull request as ready for review May 22, 2024 09:00
@akoeplinger
Copy link
Member

The underlying problem is this xharness issue: dotnet/xharness#908

The PR looks good to me, though maybe we should also spend some time to actually fix xharness instead?

@mkhamoyan
Copy link
Contributor Author

mkhamoyan commented May 24, 2024

I'll merge this to reduce pipeline noise until we fix xharness.

@mkhamoyan mkhamoyan merged commit f4c9264 into dotnet:main May 24, 2024
83 of 88 checks passed
Ruihan-Yin pushed a commit to Ruihan-Yin/runtime that referenced this pull request May 30, 2024
Update test cases to skip on wasm from outerloop attribute.
@github-actions github-actions bot locked and limited conversation to collaborators Jun 23, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
arch-wasm WebAssembly architecture
Projects
None yet
Development

Successfully merging this pull request may close these issues.

WebSocket failures on wasm test runs
3 participants