Add mono interpreter test leg to CI #35568
Conversation
|
Tagging subscribers to this area: @ViktorHofer |
steveisok
force-pushed
the
enable-interp-ci-test-lane
branch
from
28fde18
to
d483311
Compare
| # - Windows_NT_x64 | ||
| #- OSX_x64 | ||
| #- Linux_arm64 | ||
| - Linux_x64 |
There was a problem hiding this comment.
It makes me wonder if we should add a new platform-matrix and consume more resources by adding more legs. Should we instead introduce a new multi-helix-job project that calls into sendtohelix.proj multiple times with different variables in parallel?
Just throwing some ideas here, that will also simplify the build whenever we move mono to build and run tests as part of the same build leg, so that way we wouldn't need to build the tests twice, etc.
There was a problem hiding this comment.
I think there are efficiencies to be had for sure. It would be nice if we could run multiple jobs like you describe and still maintain a 'separate leg' view in the UI. I feel it's important to have the interpreter be first class / distinct in that regard.
We definitely should discuss / throw out more ideas.
There was a problem hiding this comment.
jobs like you describe and still maintain a 'separate leg' view in the UI
I don't think that would be possible, but I think I wouldn't mind having one leg for the mono tests and then having multiple test modes in that single run leg. our infra should be resilient to display the test results correctly for that.
Runtime tests do that, they send regular runs and non tiered compilation runs I believe.
There was a problem hiding this comment.
Discussed offline and @steveisok is going to open a issue tracking this and we can do it in a follow up PR.
There was a problem hiding this comment.
What is the tracking issue? We should add it to the change here for tracking.
eng/testing/tests.props
Outdated
| @@ -18,13 +18,20 @@ | |||
| <_withoutCategories Condition="'$(ArchiveTests)' == 'true'">$(_withoutCategories);IgnoreForCI</_withoutCategories> | |||
| <_withoutCategories Condition="'$(TestScope)' == '' or '$(TestScope)' == 'innerloop'">$(_withoutCategories);OuterLoop</_withoutCategories> | |||
| <_withoutCategories Condition="!$(_withCategories.Contains('failing'))">$(_withoutCategories);failing</_withoutCategories> | |||
| <MonoEnvOptions Condition="'$(MonoEnvOptions)' == '' and '$(MonoEnableInterpreter)' == 'true'">--interpreter</MonoEnvOptions> | |||
There was a problem hiding this comment.
Is there a extensibility point to append to this env options? Like if I want to debug tests?
There was a problem hiding this comment.
Yeah, that would be one. We may also want to provide extra params to the runtime. Enable logging, for example.
There was a problem hiding this comment.
Let's introduce a variable or use this one to add the mono run settings.
src/libraries/System.Net.ServicePoint/tests/ServicePointManagerTest.cs
Outdated
Show resolved
Hide resolved
steveisok
force-pushed
the
enable-interp-ci-test-lane
branch
from
dc44a77
to
ea31a99
Compare
| @@ -7,6 +7,8 @@ parameters: | |||
| isOfficialBuild: false | |||
| liveRuntimeBuildConfig: '' | |||
| runtimeFlavor: 'coreclr' | |||
| runtimeDisplayName: 'coreclr' | |||
| interpreter: '' | |||
There was a problem hiding this comment.
As we discussed offline, maybe it makes sense to use the already known runtimeMode as a switch instead of adding a boolean flag: https://github.com/dotnet/runtime/blob/master/eng/pipelines/runtime.yml#L746
This change enables the interpreter on CI as well as providing an option for the local test run.
…d be visible both when running on helix and local. Added issue for ServicePoint test
steveisok
force-pushed
the
enable-interp-ci-test-lane
branch
from
45358d0
to
3622d83
Compare
There was a problem hiding this comment.
As discussed offline let's address: https://github.com/dotnet/runtime/pull/35568/files#r423356379 in a follow up PR.
ghost
locked as resolved and limited conversation to collaborators
This change enables the interpreter on CI as well as providing an option for the local test run.