Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add new Pipeline for Running Libs Tests with TestReadyToRun #91229

Merged
merged 54 commits into from
Dec 14, 2023
Merged

Add new Pipeline for Running Libs Tests with TestReadyToRun #91229

Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
58471ac
New beginning for the TestReadyToRun for the libraries.
ivdiazsa Aug 28, 2023
d424419
EXPERIMENTING: Added script extensions, archive extensions, and other…
ivdiazsa Aug 28, 2023
cbae83d
EXPERIMENTING: Forgot to add some curly braces...
ivdiazsa Aug 28, 2023
bbcbda5
Recovered some variables that were being eaten and removed some other
ivdiazsa Aug 28, 2023
74c393d
EXPERIMENTING: Added OSX and Windows to see if the CI will deign run
ivdiazsa Aug 28, 2023
f39ec24
EXPERIMENT ONLY: Added some missing parameters for
ivdiazsa Aug 29, 2023
e76d7b2
Is this it?????
ivdiazsa Aug 29, 2023
800bf85
Now that we know the libraries tests can be built properly, let's add
ivdiazsa Aug 30, 2023
4888506
Discovered that when packs are involved, everything has to be in the
ivdiazsa Aug 30, 2023
f95be94
Disabled the R2R-incompatible tests when TestReadyToRun is present.
ivdiazsa Aug 31, 2023
ce0beb7
Changed 'TargetFrameworks' to use 'net8.0' instead of '$(NetCoreAppCu…
ivdiazsa Sep 5, 2023
084dcd2
CI versioning nonsense again...
ivdiazsa Sep 5, 2023
d6145d9
Merge branch 'main' into r2r-libs-take-3
ivdiazsa Sep 5, 2023
7a98a59
Still does not work. Just saving my changes.
ivdiazsa Sep 19, 2023
57a1ed0
Merge with main
ivdiazsa Sep 19, 2023
9c9845c
Merge with main
ivdiazsa Sep 27, 2023
201c744
Merge branch 'main' into r2r-libs-take-3
ivdiazsa Sep 29, 2023
0e5318e
Merge branch 'main' into r2r-libs-take-3
ivdiazsa Oct 4, 2023
94b209d
Merged with Main AND IT WORKED LOCALLY!
ivdiazsa Oct 19, 2023
baf03ff
Updated the 'extraSteps' and 'extraStepsTemplate' to the new
ivdiazsa Oct 20, 2023
bd083f9
Reenabled all the pipelines and moved the TestReadyToRun one to the
ivdiazsa Oct 20, 2023
70a5310
Removed repeated arguments from the post build steps.
ivdiazsa Oct 20, 2023
29bdc40
Reverted weird change in System.Runtime/Directory.Build.props
ivdiazsa Oct 23, 2023
0dafa7a
Removed unnecessary import in Build.proj
ivdiazsa Oct 23, 2023
0987abf
As per Jeremy's feedback, now trying to set the TestReadyToRun template
ivdiazsa Oct 23, 2023
c050621
Reenabled the problematic tests to see if they still fail. If yes, then
ivdiazsa Oct 23, 2023
21f95ef
Merge branch 'main' into r2r-libs-take-3
ivdiazsa Oct 24, 2023
59af6bb
Found a workaround for the libraries tests!
ivdiazsa Oct 30, 2023
52875a7
Forgot to specify the test group for the new pipelines.
ivdiazsa Oct 30, 2023
43eee05
Helix doesn't want to work so how about we try running them as we
ivdiazsa Oct 30, 2023
4666ac0
Disabled tests incompatible with TestReadyToRun.
ivdiazsa Oct 30, 2023
79a9028
Changed the test disabling mechanism to the already defined
ivdiazsa Oct 31, 2023
e95e20d
Moved the 'UseLocalAppHostPack' setting to eng/tests.singlefile.targets.
ivdiazsa Oct 31, 2023
e857281
Temporarily restored 'UseLocalAppHostPack' to
ivdiazsa Nov 1, 2023
d193ab5
Extended the TestReadyToRun pipelines timeout because building all
ivdiazsa Nov 2, 2023
7a3a1d5
Testing removing the .dbg files since those are responsible for most
ivdiazsa Nov 8, 2023
cd94d5a
Removed the debugging files from the "To Copy" lists, rather than
ivdiazsa Nov 10, 2023
3c54a84
Replaced Remove+Condition with Remove->WithMetadataValue for better
ivdiazsa Nov 10, 2023
879e5ea
Merge branch 'main' into r2r-libs-take-3
ivdiazsa Nov 14, 2023
11cdb6b
Tried adding to pretest setup as per Viktor's suggestion.
ivdiazsa Nov 14, 2023
19bb85f
Testing something...
ivdiazsa Nov 20, 2023
81cb9e3
Fixed the R2R-precompiled pack not built on time.
ivdiazsa Nov 21, 2023
47760b3
Moved the R2R-precompiled runtime generation from Subsets.props to pr…
ivdiazsa Nov 22, 2023
0728170
Added the ArchiveTests property to get the test artifacts packed
ivdiazsa Nov 27, 2023
4262460
Excluded dotnet-host installers in TestReadyToRun builds because we
ivdiazsa Nov 27, 2023
403a5f9
Disabled System.Runtime.Loader.Tests because they can't be run due to
ivdiazsa Nov 30, 2023
797f2c0
Disabled the other faulty System.Runtime.Loader* test, and linked to a
ivdiazsa Nov 30, 2023
048d35e
Fixed the live-ref-pack getting removed from publish files.
ivdiazsa Dec 5, 2023
9939472
Added a copy function
ivdiazsa Dec 6, 2023
033508d
Forgot to actually set the files to copy.
ivdiazsa Dec 6, 2023
54088ef
Merge branch 'main' into r2r-libs-take-3
ivdiazsa Dec 7, 2023
7a23788
Testing something...
ivdiazsa Dec 7, 2023
7c97aa7
Changed the TestReadyToRun pipeline dynamics to only enable the tests
ivdiazsa Dec 12, 2023
f54af82
Added comments to tests.singlefile.targets, and disabled the
ivdiazsa Dec 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Build.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@

<Import Project="$(RepositoryEngineeringDir)SubsetValidation.targets" />
<Import Project="$(RepositoryEngineeringDir)restore\optimizationData.targets" Condition="'$(DotNetBuildFromSource)' != 'true'" />
<Import Project="$(RepositoryEngineeringDir)targetingpacks.targets" />
ViktorHofer marked this conversation as resolved.
Show resolved Hide resolved

<Target Name="BuildLocalTasks"
BeforeTargets="Build">
Expand Down
2 changes: 2 additions & 0 deletions eng/pipelines/common/platform-matrix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ parameters:
jobs:

# Linux arm

trylek marked this conversation as resolved.
Show resolved Hide resolved
- ${{ if or(containsValue(parameters.platforms, 'linux_arm'), in(parameters.platformGroup, 'all', 'gcstress')) }}:
- template: xplat-setup.yml
parameters:
Expand All @@ -45,6 +46,7 @@ jobs:
${{ insert }}: ${{ parameters.jobParameters }}

# Linux armv6

- ${{ if containsValue(parameters.platforms, 'linux_armv6') }}:
- template: xplat-setup.yml
parameters:
Expand Down
42 changes: 42 additions & 0 deletions eng/pipelines/libraries/build-for-test-ready-to-run.yml
View file
Open in desktop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can inline this template into the crossgen2.yml file to reduce nesting.

Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
parameters:
archType: ''
container: ''
buildConfig: ''
crossBuild: false
dependOnEvaluatePaths: false
helixQueues: ''
osGroup: ''
osSubgroup: ''
platform: ''
pool: ''
runtimeFlavor: 'coreclr'
shouldContinueOnError: false
targetRid: ''
variables: []

jobs:
- template: /eng/pipelines/common/global-build-job.yml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may be mistaken but I think that "global-build-job.yml" is the pipeline that's supposed to test "local dev experience" in terms of building and running everything in sequence on the same machine. Is that what we really need here? I would assume that we should be able to reuse the library build artifacts and possibly just build the live CoreCLR subrepo including Crossgen2, possibly along with product.packs in case we want to use the "shipping NativeAOT Crossgen2" for the testing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've been moving to using the global-build-job.yml template as the "build the repo with the root build script" command instead of using the separate CoreCLR, Libraries, etc build jobs for pipelines other than the runtime pipeline.

parameters:
buildArgs: -s clr+libs+packs
-c ${{ parameters.buildConfig }}
buildConfig: ${{ parameters.buildConfig }}

archType: ${{ parameters.archType }}
container: ${{ parameters.container }}
crossBuild: ${{ parameters.crossBuild }}
dependOnEvaluatePaths: ${{ parameters.dependOnEvaluatePaths }}
helixQueues: ${{ parameters.helixQueues }}
nameSuffix: TestReadyToRun_Libraries
osGroup: ${{ parameters.osGroup }}
osSubgroup: ${{ parameters.osSubgroup }}
platform: ${{ parameters.platform }}
pool: ${{ parameters.pool }}
runtimeFlavor: ${{ parameters.runtimeFlavor }}
shouldContinueOnError: ${{ parameters.shouldContinueOnError }}
targetRid: ${{ parameters.targetRid }}
timeoutInMinutes: 120
variables: ${{ parameters.variables }}

extraStepsTemplate: '/eng/pipelines/libraries/run-test-ready-to-run.yml'
extraStepsParameters:
variables: ${{ parameters.variables }}
15 changes: 15 additions & 0 deletions eng/pipelines/libraries/run-test-ready-to-run.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
parameters:
archType: ''
buildConfig: ''
osGroup: ''
variables: []

steps:
- script: $(Build.SourcesDirectory)$(dir)build$(scriptExt) -ci
-subset libs.tests -test
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we intending to run tests locally instead of on Helix? If so, we should remove the helixQueuesTemplate option at https://github.com/dotnet/runtime/pull/91229/files#diff-b7cad421e96ec3229e0085c3a9f978858632fb29a034d369281db897678636f1R63.

If not, we should use the libraries/helix.yml template instead of the -test flag.

Also, any reason that this needs to be a separate invocation of the build script from the one in build-for-test-ready-to-run.yml?

Also, we have the global-build-step.yml template, which we could use instead of manually invoking the build script here. That template would handle the -arch and -os flags for us.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the observations Jeremy. You bring up some interesting points I had not considered that will certainly help improve this PR:

Are we intending to run tests locally instead of on Helix?

We want them to run on Helix. So I'll check out libraries/helix.yml as you suggested.

Also, any reason that this needs to be a separate invocation of the build script from the one in build-for-test-ready-to-run.yml?

This was because originally, we needed to have the packs subset built before attempting to run the tests, in order to use the fully self-contained Crossgen2. This was to get around that error where it tried to run the crossgen2 executable directly and failed saying You must install .NET to run this application.

However, with the most recent changes to the Crossgen2 infrastructure, I just confirmed that this is no longer the case. We can now run say --subset clr+libs+libs.tests in the same call without issues. Will adjust it in CI as well.

-arch ${{ parameters.archType }}
-os ${{ parameters.osGroup }}
-c ${{ parameters.buildConfig }}
/p:TestReadyToRun=true
/p:RunSmokeTestsOnly=true
displayName: Build and Run Libs Tests with TestReadyToRun
Loading