Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test build target agnostic separately from target specific tests #34658

Merged
merged 10 commits into from
Apr 9, 2020

Conversation

sdmaclea
Copy link
Contributor

@sdmaclea sdmaclea commented Apr 7, 2020

Part of #33066

Modifies build-test to add options to allow building target agnostic tests separately from target specific tests.

  • Audited the TraitTags found a few cases where TraitTags were missed in Added OS and Bit Specific tags to accordingly .csproj files. #637. Also added ArchSpecific
  • @VSadov is working on a generic filtering mechanism. While waiting for it this PR add an extra property CLRTestNeedTarget to indicate a test which is not target agnostic.
  • I had originally tried to build the agnostic tests with AnyOS and AnyCPU, but I ran into lots of issues. So I reverted that.

@sdmaclea sdmaclea added this to the 5.0 milestone Apr 7, 2020
@sdmaclea sdmaclea self-assigned this Apr 7, 2020
@ghost
Copy link

ghost commented Apr 7, 2020

Tagging @ViktorHofer as an area owner

@sdmaclea
Copy link
Contributor Author

sdmaclea commented Apr 7, 2020

I tested manually building on Linux.  It looks good to me.
I will try building tests on Windows too now.

@sdmaclea
Copy link
Contributor Author

sdmaclea commented Apr 7, 2020

The simple filtering used in this PR is relatively slow. It takes a while for MSBuild to find the tests to build (when finding the small set of target specific tests).

Generic filtering may be slower.

It is possible we may need to build test lists to make this faster....

Copy link
Member

@jkoritzinsky jkoritzinsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we do a string for CLRTestNeedTargetToBuilds value instead of an opaque number that has to be documented somewhere else?

@sdmaclea
Copy link
Contributor Author

sdmaclea commented Apr 7, 2020

Can we do a string for CLRTestNeedTargetToBuilds

You're going to make me do the hard work of naming the states.

Effectively right now the states are

  • CLRTestNeedTargetToBuild == 0 : False
  • CLRTestNeedTargetToBuild == 1: True
  • CLRTestNeedTargetToBuild == '' BuildAll

@jkoritzinsky How does BuildTargetGenericTests and BuildTargetSpecificTests sound for strings. Keeping an empty string for the default behavior.

@jkoritzinsky
Copy link
Member

Those work. We could also shorten them to "TargetGeneric" and "TargetSpecific" if you would like.

@sdmaclea
Copy link
Contributor Author

sdmaclea commented Apr 7, 2020

What about the script option? Sounds like they should change to

  • -buildTargetSpecific
  • -buildTargetGeneric

or

  • -targetSpecific
  • -targetGeneric

@ivdiazsa
Copy link
Member

ivdiazsa commented Apr 7, 2020

I agree with @jkoritzinsky's suggestions. Other than that, LGTM! I'm assuming the test lists building would be done in a separate PR?

@ivdiazsa
Copy link
Member

ivdiazsa commented Apr 7, 2020

What about the script option? Sounds like they should change to

-buildTargetSpecific
-buildTargetGeneric

or

-targetSpecific
-targetGeneric

Both are fine by me. I would vote for the latter choices. Since the script is already called build-test, the former ones are a bit redundant.

@sdmaclea
Copy link
Contributor Author

sdmaclea commented Apr 8, 2020

I'm assuming the test lists building would be done in a separate PR?

@VSadov Is working on generic filtering. I was assuming we would wait and see what he comes up with before engineering another solution,

@sdmaclea
Copy link
Contributor Author

sdmaclea commented Apr 8, 2020

I made the renames per feedback.

I also did a quick benchmark of test building on my local Hyper-V Ubuntu machine. It took 10 minutes to build all the tests. When using testSpecific, the build time dropped to 3 minutes.

@sdmaclea
Copy link
Contributor Author

sdmaclea commented Apr 9, 2020

quick benchmark - Windows results were about the same as Linux Hyper-V on the same machine.

@sdmaclea sdmaclea merged commit 0b02d28 into dotnet:master Apr 9, 2020
@ghost ghost locked as resolved and limited conversation to collaborators Dec 9, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants