Skip to content

Conversation

theopnv
Copy link
Contributor

@theopnv theopnv commented Aug 26, 2020

Purpose of this PR

Some standalone tests were inconclusive because sometimes they timeout.
Splitting the player build and the execution of the tests will hopefully reduce the number of flaky tests.

  • Main changes:
    • Split build & execution for Linux tests (most of it was already written in linux_split.py, thanks to @kivistikliis. It is now simply renamed linux.py and the old file was deleted).
    • Split build & execution for the smoke test pipeline, which can now be managed through its own metafile instead of being part of the abv python script (again, thanks @kivistikliis for the help).
  • Small changes:

Testing status

Automated Tests: What did you setup? (Add a screenshot or the reference image of the test please)

Yamato: (Select your branch):
https://yamato.prd.cds.internal.unity3d.com/jobs/902-Graphics/tree/yamato%252Fstandalone-tests%252Fsplit-build-execution

@theopnv
Copy link
Contributor Author

theopnv commented Sep 11, 2020

tldr: we can't have both smoke tests and vfx tests working without the vfx tests fixed first. They both are green on different trunk changesets. Merging this PR and updating the pin will fix the smoke tests and break the vfx tests in their current state.

Note: when running ABV on trunk, we're using the changeset 90b2443a8df13ecc935d91f85d723464f8a42028 (2020.2.0b2). It won't work with the smoke tests, since the issue was fixed in 2020.2.0b3 (ef376aff2859d7502bf9ded55dcc5d01153ab991). If we want the smoke tests to work we have to pin this version in the trunk ABV. This is a custom revision ABV run with this changeset. A few tests are failing:

  One or more packages could not be added to the local file system:
    com.unity.burst: shasum check failed for C:\Users\bokken\AppData\Local\Unity\cache\npm\artifactory.prd

I am running a test for custom revision ef376aff to see if it is also failing on master ABV.

  • VFX_URP on Win_DX11_playmode
  • VFX_HDRP on Win_DX11_playmode_XR
  • VFX_HDRP on Win_DX11_playmode
    Which are failing on master ABV if we update trunk to a newer version. This is a known issue, and the reason why we stayed at this version.

@theopnv theopnv changed the title Split build & Execution for standalone tests (smoke, linux) Split build & Execution for standalone tests (smoke, linux) [skip ci] Oct 6, 2020
@theopnv
Copy link
Contributor Author

theopnv commented Oct 6, 2020

The PR is ready to be reviewed and merged.
The nigthly I started for this PR is red:

  • URP and HDRP failures we're aware of (see the nightly report channel)

Note: What this PR has impacted:

  • Smoke tests are all green. The standalone one has been re-enabled and now has a dedicated build step.
  • The HDRP on Linux Standalone test now has a dedicated build step.

Note 2: For teams other than the gfx-sdets assigned to this PR: It is because I merged master into this branch and while doing so the git hooks detected wrong casing in some files, that I fixed. Reminder: the git hooks verify that files that compile fine on windows and mac won't break when compiling on Linux because of the case-sensitivity.

@theopnv theopnv marked this pull request as ready for review October 6, 2020 10:16
@theopnv theopnv requested a review from a team as a code owner October 6, 2020 10:16
@theopnv theopnv requested a review from eh-unity October 6, 2020 10:16
@ellioman ellioman removed the request for review from eh-unity October 7, 2020 12:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants