Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrating more critical tests ahead of release. #487

Merged
merged 13 commits into from
Sep 30, 2023

Conversation

TheDuckCow
Copy link
Member

This includes QA checking scripts, which have already uncovered one issue with one of the updated rigs.

Recent run:

-------------------------------------------------------------------------------
bversion   	ran_tests	ran	skips	failed	errors
-------------------------------------------------------------------------------
(3.6.2)   	all_tests	46	2	0	No errors
(4.0.0)   	all_tests	46	2	0	No errors
(3.5.1)   	all_tests	46	2	0	No errors
(3.4.0)   	all_tests	46	2	0	No errors
(3.3.1)   	all_tests	46	2	0	No errors
(3.2.1)   	all_tests	46	2	0	No errors
(3.1.0)   	all_tests	46	2	0	No errors
(3.0.0)   	all_tests	46	2	0	No errors
(2.93.0)   	all_tests	46	2	0	No errors
(2.90.1)   	all_tests	46	2	0	No errors
(2.80.75)   	all_tests	44	4	0	No errors
tests took 200s to run

@TheDuckCow TheDuckCow marked this pull request as ready for review September 28, 2023 04:45
@TheDuckCow
Copy link
Member Author

Alright, with this I've done testing across versions of blender 2.80 up through and including 4.0.

There are some issues that remain, I'll comment on the main dev to main PR branch for those.

@TheDuckCow TheDuckCow mentioned this pull request Sep 28, 2023
11 tasks
@TheDuckCow
Copy link
Member Author

Even more tests migrated, and now geo nodes in 4.0+ is working without errors:

-------------------------------------------------------------------------------
bversion   	ran_tests	ran	skips	failed	errors
-------------------------------------------------------------------------------
(3.6.2)   	all_tests	49	2	0	No errors
(4.0.0)   	all_tests	49	2	0	No errors
(3.5.1)   	all_tests	49	2	0	No errors
(3.4.0)   	all_tests	49	2	0	No errors
(3.3.1)   	all_tests	49	2	0	No errors
(3.2.1)   	all_tests	49	2	0	No errors
(3.1.0)   	all_tests	49	2	0	No errors
(3.0.0)   	all_tests	49	2	0	No errors
(2.93.0)   	all_tests	48	3	0	No errors
(2.90.1)   	all_tests	48	3	0	No errors
(2.80.75)   	all_tests	47	4	0	No errors

I realized I hadn't tested sync materials or the UV node tools at all, so I'll need to do a trial run of that later (and possibly migrate those tests too). Still, presume this branch is review-ready.

This was a weird one. Essentially, running the unittests would always
result in the `mcprep_data_update.json` file being renamed to
`mcprep_data.json` (and since that file isn't tracked in git, would look
like the file was deleted wrt to local changes). This was happening, out
of all things, due to this line in the unittest runner:

unittest.TestLoader().discover(MODULE_DIR, "*_test.py")

This is because the discover command clearly has to attempt loading of
the modules it's scanning over, and at some point we import the env
module and thus the __init__ code on module load would trigger,
resulting in the file rename. By moving the rename into the register
function, we ensure it only happens in the context of an addon, which
is all we really need for it.
@TheDuckCow
Copy link
Member Author

Sigh, a couple more tests that I decided to move over now are failing. test_sync_materials;test_uv_transform_detection. Still checking if they are actual failures in code, or just test fixture issues. I have some time in the morning (~8-10hr from now) when hopefully I can finish wrapping this all up.

Unfortunately, we do still have the issue that the test runner renames
the mcprep_data_update.json file, and modifies checked-in mtl files,
but we can address this in the future.
The new test runner in python is straight forward enough to use that I
feel there is no longer a need for any system-specific shell files.
@TheDuckCow
Copy link
Member Author

Alright, I'm making it official - we're merging this! 3.5 here we come

@TheDuckCow TheDuckCow merged commit 968f2bd into dev Sep 30, 2023
@TheDuckCow TheDuckCow deleted the 424-more-test-migrations branch September 30, 2023 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant