Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev to Master for MCprep 3.5 #479

Merged
merged 223 commits into from
Sep 30, 2023
Merged

Dev to Master for MCprep 3.5 #479

merged 223 commits into from
Sep 30, 2023

Conversation

StandingPadAnimations
Copy link
Collaborator

@StandingPadAnimations StandingPadAnimations commented Sep 17, 2023

Development on new features has calmed down for the most part, so I think it's safe to say we're now in the final stage before release. While GitHub says only 1 approval is needed, this PR needs approval from all developers to be merged.

Checklist of things we need to do before merging

  • Review code one last time to ensure consistency
  • Verifying MCprep works with AgX (without artistic handling)  #466
  • Get MCprep assets working with Git LFS and put Git LFS instructions in CONTRIBUTING.md (I haven't been able to pull assets yet)
  • Update all MCprep assets to Minecraft 1.20 and add newly submitted rigs (EDIT: we can't accept the rigs for this release cycle since the asset creators have not yet responded)
  • Run all MCprep tests on all possible platforms (preferably, though we should at least verify Windows and macOS, as those are the major platforms)
    • Test on Windows
    • Test on macOS
    • Test on Linux (if possible)
  • Make release and test updater

After release, if possible, we should publish the RC builds (after changing the commits they point to) just to make it easier for users to get. Future RC builds will be published, as MCprep 3.5 will ignore RCs.

StandingPad and others added 30 commits October 16, 2022 16:56
As we're dropping 2.7x support, this is no longer needed. Most of the
compatability functions will remain though since they're more readable.
Replaced some instances of os.path with the Path object from pathlib for
readability. Also added type annotations to MCprepEnv as we're moving
towards that. This also means MCprep will not work at all in Blender
2.7x due to the use of new syntax.

This will be the first in a long process of modernizing MCprep's code
with 2.8 style code. Blender 2.7x users may not be happy, but it's for
the better. If anything, 6 years worth of 2.7x support was a mistake (in
my opinion), as it limited what we could do and opened MCprep to even
more bugs (like in Blender 2.93 with make_annotations, which ironically
now is deprecated).
Replaced the old = syntax with the new : syntax. This also removes the
need for make_annotations, hence why they're removed.
As we're removing 2.7x support, it makes sense to remove Blender
Internal compatability as the engine is not present in Blender 2.8 and
above. Even UPBGE (a fork of the Blender game engine that's still
maintained) uses EEVEE, so there's no reason to continue supporting
Blender Internal.
The bang at the top meant Linux users couldn't run the test suite. This
changes now allows Linux users to run tests
Deprecation warning function was moved in MCprepEnv since it made more
sense to put it there
Originally matprep_cycles used to take multiple arguments, each one
representing a seperate option. This was annoying as it meant the
function has an unfathomable amount of arguments to keep track of.

This has been replaced with a dataclass (called PrepOptions) that keeps
track of all the options selected by the user. Not only does it make it
slightly easier for developers, it also gives the following benefits:
	- More info on what types the different options are
	- Better organization

Previously we couldn't do this due to 2.7x support, hence this being in
mcprep-3_5-recore.
I'm honestly not sure how I missed these, but good thing they were
caught. Now MCprep shouldn't be dealing with any errors related to env
now...
While removing the last of the ENVs, I noticed some deprecation warnings
regarding the use of this function, so I've went ahead and removed it.

Since this function calls other deprecated functions as well, what
should have been 1 warning was instead 3. Apologies to anyone who had to
suffer that in their terminal ;-;
In all honesty I doubt we'll get around to actually removing these
functions in 3.5.1, but since we're merging this with the 3.5 feature
branch, I think it's important to warn developers and make them think
about the eventual demise of these functions. Plus there's absolutely no
reason to use deprecated functions for 3.5 when we'll have nice features
like type annotations.

Anyway, hopefully after this commit, we'll merge this with the MCprep
3.5 feature branch *fingers crossed*
For some reason the spawner was missing the use of env, causing errors.
While this has been fixed, it does illustrate an issue with the current
deprecation system in that deprecated stuff will error out. More
investigation is needed.
Removed references to darker in CONTRIBUTING.md
@StandingPadAnimations
Copy link
Collaborator Author

I was able to run tests on Linux (using python ./run_tests -a. This is what I did:

  1. Get a fresh clone of MCprep
  2. Checkout to dev
  3. Pulled all LFS objects
  4. Switch to Python 3.9 with Pyenv (Python 3.11 causes the env is undefined error for some reason)
  5. Use the download_resources.sh script from the mcprep-rc branch for basic assets
  6. Set up a venv with requirements.txt
  7. Run all tests

With all of that, here are my results

-------------------------------------------------------------------------------
bversion        ran_tests       ran     skips   failed  errors
-------------------------------------------------------------------------------
(3.6.2)         all_tests       35      2       0       No errors
tests took 20s to run

I do however get some intersting errors:

Cannot load image file: 'textures/scaffolding top.png'
Cannot load image file: '/home/mahid/Documents/MCprep/test_files/test_data/scaffolding_top.png'
Failed to open dir (No such file or directory): /home/mahid/Documents/MCprep/test_files/test_data/textures/
Cannot load image file: '/home/mahid/Documents/MCprep/test_files/test_data/textures/scaffolding_side.png'
Failed to open dir (No such file or directory): textures/
Cannot load image file: 'textures/scaffolding_side.png'
Failed to open dir (No such file or directory): textures/
Cannot load image file: 'textures/scaffolding side.png'
Cannot load image file: '/home/mahid/Documents/MCprep/test_files/test_data/scaffolding_side.png'
Failed to open dir (No such file or directory): /home/mahid/Documents/MCprep/test_files/test_data/textures/
Cannot load image file: '/home/mahid/Documents/MCprep/test_files/test_data/textures/scaffolding_bottom.png'
Failed to open dir (No such file or directory): textures/
Cannot load image file: 'textures/scaffolding_bottom.png'
Failed to open dir (No such file or directory): textures/
Cannot load image file: 'textures/scaffolding bottom.png'
Cannot load image file: '/home/mahid/Documents/MCprep/test_files/test_data/scaffolding_bottom.png'
Failed to open dir (No such file or directory): /home/mahid/Documents/MCprep/test_files/test_data/textures/
Cannot load image file: '/home/mahid/Documents/MCprep/test_files/test_data/textures/sea_lantern.png'
Failed to open dir (No such file or directory): textures/
Cannot load image file: 'textures/sea_lantern.png'
Failed to open dir (No such file or directory): textures/
Cannot load image file: 'textures/sea lantern.png'

Looks like we got to fix some texture stuff in Git LFS

@StandingPadAnimations
Copy link
Collaborator Author

Based on #483, it does look like tests pass on Arch Linux. I'm not sure what are the 2 tests that are skipped, but I'll just mark Linux testing as done

@StandingPadAnimations
Copy link
Collaborator Author

Alright, just for the purposes of keeping it up to date, I've updated the mcprep-rc branch with the latest changes to dev, but there will not be a new RC (partly because no one can download them, partly because we're almost done with MCprep 3.5 anyway)

@TheDuckCow
Copy link
Member

Yeah the skipped tests are fine, they're semi annoying ones that would open up folders or webrowser pages which we can set to include if we really want to be exhaustive, but not necessary for now.

I'm surprised there aren't more errors actually, I'm not worried about having 100% of assets in the test file for git lfs, so long as tests pass. The obj files could be certainly made smaller or more specific for our needs to not have extras, but this is ok for now I'd say and not something to fret too hard over. I think we're very close, just need to align on that concept art - will chat over Discord!

@TheDuckCow
Copy link
Member

Work on releasing in progress - I've migrated a few more unit tests, which have thankfully highlighted a few more fixes to assets I'm working through right now. Mostly save versions, addressing a couple non-relative paths, etc.

Giving this another day then expecting to cut the release.

@TheDuckCow
Copy link
Member

We are even closer with #487 but I did come across a few issues that remain:

  • Render pano not working on 2.80 at least, raises an error; it is working in 3.6 and 4.0 at least
  • Spawn geo effects in 4.0 not working, raises an error
  • Crashing in blender 2.80 during image sequence effect spawning like "sweep" (not an issue in 3.6). I think this is pre-existing though, and likely not much we can do.
  • Finally, this one bug is 8x the msot frequent out of all others, and may be worth trying to squeeze in here

File "<addon_path>/mcmodel.py", line 517, in execute\n obj = add_model(os.path.normpath(self.filepath), filename)\n File "<addon_path>/mcmodel.py", line 258, in add_model\n mat = add_material(obj_name + "_" + img, tex_pth)\n File "<addon_path>/mcmodel.py", line 107, in add_material\n res = bpy.ops.mcprep.load_material(filepath=path, skipUsage=True)\n File "<addon_path>/ops.py", line 113, in __call__\n ret = _op_call(self.idname_py(), None, kw)\nRuntimeError: Error: File not found! Reset the resource pack under advanced settings (return arrow icon) and press reload materials\n\n

@StandingPadAnimations
Copy link
Collaborator Author

Based on the error, it might be a case of having to reload assets in the test (users have to reload them anyway when creating a new file)

@TheDuckCow
Copy link
Member

Regarding that error, I realize now I actually had already addressed that long ago by changing how we apply materials during the creation stage, so it should be mitigated. These other issues I expect to be pre-existing, and we can see about addressing in a patch - not worth pushing the release out furhter.

I got tied up tonight, but for real I plan to cut the release tomorrow. If you had any comments on this PR feel free to add them, but I won't wait for them as I'll focus on merging than and then doing final merging here to cut the release.

@StandingPadAnimations
Copy link
Collaborator Author

We just need to add Git LFS instructions (I did notice we need to update documentation for tests too) in CONTRIBUTING.md. Ideally we'd also have someone run a test on Windows, but I think passing on macOS and Linux should be enough.

I won't be able to push any changes today or tomorrow, so someone else needs to do those changes to CONTRIBUTING.md

@zNightlord @MajesticalDiscomfort, any comments before we release MCprep 3.5 tomorrow?

@StandingPadAnimations StandingPadAnimations marked this pull request as ready for review September 29, 2023 14:29
@StandingPadAnimations
Copy link
Collaborator Author

Removed this from “Draft” status since we plan to merge tomorrow

@TheDuckCow
Copy link
Member

Thanks - I'm doing some final sorting of things, and then I'll merge the other PR followed by this one. This is it for real - just a couple hours at most now!

This was a weird one. Essentially, running the unittests would always
result in the `mcprep_data_update.json` file being renamed to
`mcprep_data.json` (and since that file isn't tracked in git, would look
like the file was deleted wrt to local changes). This was happening, out
of all things, due to this line in the unittest runner:

unittest.TestLoader().discover(MODULE_DIR, "*_test.py")

This is because the discover command clearly has to attempt loading of
the modules it's scanning over, and at some point we import the env
module and thus the __init__ code on module load would trigger,
resulting in the file rename. By moving the rename into the register
function, we ensure it only happens in the context of an addon, which
is all we really need for it.
Unfortunately, we do still have the issue that the test runner renames
the mcprep_data_update.json file, and modifies checked-in mtl files,
but we can address this in the future.
The new test runner in python is straight forward enough to use that I
feel there is no longer a need for any system-specific shell files.
…ions

Migrating more critical tests ahead of release.
@TheDuckCow
Copy link
Member

Here we go, 3.5!

@TheDuckCow TheDuckCow merged commit 8722c10 into master Sep 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants