Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replaced compile scripts with mcprep-build #422

Merged
merged 33 commits into from
Jul 25, 2023
Merged
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
bcba39f
Replaced compile scripts with mcprep-build
StandingPadAnimations May 10, 2023
193458a
Fixed issue with arguments in mcprep-build
StandingPadAnimations May 10, 2023
0690e6a
Updated this branch to be up to date with milestone 3.5
StandingPadAnimations May 10, 2023
a303cae
Updated CONTRIBUTING.md for mcprep-build
StandingPadAnimations May 10, 2023
3e2f5a5
Update CONTRIBUTING.md
StandingPadAnimations May 11, 2023
419d706
Fixed paths and removed redundent moves
StandingPadAnimations May 12, 2023
f475012
Added flake8 to dependencies
StandingPadAnimations May 12, 2023
98a5bcb
Added flake8 check
StandingPadAnimations May 12, 2023
b31ab8a
Fixed destination folder issue and formatting
StandingPadAnimations May 12, 2023
79e5c7d
Added bpy-build.yaml for bpy-build
StandingPadAnimations May 14, 2023
b82549e
Added the during_build part to bpy-build.yaml
StandingPadAnimations May 14, 2023
71e7e08
Moving from mcprep-build to bpy-addon-build
StandingPadAnimations May 15, 2023
2bd8b60
Added flake8 check by default
StandingPadAnimations May 15, 2023
2ad1f1b
Updated bpy-addon-build version
StandingPadAnimations May 15, 2023
24b62a0
Updated bpy-addon-build version again
StandingPadAnimations May 15, 2023
bd0a9cd
Updated again because I can't stop making errors
StandingPadAnimations May 15, 2023
2b7dd04
Updated again this time for the last time
StandingPadAnimations May 15, 2023
3757350
Commiting poetry.lock cause
StandingPadAnimations May 15, 2023
09b645f
Updated bpy-addon-build for the 5th time
StandingPadAnimations May 15, 2023
c648fd8
Updated bpy-addon-build version
StandingPadAnimations Jun 26, 2023
6ea6825
Removed bv28 call
StandingPadAnimations May 11, 2023
81d2bd6
Revert "Removed bv28 call"
StandingPadAnimations May 12, 2023
2783954
Fixed syntax mistake for dataclass type
StandingPadAnimations Jun 17, 2023
b934bc4
Added use of `__slots__` for PrepOptions
StandingPadAnimations Jul 3, 2023
7f6b900
Fixed elif typo
StandingPadAnimations Jul 3, 2023
6b3b298
Replaced compile scripts with mcprep-build
StandingPadAnimations May 10, 2023
e915ecd
Added all supported versions to bpy-build file
StandingPadAnimations Jul 5, 2023
4c000a0
Updated dependencies
StandingPadAnimations Jul 5, 2023
cc9158d
Removed the old mcprep-build script
StandingPadAnimations Jul 5, 2023
d292559
Update the contributing guide with venv instructions. Adding gitignore.
TheDuckCow Jul 16, 2023
b8518f1
Disabling flake8 in the build script.
TheDuckCow Jul 16, 2023
483f6e4
Set upper bound for bpy-build
StandingPadAnimations Jul 25, 2023
456b3f9
Fixed merge conflicts with milestone-3-5-0
StandingPadAnimations Jul 25, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,6 @@ MCprep_addon/MCprep_resources/resourcepacks/mcprep_default/materials.blend1
mcprep_venv_*
.cache
.python-version
venv/
MCprep_addon/MCprep_resources/
*.sublime-*
57 changes: 29 additions & 28 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,12 @@ When it comes to code being reviewed, expect some discussion! If you want to con

## Keeping MCprep compatible

MCprep is uniquely made stable and functional across a large number of versions of blender. As of April 2022, it still even supports releases of Blender 2.79 while simultaneously supporting Blender 3.1+, and everything in between.
MCprep is uniquely made stable and functional across a large number of versions of blender. As of April 2022, it still even supports releases of Blender 2.8 while simultaneously supporting Blender 3.5+, and everything in between.

This is largely possible for a few reasons:

1. Automated tests plus an automated installer makes ensures that any changes that break older versions of blender will be caught automatically.
1. Abstracting API changes vs directly implementing changes. Instead of swapping "group" for "collection" in the change to blender 2.8, we create if/else statements and wrapper functions that fetch the attribute that exists based on the version of blender. Want more info about this? See [the article here](https://theduckcow.com/2019/update-addons-both-blender-28-and-27-support/).
1. No python annotations. This syntax wasn't supported in old versions of python that came with blender (namely, in Blender 2.7) and so we don't use annotations in this repository. Some workarounds are in place to avoid excessive printouts as a result.

## Internal Rewrites
MCprep has a separate branch for internal rewrites based on the dev branch. Sometimes, internal tools are deprecated, and requires features to be changed to reflect those deprecations.
Expand All @@ -45,11 +44,32 @@ As above, a critical component of maintaining support and ensuring the wide numb

### Compile MCprep using scripts

Scripts have been created for Mac OSX (`compile.sh`) and Windows (`compile.bat`) which make it fast to copy the entire addon structure the addon folders for multiple versions of blender. You need to use these scripts, or at the very least validate that they work, as running the automated tests depend on them.
MCprep uses the [bpy-addon-build](https://github.com/Moo-Ack-Productions/bpy-build) package to build the addon, which makes it fast to copy the entire addon structure the addon folders for multiple versions of blender.

The benefit? You don't have to manually navigate and install zip files in blender for each change you make - just run this script and restart blender. It *is* important you do restart blender after changes, as there can be unintended side effects of trying to reload a plugin.
The benefit? You don't have to manually navigate and install zip files in blender for each change you make - just run the command and restart blender. It *is* important you do restart blender after changes, as there can be unintended side effects of trying to reload a plugin.

Want to just quickly reload some files after only changing python code (no asset changes)? Mac only: Try running `compile.sh -fast` which will skip copying over the resources folder and skip zipping the addon.
As a quick start:

```bash
# Highly recommended, create a local virtual environment (could also define globally)
python3 -m pip install --user virtualenv


python3 -m pip install --upgrade pip # Install/upgrade pip
python3 -m venv ./venv # Add a local virtual env called `venv`

# Activate that environment
## On windows:
.\venv\Scripts\activate
## On Mac/linux:
source venv/bin/activate

# Now with the env active, do the pip install (or upgrade)
pip install --upgrade bpy-addon-build

```

Moving forward, you can now build the addon for all intended supported versions using: `bpy-addon-build -b dev`

### Run tests

Expand Down Expand Up @@ -107,7 +127,7 @@ At the moment, only the project lead (TheDuckCow) should ever mint new releases
- Tag is in the form `3.3.1`, no leading `v`.
- The title however is in the form `MCprep v3.3.0 | ShortName` where version has a leading `v`.
- Copy the body fo the description from the prior release, and then update the text and splash screen (if a major release). Edit a prior release without making changes to get the raw markdown code, e.g. [from here](https://github.com/TheDuckCow/MCprep/releases/edit/3.3.0).
1. Run `compile.sh` or `compile.bat` with no fast flag, so it does the full build
1. Run `bpy-addon-build.py` to build the addon
1. Run all tests, ideally on two different operating systems. Use the `./run_tests.sh -all` flag to run on all versions of blender
1. If all tests pass, again DOUBLE CHECK that "dev" = false in conf.py, then
1. Drag and drop the generated updated zip file onto github.
Expand All @@ -122,24 +142,7 @@ At the moment, only the project lead (TheDuckCow) should ever mint new releases



## Creating your blender_installs.txt and blender_exects.txt


Your `blender_installs.txt` defines where the `compile.sh` (Mac OSX) or `compile.bat` (Windows) script will install MCprep onto your system. It's a directly copy-paste of the folder.

On a mac? The text file will be generated automatically for you if you haven't already created it, based on detected blender installs. Otherwise, just create it manually. It could look like:

```
/Users/your_username/Library/Application Support/Blender/3.1/scripts/addons
/Users/your_username/Library/Application Support/Blender/3.0/scripts/addons
/Users/your_username/Library/Application Support/Blender/2.93/scripts/addons
/Users/your_username/Library/Application Support/Blender/2.92/scripts/addons
/Users/your_username/Library/Application Support/Blender/2.90/scripts/addons
/Users/your_username/Library/Application Support/Blender/2.80/scripts/addons
/Users/your_username/Library/Application Support/Blender/2.79/scripts/addons
/Users/your_username/Library/Application Support/Blender/2.78/scripts/addons
/Users/your_username/Library/Application Support/Blender/2.72/scripts/addons
```
## Creating your blender_execs.txt

Your `blender_execs.txt` defines where to find the executables used in the automated testing scripts. Only these executables will be used during automated testing, noting that the testing system only supports blender version 2.8+ (sadly, only manual testing is possible in blender 2.7 with the current setup). It could look like:

Expand All @@ -160,11 +163,9 @@ Also note that the first line indicates the only version of blender that will be

Support for development and testing should work for both platforms, just be aware the primary development of MCprep is happening on a Mac OSX machine, so the mac-side utility scripts have a few more features than windows:

- Only the mac `compile.sh` script has the `-fast` option to quickly reload python files (it won't copy over the blends, textures, and won't create a new zip file, all of which can be slow)
- Only the mac `compile.sh` has the feature of auto-detecting local blender executable installs. This is because on windows, there is a lot of variability where blender executables may be placed, so it should just be manually created anyways.
- Only the mac `run_tests.sh` script has the `-all` optional flag. By default, the mac script will only install the
- Only the mac `run_tests.sh` script has the `-all` optional flag. By default, the mac script will only install the first line in the file.

One other detail: MCprep uses git lfs or Large File Storage, to avoid saving binary files in the git history. Some Windows users may run into trouble when first pulling.
One other detail: MCprep uses Git LFS or Large File Storage, to avoid saving binary files in the git history. Some Windows users may run into trouble when first pulling.

- If using Powershell and you cloned your repo using SSH credentials, try running `start-ssh-agent` before running the clone/pull command (per [comment here](https://github.com/git-lfs/git-lfs/issues/3216#issuecomment-1018304297))
- Alternatively, try using Git for Windows and its console.
Expand Down
27 changes: 26 additions & 1 deletion MCprep_addon/materials/generate.py
Original file line number Diff line number Diff line change
Expand Up @@ -246,9 +246,34 @@ def checklist(matName, listName):
return True
return False

# Dataclass representing all options
# for prep materials
#
# We use __slots__ since __slots__ prevents the
# following bug:
# p = PrepOptions(...)
# p.psses["..."] = "..."
#
# Where a non-existant variable is used. In
# Python, this would create a new variable
# "psses" on p. To prevent this, we use __slots__.
#
# In addition, access to objects in __slots__ is
# faster then it would be normally
#
# Python dataclasses have native support for __slots__
# in 3.10, but since 2.8 uses 3.7, we have to use
# __slots__ directly
@dataclass
class PrepOptions:
passes: dict[str]
__slots__ = ("passes",
"use_reflections",
"use_principled",
"only_solid",
"pack_format",
"use_emission_nodes",
"use_emission")
passes: dict[str, str]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This typed dict creates syntax errors in blender 2.80, and regardless I don't see how is related to the PR topic itself. At the very least, will need to change this to support typing that was around in python3

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was added in the milestone branch and was synced up

use_reflections: bool
use_principled: bool
only_solid: bool
Expand Down
2 changes: 1 addition & 1 deletion MCprep_addon/mcprep_ui.py
Original file line number Diff line number Diff line change
Expand Up @@ -2082,7 +2082,7 @@ def unregister():
for cls in reversed(classes):
bpy.utils.unregister_class(cls)

elif hasattr(bpy.types, "VIEW3D_MT_add"): # 2.8
if hasattr(bpy.types, "VIEW3D_MT_add"): # 2.8
bpy.types.VIEW3D_MT_add.remove(draw_mcprepadd)

if hasattr(bpy.types, "IMAGE_MT_uvs"): # 2.8 *and* 2.7
Expand Down
21 changes: 21 additions & 0 deletions bpy-build.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
addon_folder: MCprep_addon
build_name: MCprep_addon

install_versions:
- '3.6'
- '3.5'
- '3.4'
StandingPadAnimations marked this conversation as resolved.
Show resolved Hide resolved
- '3.3'
- '3.2'
- '3.1'
- '3.0'
- '2.93'
- '2.90'
- '2.80'

during_build:
default:
- [] # Re-enable later: [flake8 --extend-ignore W191 .]
dev:
- create_file("mcprep_dev.txt")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this do? Just creates the empty file, for the addon to recognize run as a dev build?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, it just creates an empty file

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be good to add a yaml comment inline which clarifies why we do this. Maybe:

# Create dev file to force dev tracking and other behavior


84 changes: 0 additions & 84 deletions compile.bat

This file was deleted.

Loading