Revamp Materials System

[StandingPadAnimations]

Current proposal

Instead of generating materials in Python directly, we should create materials in a blend file that is then sourced by the Prep Materials operator. Each material would have the format <pack-format>_mcprep-default. This would make it easier for users to add new pack formats, edit existing pack formats, and reduce overall code size. The downside however is that MCprep dev builds would require a blend file for all pack formats.

In this blend file, there would also be a JSON file that may look something like this (I know this example sucks, but bare with me):

{
    "pack-format-1" : {
        "has_emission" : true
        ...
    }
    ...
}

This JSON file will define extra attributes that each material has, such as whether a material has emission nodes or not. We could expand this further to also allow mapping materials to version or even engine.

Old proposal (kept for transparency)

MCprep's sync materials option allows users to make custom materials that get automatically applied when prepping. But currently, it mainly works on a per-material basis. As Minecraft gets updated and the number of blocks grows, it becomes harder and harder for users to change the material for every block. This also makes using external render engines hard as all materials have to be changed.

A solution to these problems would be a default material. Basically, MCprep would look for a material in the materials.blend file with the naming sequence default_packformat_blenderversion_renderengine. Then when prepping, it'll just swap textures.

This would also benefit development. It would move the default material setup out of Python, which makes navigating MCprep code much easier. With the packformat part of the name, it would allow users to make completely different materials for different formats. And users in older versions of Blender can make node-setups that work for them using the blenderversion part of the name.

For example, a simple material for 2.8 EEVEE would be named default_simple_28_eevee and one for seus 3.0 Cycles would be called default_seus_30_cycles

[StandingPadAnimations]

For versions like 2.83, it would be an extra digit like 283. For 2.93.5 however, it would simply be 293

[Roni-Raihan]

If we extract using mineways, the material will be different, but have the same texture file. Only, the location of the uv is different.

[StandingPadAnimations]

That has issues when using texture packs

[StandingPadAnimations]

And also, it does not work for PBR textures since they require separate maps

[TheDuckCow]

I think this is going to be a great redesign, thanks for reporting the idea and the follow up points. It will also help clean up a fair bit of legacy code.

Thinking about it a little more, supporting new render engines will definitely be easier, but some python code will be required. However, it could be limited to very specific interfaces - think like overriding a function that each render engine would need implemented for "find the image thingy and replace it". For Cycles/Eevee, it'd be a function which iterates through nodes of the texture image type, for blender internal it's iterating through texture layer slots, and so forth. There would be a similar function or sub function for getting/updating the PBR passes (and whether or not they exist).

The version convention is nice as it builds in some way to create cross compatibility without forcing everything in one file. I like the idea of a "greedy" numbering system. So if an engine is matched, it tries to use the first matching one even if it doesn't exactly meet the blender version number of the file. This way, each version number can be interpreted as a max number (ie use this file for blender 2.80 and lower). It also means the latest supported materials would be in files that don't have a blender version specified, future proofing them somewhat assuming those parts of the material don't break. Another idea is even offering to search for any file that has the matching

Regarding Mineways and UVs: (thanks @Roni-Raihan for pointing the concern out!) At face value, that indeed shouldn't cause any issues since UVs are generically treated as an input to the materials themselves. Maybe this varies per render engine, but those I'm aware of it should be fine. There is a separate idea however on how to dynamically adjust UV layouts via transforms to support swap texture pack and animate textures with Mineway's all-in-one image export mode. That hasn't been implemented yet, but would ultimately be very engine-specific if it is. At worst, it could be engine restricted.

Final note: This file could potentially be a useful place to actually generate and save a material library for use in blender Assets Manager. Though maybe that muddies things up and only a single starter material should be in this one place, while generated libraries should be in another file. Something to consider.

[StandingPadAnimations]

For #290, I'll try implementing this

[StandingPadAnimations]

Some proof of concept (somehow I finished all the code in a single day, that's a first)

2022-04-23.23-34-08.mp4

[StandingPadAnimations]

Of course for other engines, perhaps a JSON or YMAL (I'm leaning towards the latter as it would have to be easy for the user) file is needed to allow users to define the API methods needed to replace and edit materials for other engines. This also means if a new engine needs support, it should be relatively easy to add support without massive changes

[StandingPadAnimations]

One thing I'll have to fix is the fact that custom default materials are added after materials get prepped the first time. This is particularly annoying with large amounts of materials

But I'm also thinking of a new idea - a way for users to make prepping functions in an easy way. This could be in the form of a folder of extension Python scripts that MCprep loads (perhaps a dictionary of functions), and some functions to handle stuff like texture paths

This has the advantage where engines that may not use the shader editor in favor of their own shader editor (like Luxcore) or don't use the Cycles image texture node can be supported.

Of course the latter will be placed on the back burner for now

[TheDuckCow]

Of course for other engines, perhaps a JSON or YMAL (I'm leaning towards the latter as it would have to be easy for the user) file is needed to allow users to define the API methods needed to replace and edit materials for other engines. This also means if a new engine needs support, it should be relatively easy to add support without massive changes

I'd be wary about going too deep in that direction. This is what technology like Material X is meant to eventually solve, so I don't want to end up inventing our own variant that we will have to maintain and be bespoke per engine. But it comes down to how simple or complex such YAML/JSON files would be if we only need a very limited set of configurable options.

[StandingPadAnimations]

With #431 and #432, I think it's a good time to start focusing more on this, with an alternative method.

One alternative route would be abolishing the current naming system in this proposal with any material name and _mcprep-default at the end. Then query materials with that ending and add that to the pack format options. With that, we could remove hardcoded pack formats

@zNightlord does that sound good to you?

[zNightlord]

Sounds good.

[StandingPadAnimations]

Alright then in that case I'll start working on this sometime this week or next week

[StandingPadAnimations]

That comment aged poorly lol.

Anyway for SEUS and Specular, we should also address an issue with the normal map node in pre-4.0 versions of Blender. It turns out there's a bug that causes normal maps to be interpreted incorrectly (dating back to 2.79, maybe earlier 😬). This was found and reported by Nugget (using Nitter for those without Twitter accounts). Thankfully there is an easy fix in pre-4.0, and the bug has been fixed in Blender 4.0 as well.

Could we just ignore this since this is Minecraft? Sure, but I think it's best to use this solution for Blender 2.8 to 3.6. The fix would make the normal strength behave as users expect, and that's always a good thing.

[StandingPadAnimations]

I've implemented this (mostly) in a fork of MCprep (https://github.com/StandingPadAnimations/Vivy). At the moment, I've added a template based version prep materials (as a separate operator), with support for:

• Diffuse map
• Specular map
• Normal map
• "Extension" materials (where one can specify seperate materials for certain block types) for the following categories:
  • Emissive blocks
  • Reflective blocks
  • Metallic blocks
  • Transmissive blocks
• Texture swap (as a separate operator)

In addition, I'm also working on a UI to abstract the creation of the JSON. Here's a video demonstrating what I have (from Mastodon since the video file is too large for GitHub), using a material that makes all blocks emissive:

Video showcasing new material system

However, after creating this, I'm hesitant to merge this upstream since (in my opinion) this clashes with MCprep's design goals. In addition, it completely changes a core part of MCprep, and inherently requires knowledge of nodes (since users have to create their own materials now), which in my experience doesn't seem to be that common of a skill in the community.

[StandingPadAnimations]

Closing to be superseded by #600

[StandingPadAnimations]

Further discussion will happen on #600