Skip to content

Modifying materials (textures and shaders)

Jump to bottom
eArmada8 edited this page Sep 9, 2023 · 11 revisions

Modifying materials (textures and shaders)

This is a deep dive on materials and how to change shaders. If you just want to change a texture, please refer to the basic modding tutorial. The second half of this tutorial on obtaining UV (texture coordinate) maps is here.

If you are looking for suitable materials, download my database of ed8_chr.fx switches here (also available in CSV for those who prefer spreadsheets). (DB for ed8_map.fx here / CSV here) To use the database, download DB Browser for SQLite which is free and available in both install and portable format, instructions here.

Understanding materials

Materials are the combination of textures and a pixel shader (a small program used by the GPU to draw the object). The shader will apply the textures to the object, and then modify the appearance (e.g. applying shininess and transparency). We can change textures but shaders cannot be modified. We can exchange one shader for another, however. For a deep discussion on why shaders cannot be directly modified, please refer to this tutorial.

Every mesh in a model must have a material applied to it. This assignment is made in the toolset using the .material file, which is a plain text file in JSON format. If you edit this file in notepad, you will see its assigned material. For example, in Emma CS4 default (C_CHR005_C10.pkg), her skirt and boots (polySurface4962_07.vb) are assigned to "new_uwagi_collada".

Contents of polySurface4962_07.material:

{
    "material": "new_uwagi_collada"
}

Looking in metadata.json, we can see the material "new_uwagi_collada". Note that build_collada.py will look in metadata.json for the actual material definition, as polySurface4962_07.material is nothing more than an assignment.

        "new_uwagi_collada": {
            "shader": "shaders/ed8_chr.fx#FD475805C05CFAA3CD419F156CC20E72",
            "skinned_shader": "shaders/ed8_chr.fx#0FBE93BD2C4F285287841E563DFA6A1C",
            "shaderTextures": {
                "DiffuseMapSampler": "assetconv_temp/dds_dae/D3D11/chr005_03_conv.dds",
                "NormalMapSampler": "assetconv_temp/dds_dae/D3D11/chr005_03n_conv.dds",
                "CartoonMapSampler": "assetconv_temp/dds_dae/D3D11/dlight_all_conv.dds"
            },
            "shaderParameters": {
                "PerMaterialMainLightClampFactor": [
                    1.0499999523162842
                ],
                "GameMaterialID": [
                    0.0
                ],
...continued...

There are 5 sections to each material:

  1. The shader itself. There are two for a model with skeletal animation, "shader" and "skinned_shader". (Without skeletal animation, there is only one.)
  2. The texture assignments. Above you can see a base texture (DiffuseMapSampler), a normal map (NormalMapSampler) and a gradient map (CartoonMapSampler).
  3. Tuneable shader parameters. The parameters themselves can be tuned by changing the numbers; however the parameters themselves cannot be changed. For example, if "PerMaterialMainLightClampFactor" is not present, you cannot add it. You also cannot delete parameters that the shader expects to be present.
  4. Sampler definitions. These are settings for the textures, such as whether texture coordinates (UV maps) wrap, clamp or mirror (ping-pong) etc. Similar to #3 above, you can change the settings, but you cannot add or remove sections.
  5. Shader switches. These tell the compiler how to configure the shader (what features are present or absent). These also cannnot be changed.

Modifying the material without changing it

To change textures, see the basic modding tutorial. Note that if you are not changing the material, then you can change the textures but NOT the texture slots. (e.g. If the material expects a normal map, you must provide one.) You cannot simply remove the slot and you cannot add new slots, unless you change the material.

To change the tuneable parameters, simply modify the numbers. Send "BloomIntensity" into the stratosphere if you like. We do not know what the valid ranges are for each parameter. Experiment, or look at other models / materials. Again, you cannot add or remove parameters without changing the material.

To change the texture sampler definitions, you can edit the values. Valid values for m_minFilter and m_magFilter include 'NEAREST', 'LINEAR', 'NEAREST_MIPMAP_NEAREST', 'LINEAR_MIPMAP_NEAREST', 'NEAREST_MIPMAP_LINEAR', and 'LINEAR_MIPMAP_LINEAR'. Valid values for m_wrapS, m_wrapT and m_wrapR include 'CLAMP', 'REPEAT', 'CLAMP_TO_EDGE', 'CLAMP_TO_BORDER', and 'MIRROR'. Again, you cannot add or remove sampler definitions without changing the material. Here is a tutorial on what these values mean - keep in mind the tutorial was written for WebGL, but the fundamentals are the same. See here for a discussion on the 3 different CLAMP options. Knowing CLAMP is optional, 99% of character models use REPEAT and MIRROR. Also, as of ModTools v2.2.4, wrap is confirmed working, min/magFilter status is unverified.

You cannot change the shaders or the shader switches without changing the material. Changing the shaders themselves is changing the materials, see the next section on how to do this. (Changing the .fx file without changing the parameter list will lead to crashes.) Changing the shader switches will have no effect at all; those switches will be compiled into dummy .fx.phyre files that are discarded.

Changing the material

Materials can be swapped out with other valid materials. This is mainly if you need shader features that the current material does not provide.

For this tutorial, I am going to attempt to shorten Emma's skirt (polySurface4962_07) very slightly, using texture transparency. Here you can see I have removed the bottom of her skirt. (I have also deleted the fringe from polySurface4962_06.vb)

Notice however that the transparent part of the skirt is black, instead of transparent:

We can see why when we look at the material shader. Above, I had identified Emma's skirt as "new_uwagi_collada" by peeking inside polySurface4962_07.material. Going into "shaderSwitches" within "new_uwagi_collada" in metadata.json, I can see that "ALPHA_TESTING_ENABLED" is set to "0" instead of "1". Remember, I cannot change these switches.

Instead, I will replace the entire material. I will need to find a suitable donor material, either from this model or another model. Luckily for me, there is a material in Emma's model, "new_zubon_collada", that has identical features enabled, except it also has "ALPHA_TESTING_ENABLED". (Here you can see me comparing "new_uwagi_collada" against "new_zubon_collada" in WinMerge.) There are links to a material database at the top of the article.

I will replace "new_uwagi_collada" with a copy of "new_zubon_collada", while preserving as many tuneable parameters as possible. Note that shaders/ed8_chr.fx#FD475805C05CFAA3CD419F156CC20E72 has been changed to shaders/ed8_chr.fx#CB108FCD611C9550ED008F624F285C83 and shaders/ed8_chr.fx#0FBE93BD2C4F285287841E563DFA6A1C has been changed to shaders/ed8_chr.fx#FAF3642958A16D01BAC185C413FA58CE. This is because #CB108... and #FAF36... are the actual compiled shaders that have ALPHA_TESTING_ENABLED turned on, whereas #FD475... and #0FBE9... have it turned off.

NOTE: The following images may be a bit confusing. The left panel is "new_zubon_collada" which I am using as a template. The right panel is "new_uwagi_collada" after I have changed it. I am doing this to show you what I am copying over, and what I am not copying over.

  1. I am copying over the shaders. I am not changing the name of the material. Also notice I am not changing the textures. However, the slots must match the shader. So if "new_zubon_collada" has DiffuseMapSampler, NormalMapSampler, CartoonMapSampler, then the new "new_uwagi_collada" will need exactly those as well.

I have copied "AlphaThreshold", a new parameter that was not on the right, over to the right. It is a coincidence that ShadowColorShift and RimLitColor happen to be identical; had they been different, I would not have changed them (because I do not want to change the look).

Again, the samplers present must be the same, but the values can be different. This model is expecting the texture coordinates to mirror (ping-pong) on the Y-axis, not wrap (repeat). Let's keep m_wrapT set to "MIRROR".

Optional: The switches should match. We are using #CB108... and #FAF36... and so "ALPHA_TESTING_ENABLED" will be set to "1". (Again, these switches have no actual effect, so you can skip updating them. But I tend to update them anyway. If I were to reuse this entire block later, I will have an accurate list of switches.)

In this case, I copied the material directly from within the model, so the shader files are already in the build folder. If you look inside {Work Folder}/C_CHR005_C10/, you will see ed8_chr.fx#CB108FCD611C9550ED008F624F285C83.phyre and ed8_chr.fx#FAF3642958A16D01BAC185C413FA58CE.phyre present and ready to go. If I had chosen to copy the material from another .pkg (e.g. if I used something from Musse or whatever), I would have to copy the corresponding ed8_chr.fx#_____.phyre files into my build folder.

This time, after I compile the model, the transparency is working!

UV maps and partial texture transplant

For a slightly more complex tutorial on texture manipulation than the basic tutorial, I am going to remove Emma's tights. This is a texture transplant tutorial, where I will move a portion of one texture to another texture, which will necessitate understanding texture coordinates (UV maps). (I suppose this will be slightly NSFW, discretion advised. Nothing naughty in-game will be seen though! Understand that even Falcom's 3D modelers have to do this stuff to make proper models.)

You should be comfortable with all the steps of the basic tutorial before attempting this tutorial.

  1. I will again be working with Emma's CS4/Reverie default that I used in the basic tutorial (C_CHR005.pkg). I will need a texture without tights, so I will borrow that from Sharon (C_CHR025.pkg) as the UV map is compatible. Decompile both models.

  2. First, understand that Blender does not support BC7 textures, which CS3+ games use. So let's convert the textures we need to PNG. You certainly can just convert all the textures to PNG (I do that, using Texconv), but we will just convert what we need in this tutorial. Emma's tights (legs) are part of polySurface4962_04.vb. Looking at meshes/polySurface4962_04.material, we see her tights are assigned to "chr001_munamoto" in metadata.json. In metadata.json, we can see we need to edit "assetconv_temp/dds_dae/D3D11/chr005_03_conv.dds" so open that file in Paint.NET, then save it as a PNG.

  3. In Blender, import polySurface4962_04.vb from Emma. In the Topbar, go to the Shading workspace. Notice that unlike the Layout workspace, this workspace has by default 6 editors: File Browser and Image Editor on the Left, 3D Viewport and Shader Editor in the center, and Outliner and Properties on the right.

  4. In the Shader Editor (center bottom), you will see an empty material node tree. In the Header of the editor (the top bar), press New to place down a new material (Principled BDSF will show up). Find your chr005_03_conv.png in Windows File Explorer and drag it onto the node tree space, to the left of the material. Connect the two by drag-clicking from the yellow dot in the upper right corner of the png (Color node) to the yellow dot on the upper left corner of the material (Base color node). Your workspace should now look like this:

When you do this tutorial, you probably won't see the skirt covering the underwear, but for the purposes of the tutorial I've kept it covered.

Texture coordinate wrapping, mirroring, and other nonsense

Now is actually a good time to talk about coordinate wrapping, mirroring, and other nonsense. To make the life easier of 3D Modelers, texture coordinates can be off the texture, and the texture will repeat over and over again so that the coordinates will once again fall on the map. This is advantageous so that, for example, the texture of the right hand and the left hand can be the same, and the texture map can be separate for each (instead of the two being right on top of each other. For example, here is the map for Emma's skirt, which is polySurface4962_07.vb:

You can see that the texture coordinates on the left are meant to be projected onto a mirror image of the texture on the right. And looking at "new_uwagi_collada" which is mapped to polySurface4962_07.vb, we can see the texture does indeed dictate mirroring on the X-direction and Y-direction. (S/T/R {and U/V/W} is X/Y/Z)

                "DiffuseMapSamplerSampler": {
                    "m_minFilter": "LINEAR_MIPMAP_LINEAR",
                    "m_magFilter": "LINEAR",
                    "m_wrapS": "MIRROR",
                    "m_wrapT": "MIRROR",
                    "m_wrapR": "REPEAT",
                    "m_lodBias": 0.0,
                    "m_maxAnisotropy": 0.0,
                    "m_borderColor": 0,
                    "m_baseLevel": 0,
                    "m_maxLevel": 1024,
                    "m_flags": 0,
                    "mu_memberLoc": 94205,
                    "mu_memberClass": "PSamplerState",
                    "mu_gltfSamplerIndex": 140
                },

Interestingly, the normal map is MIRROR/REPEAT/REPEAT instead of MIRROR/MIRROR/REPEAT. Luckily, if you look at the actual texture coordinates, all the Y coordinates are between 0 and 1 (the bottom and the top of the texture), so it does not matter if Y is MIRROR or REPEAT. And this is not a 3D texture, so Z (m_wrapR) does not mean anything.

This means that the game will repeat the textures over and over like this when mapping the coordinates:

When you first apply the texture in Blender, however, it looks like this:

That is because Blender wraps (repeats) by default, instead of mirror (ping-pong). It is repeating the textures like this instead:

If you really wanted to see what the texture should look like in Blender, you will need to recreate this node tree:

And since the normal map is MIRROR/REPEAT/REPEAT, this is the correct node tree for the normal map:

Note: You do not need to re-create these node trees, unless for some reason you need to see the results in Blender. I have only put them together for illustration purposes. However, if you want to re-create the node tree, they can be found in the Add menu at the top of the Shader editor, right above the nodes.

UV Map: Add -> Input -> UV Map
Separate / Combine XYZ: Add -> Convertor -> Separate / Combine XYZ
Ping-Pong / Wrap: Add -> Convertor -> Math

Please try this interactive wrap tool to see REPEAT, CLAMP_TO_EDGE and MIRROR in action! This tool is from here.

Obtaining a UV map for evaluating textures

In this tutorial, I will not be changing the UV map; instead I will be changing the texture. You certainly could change the map instead, and I will show the basic steps below. For now, we will get the UV map so we can manipulate the texture in Paint.NET.

  1. Now that you have applied chr005_03_conv.png to polySurface4962_04.vb, make sure polySurface4962_04.vb is selected, and go to the UV Editing workspace on the Topbar. Note that two large editors predominate this workspace, UV Editor on the left and 3D Viewport in the middle. Outliner and Properties continues to be on the right.

  2. In the 3D Viewport (center right), be sure that polySurface4962_04.vb is the active mesh, and the viewport is in Edit Mode. Everything should be selected (in bright orange); if this is not the case, Select Menu -> All (Press A) in the 3D Viewport. Only vertices selected on the right will show up in the UV editor on the left.

  1. This time we are lucky, the UV map is entirely within the texture, no need to worry about wrapping or mirroring. Select the entire map, Select Menu -> All (Press A) in the UV Editor. Export the map, UV Menu -> Export UV Layout. Save the PNG file.

Note: Often the map will not actually be on the texture, and will rely on texture mirroring / wrapping. For instance, this is a very common scenario in the Cold Steel games:

In order to export and preview this UV, you will need to bring it down to exactly where it would be if it were not wrapped/mirrored. In the Sidebar, open the Image tab, and click on Y. Here it is 2.40171, so subtract 2 and make it 0.40171. If it is negative, then remove the whole number component, then add to 1. For example, if it is -2.59829, then remove the whole number component (the 2, resulting in -0.59829), then add to 1 (1 + -0.59829 = 0.40171).

Transplanting the texture

  1. Open chr005_03_conv.png (or chr005_03_conv.dds) in Paint.NET. Drag your new UV map (polySurface4962_04.vb.png) that you exported in the step above onto your texture. It will ask if you want to open the image or to Add Layer, select Add Layer. You will now have the UV map on top of the texture as a separate layer, and you can hide the layer or reduce the opacity if you like to see the layer below better. We want to replace the area in the red circle.

  1. Open (as a separate image) our donor texture, which is chr025_03_conv.dds from C_CHR025.pkg. The same area is pointed to by the red arrow. Notice that it's in a different place, is a different size, and is a different orientation. We will need to move it over.

  1. First rotate the image 90 degrees, Image Menu -> Rotate 90° Clockwise (Ctrl-H). Then select the leg texture using the select tool (Press S), and copy it, Edit Menu -> Copy (Ctrl+C). Be sure to grab the skin above the underwear as well, Emma's UV map requires that section.

  2. In the Emma texture we are working on, paste the leg texture as a new layer, Edit -> Paste into New Layer (Ctrl+Shift+V). Notice that this layer is at the top, above even the UV map. Drag the layer in the Layers Window so that it is between the UV map and the base texture.

  1. Using the Move Selected Pixels tool (Press M), drag the new texture so that it is under the UV map exactly where it needs to be. It should look like this:

Note we are lucky here, it fits pretty much exactly. Sometimes we need to elongate the legs, etc. But always verify that everything is where it needs to be, especially the knees!

You will notice in the screenshot above that there is a lot of Sharon's map overlapping Emma's map, as my cut and paste is quite crude. This does not matter, because none of those areas are used by this mesh, as you can tell because none of the UV is touching those areas. We will only use this edited texture for this specific mesh, so we do not need to be precise. All the remaining meshes will use the stock chr005_03_conv.dds instead.

  1. Hide the UV map and save the texture as a new texture in dds format, BC7 UNORM (Linear), in the model texture folder (assetconv_temp/dds_dae/D3D11). See the basic tutorial for a screenshot of the exact settings. I am naming the file "EmmaNoTights.dds".

  2. Edit metadata.json, specifically the "chr001_munamoto" material which belongs to polySurface4962_04.vb, setting "DiffuseMapSampler" to "assetconv_temp/dds_dae/D3D11/EmmaNoTights.dds".

Note: Depending on your material, you may also need to edit the normal map to match. In this case, the normal map matches, so we don't need to do that.

  1. Compile and test your model. Congratulations on your mod, you have made it to the end of this tutorial!

*Note: You may wonder if you need to bother with the underwear etc. It is generally a good practice to do so, in case the odd camera angle shows your work. However, getting it very precise laterally is more important, or strange things like black streaks might run down the legs from the underwear texture, etc.

Manipulating the UV map itself

Above, I chose to paste Sharon's leg texture directly into Emma's texture map, exactly where the prior map was. I could also have chosen to use Sharon's texture directly, and moved the texture coordinates instead. I chose not to do this, since there are other parts of the map that polySurface4962_04.vb was using, such as the hands. But if you want to edit the UV map, it is very straightforward to do so.

All the same things you learned in the basic tutorial for manipulating the 3D Viewport and selecting, you do here in the 2D Viewport. Use the circle select, grab the vertices you want to transform, and transform them (transport, rotate, scale) using the tools. Do not forget to go back into Object mode in the Layout workspace when you are done, and export the mesh.

Clone this wiki locally