Introducing a Dolphin graphics mod editor

[iwubcode]

This is a very early look at a new feature for graphics mods. The ability to edit them in realtime using an editor!

I'll start with some shiny features.

When you start the editor up, you will be greeted with a simple menu. File > Open to open an existing mod*. File > New > Inspect allows you to use the properties/scene panel to look around and get comfortable with the editor or File > New > Project to create an actual new mod and have access to assets ( as well as save your work! ).

Once you have a new mod project started, you will see the scene panel. The scene panel allows you to view targets (currently displayed by texture name). Whatever is chosen in the scene view will be highlighted in the game:

In addition to viewing selections, you can also rename the objects. The editor will keep track of them as they live throughout the game:

But the graphics mod editor wouldn't be much if it just was used to view the scene. It can also attach graphics mods to targets. The most simple are still the "skip" and the "move" actions and you can see me moving an element below:

Finally and really most importantly, the editor allows the user to create and edit materials dictated by a shader:

Why?

As I've talked to various users, they've all agreed that the new features for graphics mods were really interesting but they all voiced one concern: the complexity. One of the reasons things are so complex is to allow for a more flexible system and a way to provide more features. An editor simplifies some of the complex pieces and enables the ability to add even more features that weren't possible.

Notes

My plan is that this PR will be somewhat long lived. I hope to break off pieces into separate PRs, slowly working toward a merge of the feature. I want to take into account any user feedback and give this a bit of time. I also will probably add new features. I do not want to rush this because once this is merged, some sort of versioning structure will need to be defined for user mods...

The editor is currently always on in this PR for ease of testing. I haven't exactly decided on how or rather where to expose a setting to turn it on.

Future Work

This opens up so many possibilities, some are listed below:

• Due to the visual nature, we could display lights and allow users to add new lights. Of course this requires some changes to the lighting system..
• Cube map integration
• Drag/drop from outside Dolphin (ex: a texture)
• A visual shader editor
• Undo system
• A rewrite of graphics mods to not just target things by texture. The texture targeting was a necessity because Dolphin had no way to allow for anything else. This will have many benefits..
• Integration with fifo log or possibly other data being displayed. It may be interesting to enhance the editor to support more than graphics mods

[iwubcode]

Still working out the other builder issues.

I've been researching different capabilities to prepare graphics mods 2.0 (gm2.0), the feature that will rework graphics mods to allow for better detection and integration with this editor!

This update takes a step toward implementing gm2.0. I don't think anyone besides me has been using this feature but any saved projects are invalidated with this change. My thought is there will be one more break as I plan to finish gm2.0 in the next push.

But enough of that, onto the changes.

Property Panel

The property window has been updated as part of gm2.0, to now include all the textures assigned during a particular draw call:

Additionally, you'll see other properties that you didn't see before, culling, blending, and projection type (there is also "world position" but that is experimental...)

[[ Experimental ]] Light Panel

Lights are now shown in the light panel. This may not stay in the final build. Targeting lights is very difficult because the information provided to Dolphin is not static. Something like position is relative to the game's camera. The current implementation tries to target a light based on the non position based properties but that isn't very effective.

Clicking on a light in the panel will blue color the lighting contributions of those lights so you can see where they are in effect:

(red circle for emphasis to show all the light's contributions)

In the future, lights will probably be all off or on. However, I do plan on supporting custom lights as part of gmod2.0.

Scene Panel

The context menu for working with scene objects has been updated into individual sections:

Mesh Exporting

Yes, you read that correctly. Dolphin now has the ability to export meshes. Keen-eyed readers will have noticed the "export" option above. Individual meshes can be exported, or the entire scene can be exported:

Meshes are exported in the open source format "GLTF". Meshes are exported to the Dump/Meshes folder. Textures for the meshes are dumped to Dump/Textures as usual.

Mesh details and textures are automatically set, allowing full scenes to be dumped with relative ease. However, some properties are still outstanding like blending.

It is worth noting, that "skinned" meshes (meshes that are animated) come in two flavors. Some skinned meshes are implemented by the game using the CPU. These are like moments in time:

(The main character in Opoona is implemented as a CPU skinned mesh, caught in the moment of a quirky pose; viewed in Blender)

However, GPU skinned meshes will be in their TPOSE or rest pose:

Unfortunately, due to Dolphin just building an index buffer as it gets triangles, some skinned meshes are completely broken:

But viewing the points, you can see the data is mostly sensible:

I see the issue but it's not clear to me why the data is in this layout. I have confirmed a couple of games on disk and they don't match, so not sure why they end up that way. Regardless, Dolphin can't read them at the moment.

Mesh Importing

You didn't think I'd implement mesh exporting without importing did you?
Importing is also possible! However, you can only replace static meshes at this time.

(the image with the bag is the original, the latter is a tree I made in Blender and used as a replacement)

Just like with exporting, importing takes a "GLTF". Right click on the GLTF in the asset window and choose "Import" then Dolphin specific files will be generated. These can be included in the project. Eventually, we will support dragging and dropping a GLTF.

Materials are not imported at this time because we lack an included PBR shader in Dolphin. However, you may match the mesh materials with Dolphin materials.

Just like with exporting, there are some issues. Some models I have found fail due to some requirements Dolphin enforces (I need to catch these and suggest workarounds). Models that have their mesh negatively scaled will also have strange rendering.

(Mythra from Xenoblade2 has a negative scale to mirror her body, this seems to cause issues when rendering due to the UVs being incorrect, subtracting 1.0 from the x component fixes it but then flips the other side)

.

It's a small step in the right direction but there are countless features needing to be implemented and an obvious layer of polish missing. If anyone is a graphics programmer and interested, I'd love to have some help (please ping me on Discord in Dolphin server if interested) . For everyone else, please have patience and I promise good things will come.

Until next time!~