Introducing a 'GraphicsMod' system for creators

[iwubcode]

Introducing a new configurable graphics system that I'm dubbing "GraphicsMod". This idea has been in the works a while and finally can be shown off.

What is a 'GraphicsMod'

What is a "GraphicsMod"? It's a feature in Dolphin that allows modifying various graphical operations in Dolphin.

A graphics mod consists of a series of features. Each feature is a group name and an action to apply to that group. A group consists of a list of targets (the graphic operation) that a given action will be applied to.

A mod doesn't have to provide groups and features. We can provide mods that just give groups at the Dolphin system level. This allows us to define objects per game but for Dolphin to interact with these objects in a generic way.

Actions Supported

Not too many actions are available at the moment. I've kept them very simple: Skip, Print (for debugging), Move, Scale.

What does it do

There are a couple benefits to users at the moment, with many more to come in future PRs if this gets accepted.

Bloom

I've included in this Dolphin build, definitions for a few games that have issues with bloom. Examples: Arc Rise Fantasia, Little King's Story, Last Story, and Mario Kart Wii

With these definitions, a simple graphics mod to remove bloom looks like this:

{
	"meta":
	{
		"title": "Bloom Removal",
		"author": "iwubcode",
		"description": "Remove bloom"
	},
	"features":
	[
		{
			"group": "Bloom",
			"action": "skip"
		}
	]
}

But at the same time, here's a mod to use native bloom:

{
	"meta":
	{
		"title": "Native Bloom",
		"author": "iwubcode",
		"description": "Bloom at native resolution"
	},
	"features":
	[
		{
			"group": "Bloom",
			"action": "scale",
			"action_data": {
				"X": 1.0,
				"Y": 1.0,
				"Z": 1.0
			}
		}
	]
}

Any game that uses this mod and defines the group "Bloom", suddenly has this feature available. That's the power of graphics mods!

Mario Kart all bloomy:

Mario Kart no bloom:

Arc Rise Fantasia hi-res bloom issues:

Arc Rise Fantasia bloom scaled:

HUD

Likewise, games can also define textures as being part of the HUD. Doing so allows you to easily remove the HUD:

{
	"meta":
	{
		"title": "Remove HUD",
		"author": "iwubcode",
		"description": "Removes the game's HUD by moving it"
	},
	"features": [
		{
			"group": "HUD",
			"action": "move",
			"action_data": {
				"X": 10000.0,
				"Y": 10000.0,
				"Z": 10000.0
			}
		}
	]
}

I could have used the "skip" action to actually remove the drawing of the HUD. However, I used the "move" action here to illustrate how to pass data to the action at configuration time. This allows creators to provide custom packs to change the layout of the screen. This can also be used to support Ultrawide monitors.

Fragile with a HUD:

Fragile with no HUD (much better for screenshots!):

Defining targets

Defining targets is fairly easy. There are currently three supported targets: drawtexture, projection, and efb.

The way we distinguish targets is by texture. For drawtexture and projection this is a texture name that is what authors of texture packs are used to.

For efbs, I decided to use the efb details ( size, format ) to distinguish. This does mean there is the possibility for overlap with other effects but given the dynamic nature, it is very difficult to distinguish in any other meaningful manner.

All types use the dump feature, to make defining targets easy. Just dump either textures or efbs and copy / paste the filename (without extension) into the json document. A final document looks something like this (cut down for readability):

{
	"meta":
	{
		"title": "Bloom and HUD definitions",
		"author": "iwubcode"
	},
	"groups": [
		{
			"name": "HUD",
			"targets": [
				{
					"type": "draw_started",
					"texture_filename": "tex1_96x96_7b5b0f693c1200ad_5"
				},
				{
					"type": "projection",
					"value": "2d",
					"texture_filename": "tex1_96x96_7b5b0f693c1200ad_5"
				}
			]
		},
		{
			"name": "Bloom",
			"targets": [
				{
					"type": "efb",
					"texture_filename": "efb_frame_320x240_6_c1"
				}
			]
		}
	]
}

(astute readers may notice that the projection and drawtexture is using the same texture name, that is so that the texture can be moved or skipped)

UI

Graphic mods are enabled under the "Advanced" tab next to texture loading:

They can be individually enabled by right clicking on the game of interest, it looks like this if you don't have graphic mods enabled:

Once it is enabled, it looks like:

Clicking on a mod:

Outstanding Items

No outstanding items

Work completed:

• Code cleanup
• Tweak UI
• Fix Performance Issues

Future Work

• VR / Stereoscopic rendering - add an action to give 2d draws some depth.
• Expose the action API to scripting or possibly a shared library interface to allow users to create their own actions. One advantage might be to allow them to draw something of their own (like a different HUD)
• Hook into Post processing shader system overhaul! #10362 to allow post processing effects on specific game graphics (ex: depth effect before bloom or HUD)
• FreeLook - add an action to inverse or fixup game actions (ex: shadows) that don't always play well with FreeLook. This may or may not be possible (needs investigation)
• Add an action that allows you to attach custom shaders to specific textures
• Dynamically enable mods based on memory addresses existing or equal to a value

[iwubcode]

Here's a silly repo of the graphic mods specified in the opening post: https://github.com/iwubcode/DolphinGraphicsMods

The graphical target definitions for a handful of games are in Dolphin's system directory.

[lioncash]

Just a cursory review

[TryTwo]

I can't get it to show/load any mods. The uploaded .txt files for the games are all empty, so I may not be filling them correctly.

[iwubcode]

I can't get it to show/load any mods. The uploaded .txt files for the games are all empty, so I may not be filling them correctly.

Yes, the txt files are empty, the code is just looking at the filename.

Copy from this repo the 'GraphicMods' folder to the 'Load' folder in your user directory (replacing the existing 'GraphicMods' folder if there is one). Then right click on a game in your game list and go to "Properties". Look at the "Graphic Mods" tab. If there are no definitions for a game, then the mods won't show up.

Available definitions are (not necessarily complete):

• Arc Rise Fantasia
• Last Story
• Little King's Story
• Mario Kart Wii
• Rune Factory Frontier

I'll be uploading more soon.

[iwubcode]

Updated to add a few more games that suffer bad bloom:

• Donkey Kong Country Returns
• Link's Crossbow Training
• Okami
• Twilight Princess
• Xenoblade Chronicles

I also added the HUD for Fragile.

Last night I tried to make a remove DOF solution for Last Story, I got pretty close but err, ran into some issues. Need to investigate more if this is possible.

I'm still hitting some performance issues on some games (Sin and Punishment Star Successor and parts of Last Story). Not sure what I'll be able to do yet.

[TryTwo]

Not working for me. Sometimes crashes if I open a game's properties. Sometimes crashes if I launch Mario Kart. Would be nice if someone else gave it a try.

/edit sometimes it doesn't crash. It opens the correct metadata file, but nothing appears in the widget.

[iwubcode]

@TryTwo - thanks for testing. Very strange, I've never had a crash. I will try messing around to see if I can reproduce it somehow.

Other than my dev environment, I've also tried pulling the software built here down and running it in its own environment (portable mode).

[iwubcode]

@TryTwo - I added more logs and error checking.

Do you see any errors in the logs? Does it still crash?

If there's any other information you can give to help reproduce the crash, that would be helpful.

[TryTwo]

/edit the errors I posted happen with any version. Just standard error output I guess. No other apparent errors. Possibly just related to this not loading correctly.

The only log output "refresh list" gives me is:
55:17:100 Common\FileUtil.cpp:916 I[COMMON]: GetSysDirectory: Setting to D:\Dolphin Emulator\0Dolphin-x64/Sys/:

I can breakpoint and run debug, if that helps. It looks like it loads the json in the first part of "refresh list" function.

[iwubcode]

@TryTwo - thanks. Yeah saw your original comment, that ObjectLabel error I fixed in my post processing review, I probably need to pull that out.

Did you copy the 'GraphicMods' in the repo to <user>/Load? Which game are you attempting to view?

A breakpoint may help if you have time. You'd want to put it in GraphicsModGroupConfig::Load(). Assuming you are trying MarioKart. A successful process should yield 4 values in m_graphics_mods on the last line (155). Here's a screenshot:

Another user had a similar issue when they were trying to create a mod for a new game. We couldn't figure out the issue and they weren't skilled with the debugger unfortunately. They could load existing mods however.

[TryTwo]

Yes mario kart.

I did not copy that over from the repo. So I should have documents/dolphin/load/graphics mod /all games native bloom/ & /Mario Kart Wi/i folders?

I also accidentally put GraphicsMod in a dolphin/sys/load folder and it was being pulled from that along with documents and screwing things up slightly.

The crashing stopped. I'm close to getting it working I think. Need a double check on those folders.

[iwubcode]

I did not copy that over from the repo. So I should have documents/dolphin/load/graphics mod /all games native bloom/ & /Mario Kart Wi/i folders?

Yes, copy the "GraphicsMod" folder in that repo to the user/load/ directory of your Dolphin install.

The Sys/Load/GraphicMods folder will contain all the texture definitions for various games (Mario Kart, Xenoblade, etc). The user load has what users would consider the mods "All games native bloom", "All games bloom removal" etc.

Technically it doesn't matter which you put each in. You can have them all in user/load or all in sys/load or mix and match, any option should work.

[TryTwo]

It was double loading if they are duplicated between the two directories. It was working for a second then went back to not working. I think I should check /config/GraphicsMod/ json. Where would I breakpoint for that?

The other data seems to consistently load.

[Narugakuruga]

Is this possible to make a feature that manually overrides arbitrary mipmap?
Dolphin's automatic detection cause many false positive. So it might be very good to add some "black list " or "white list" for arbitrary mipmap.

I think this can improve texture quality

[iwubcode]

@TryTwo - I was wondering if you might have had some bad config built up. The config is loaded in that same function I mentioned, it is the variable file_path. Maybe I need to rework that idea bit.

@Narugakuruga - ah, I see what you mean. What kind of false positives do you see? Any game / texture-id example? This isn't supported at the moment but it's something I can look into in the future.

[TryTwo]

if (m_mod_group.GetOrder().size() != m_mod_group.GetMods().size())
return;

Is failing with 4 vs 5. If I bypass I get 2 entries in the graphics mod list for MKW. I deleted and restarted a few more times and it might be working.

How many entries should MKW have in the mod list?

[iwubcode]

Thank you so much for all the testing @TryTwo . I will look at how I can improve that.

If I bypass I get 2 entries in the graphics mod list for MKW

Yeah that sounds correct.

[TryTwo]

MKW should use 152 114 "efb_frame_152x114_6_c49"

Thanks for helping me. I'll try to test Metroid later.

[vlad54rus]

Shadow the Hedgehog uses "efb_frame_64x64_6_c5" and "efb_frame_128x128_6_c2".

[vlad54rus]

How do i use "move" and "scale" action at the same time? Unscaled bloom with SSAA x2 and Vertex Rounding looks fantastic, but appears offset a little, i'd like to correct it.
Also, my GraphicMods menu appears empty for some reason, although the effects themselves do work Nevermind i got a typo in the metadata file.

[Narugakuruga]

@TryTwo - I was wondering if you might have had some bad config built up. The config is loaded in that same function I mentioned, it is the variable file_path. Maybe I need to rework that idea bit.

@Narugakuruga - ah, I see what you mean. What kind of false positives do you see? Any game / texture-id example? This isn't supported at the moment but it's something I can look into in the future.

Some game use a set of different mipmap to create visual effects (usually water and lava). Mario Galaxy and Monster Hunter Tri for example.
Dolphin has an option to detect these mipmap and use the "original mipmap" which can create those visual effects. But it always mistaken some normal terrian texture as special mipmap. And use original low resolution mipmap for them.

My idea is add a list for known special mipmap (which is called arbitrary (ARB) mipmap in dolphin ) .So eliminate the inaccuracy of Dolphin auto detection method. And use original mipmap for those.
In a game there's usually many mistaken normal texture, but only a few special mipmap texture. So the benefit will be more than the complexity.

Every modder can create a list for their favorite game

[iwubcode]

@AdmiralCurtiss asked that efb_frame and xfb_copy be changed to efb1 and xfb1, so that's done. Updated all the configuration to match. Also moved all the number values to the front to match (though the code supports both front and back).

Finally, I've been sitting on a Super Mario Strikers Charged bloom mod. So I added that (fixes bad bloom in the lava level).

[sepalani]

I found some minor coding style issues.

[sepalani]

The capitalization of some words at the start of some lines and the lack of punctuation (. or , or ;) feel strange. The way some parts are formulated feels misleading to me, for instance:

// If the user changed the game mods in their last game
// Then loaded a new game, (...) UI.

Shouldn't the word Then be replaced by and instead? Or am I misunderstanding it?

[iwubcode]

I've never been great at punctuation but this was written very quickly (and at the end of the night in hopes that a merge would soon follow). So my apologies for its poor quality.

I'll rewrite this.

[iwubcode]

@sepalani - done. Thank you for the review, let me know if you like the comment any better!

[JMC47]

In a stunning turn of events, I didn't read closely enough again, causing me to not do things correctly and be confused.

Overall, Graphics Mods have a bit of a learning curve to how they're structured and work if you're planning on making them, but they're very efficient in what they do, and the two-tier nature of them makes it easy to "add" support for more games if you're doing mods like removing bloom, depth of field, etc. From a creator side, I could see myself potentially using this - it's not so complicated I can't use it, but it's also powerful enough to do what it needs to do. Not exactly easy to use, but not impossible to learn.

User wise, I had a bit of confusion because if you just download this Pull Request, there are no default graphics mod commands! Once I downloaded some Graphics Mods it was easy, and they can be turned on/off really easily. You can check them while in-game, to see the difference first hand.

They work, they do some pretty cool things, and the future is bright as more features are added onto this. This initial pull may seem a bit lackluster, but I think it does enough to warrant existing and being merged. We hate bloom issues, and this gives us a pretty easy way to disable bloom or even upscale bloom.

I tested Arc Rise Fantasia and the bloom disable and bloom upscaling worked fine.

Default Settings

Bloom Disabled

Native Bloom

One potential gripe - I don't like the name "native bloom" for the proper upscaling bloom. I get that it makes the bloom always render at the correct (native) resolution, but it just makes me think of native (default) bloom. I don't know of a better name but just figured I'd mention it.

[Pokechu22]

To my understanding, "native bloom" does use native resolution for the bloom itself - i.e. it selectively disables "scale EFB copy" for the bloom texture, but when drawing the bloom to the screen it still gets linear filtering. It's not as fancy as other techniques that could be used for upscaling. But naming it "native resolution bloom" would reduce confusion. That's in https://github.com/iwubcode/DolphinGraphicsMods so it doesn't need to be handled by this PR.

I too initially got confused by nothing happening by default. It might be good to indicate that targets are present for a game on the UI in some way, but I suspect this confusion more applies to developers that know ahead of time that this PR adds a folder containing data of some sort, and less to users that don't see the added files.

[iwubcode]

Yeah, mentioned to JMC but the idea was I'd wait to include bloom actions once I had one that didn't cause the "shimmer" that the native resolution bloom does. That'll be possible once we have an action that can apply a downsampling or blurring shader to an EFB copy.

But I'd also be ok adding what I have in my separate repo now, just to get things rolling..

The movement/scaling could also be improved. It's currently just a projection matrix applied (much like the FreeLook camera) but that can cause unpredictable results depending on how the game transforms the object in question. A smarter transform would probably modify the vertices themselves and use an 'anchor point' (center/upper-left/etc). That is much more complicated, which is why it isn't here right now..

[sepalani]

Code LGTM, untested.

[mbc07]

So, I downloaded Dolphin 5.0-16766 and no matter what I do, the list from the graphics mod tab is always empty? I own several of the games that ships with built-in Graphics Mods but they won't show in the Graphics Mod tab at all, even after activating "Enable Graphics Mods" on Dolphin's Graphics Settings. Other things I tried:

• Copy all content from Sys\Load\GraphicsMods to User\Load\GraphicsMods
• Rename the TXT file with the game ID from the 3 letters ID to the full 6 letters ID
• Access the graphics mod tab while the game is already running

Am I missing something here?

[iwubcode]

@mbc07 - there are no built-in mods currently, only built-in definitions.

I am planning on submitting the mods as a PR in the next day or so. If you can't wait, you can download some in the first post of this PR.

[mbc07]

Understood. I'm currently testing the UI translation (source strings got updated a while ago) so I'll grab some. Thanks for the explanation...

[JosJuice]

I would like to note that due to the merge timing of this PR, we unfortunately most likely won't get translated strings for it pulled back into Dolphin before the next beta is released.