add shader to mgba canvas

[sonht0555]

I'm planning to add shader layers to the surface of the canvas, but I don't know which function allows redrawing the canvas so it can be edited in the frontend. Currently my rendered canvas on the frontend is simply black but in reality the backend is using it to render the simulator

Is there a way for me to use OpenGL to display the emulator on the frontend so that I can easily draw a pixel grid and have the emulator's image displayed on it?

[sonht0555]

By the way, can you build a version of https://github.com/thenick775/mgba/tree/feature/wasm like this from skyEmu, the control keys and game settings are called from the frontend, not from the in the wasm file like the version that sky builds https://web.skyemu.app

[thenick775]

This is on the roadmap, but will most likely function very similarly to the core listed above for mGBA, and will take me quite some time on my own given the level of complexity and my lack of expertise with that core.

This is pretty far out on the roadmap, if you'd like to assist, please create a fork and make a PR here if you would like to help get this feature off of the ground.

Control keys and game settings can be modified and called from the frontend for the mGBA core, in my source code see how I press virtual keys (buttonPress/buttonUnpress), rebind desktop keys (bindKey) for quick examples.

In this case the frontend is responsible for storing and maintaining all settings across core refreshes/restarts. I mainly use localStorage for this.

[sonht0555]

I really want to help you, but my abilities only stop at the frontend, I don't know anything about C. But you just let me into that RP. I will try to support you. SkyEmu is actually very good to use on mobile. but its user interface is very limited. If you can separate it like the Mgba core you are doing, it will be really good

[sonht0555]

I also built mgba emulator from your wasm above. I, like you, mainly use localstorage to store data. Looking forward to you also making skyEmu core like this mgba core. It's really easy to build a UI from your above repo branch, almost all the necessary modules are complete to call.

[sonht0555]

This is pretty far out on the roadmap, if you'd like to assist, please create a fork and make a PR here if you would like to help get this feature off of the ground.

I'm not really familiar with git, so I don't know how to create it like you're saying. I don't know how to create it. But I really want to help you, I also like emulator like you

[thenick775]

In the current mGBA version, SDL through emscripten does all the rendering in this case, see here for the full rendering implementation:
https://github.com/thenick775/mgba/blob/feature/wasm/src/platform/wasm/main.c#L82-L88

Meaning in this case, mGBA and emscripten together own drawing to the canvas, which is opaque to the frontend for the most part.

For further information on how this works in SDL see the wiki page here:
https://wiki.libsdl.org/SDL2/SDL_RenderPresent

You would need to use shaders that are compatible with the WebGL-friendly subset of OpenGL ES 2.0, I haven't had much luck doing things like that though, see my canvas reset code for an example of why.

I was not able to easily work with shaders myself to get the desired LCD fade effect, as I'm not the best with webGL, the best I could do was clear colors with my knowledge, so any LCD fade effects you see here are done on a 2d canvas copy of the webgl canvas that I can then use the usual methods to manipulate and draw.

You should be able to use webgl shaders if you have the knowledge, but you'll have to work with mGBA drawing to the canvas buffers itself, I'm not planning to change how that's done at this time.

To break down some further talking points:

I'm planning to add shader layers to the surface of the canvas, but I don't know which function allows redrawing the canvas so it can be edited in the frontend. Currently my rendered canvas on the frontend is simply black but in reality the backend is using it to render the simulator

screenShot is the only javascript function that exposes on-demand fills of the webgl canvas buffer. Rendering is handled in the testloop opaque to the frontend using the canvas.

Is there a way for me to use OpenGL to display the emulator on the frontend so that I can easily draw a pixel grid and have the emulator's image displayed on it?

You could copy the webgl canvas context to a 2d canvas and use that to render what you want, and perform any manipulations manually, but I would expect this to be much slower, where utilizing webgl shaders and attaching this to the webgl canvas context should work, but I don't have enough knowledge with raw webgl to be of much help myself there.

In essence this is what emscripten/mGBA is doing for me in my frontend, where the canvas is the pixel grid, specified by width and height attributes, as those specify the buffer dimensions. In the wasm c binding code, I use the mGBA constants GBA_VIDEO_HORIZONTAL_PIXELS/GBA_VIDEO_VERTICAL_PIXELS for setting those properties whenever the core is instantiated.

[sonht0555]

I understand, you are also quite struggling with this part of OpenGl. I'm trying to learn it too, simply I'm wanting to redraw the canvas with openGl and re-scale the pixel grid every time the canvas changes size. Doing that then I can apply lcd pixel mock filters on the pixel color layers. You also see the colors displayed on skyEmu, right? I really want to do what sky is doing for your emulator. Reading your code, I see that it uses OpenGl 3.0, but it seems that browsers cannot use OpenGl 3.0 yet.

[sonht0555]

Thank you very much for the detailed explanation above. But if possible, can you help me write the code that initializes the canvas with openGl and redraws it to simulate display on the frontend? Like the function screenshot you just wrote for me, but it can only draw the image at that time, not continuously like the canvas is displaying from mgba and emscripten running under the backend.

I really want to try applying effects, or standardizing the colors on the canvas

[thenick775]

I've tried a few variations of something like this in the past:
https://stackoverflow.com/a/71604095

I will say, I was not successful, even attempting to draw triangles in a way similar to this simply cleared the canvas buffers.

This is most likely due to my inexperience with webgl. Feel free to try something like above, but I most likely won't be able to help you for this request.

[thenick775]

Just circling back here as I have a better idea of what mGBA is doing under the hood.

I don't think it will work well to do anything with the webgl canvas context on the frontend, you'd most likely need to go into the SDL/renderer code in mGBA and write your shaders there with OpenGL instead of attempting to use WebGL