-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CSM Events and Drawing Testing #10524
Conversation
This all looks very promising! I'm very excited about the prospects of this new framework. Based on the description, I suspect it may address some of the feature requests that I had planned to submit awhile back: Popup Formspecs Formspec Page Transitions Full HUD Replacement NB: I wrote these proposals in the context of the current formspec API. So naturally, they would need to be adapted for this new framework. But I at least wanted to present what I had brainstormed thus far for your consideration. I think all three are absolutely worthy candidates for a next generation, CSM-based approach to the Minetest GUI. |
This comment has been minimized.
This comment has been minimized.
@paramat When the Lua-side GUI and HUD are created, then they will be loaded separately from the rest of CSM, so basically the API you see above and maybe one or two other things will be loaded. So, it will still be Lua, but not CSM. @sorcerykid |
v-rob, so are you saying that if SSCSM never happens this can be easily altered to be a formspec replacement? Why not do this the other way around, code this to be independent of CSM and alter this if SSCSM is added? I have seen you discussing this in IRC, i read all minetest-dev logs =) |
A little more like the other way around: it was designed for the formspec replacement in the first place with SSCSM in mind as a future extension. Currently this is in CSM because that was easiest to do (this is my first time working with C++ <-> Lua code), but it's going to have to be moved anyway, so yes, I suppose I can separate it from CSM in this PR. If there are no objections, I'll put that in the checklist. |
Well, wait for input from other core devs first, as i am not an expert in CSM and GUI =) Also, i am unsure what the best approach is. I just had a thought that, until SSCSM happens, or if SSCSM never happens, this is actually still useable by having a CSMod built-in to the client? Is this the intention? Because i am unsure i have no objections or requests currently, i will hide my earlier long comment as it was partially invalid and got sidetracked into another issue. |
|
After more testing, I found that caching and calling functions separately really has no difference, neither in Lua nor LuaJIT. So, this is a lot nicer and more versatile as it's more interactive. For instance, testing the color of a pixel is possible in this, but not in the batch call.
Inventory interaction can come later. Checking whether the screen resized is already possible with
Yes, it's getting kind of big. I'm going to keep it in one PR for now so I can get everything done together. |
1dc8bff
to
bf1a8dd
Compare
Possible future security issue: If we implement the Esc menu in Lua, what's to prevent a mod (either CPCSM or SSCSM) from overriding it somehow? There are multiple avenues that this could happen through, like changing a global function or modifying a table entry containing event/drawing callbacks, and I don't think they can all be secured. So, either we implement it in C++ or we allow modification but implement a universal exit-to-menu keyboard shortcut, or perhaps just Alt-F4. Not especially fond of any of those, so ways to secure the |
I am considering removing the |
Really, you should consider any code that is in the same environment (or perhaps even state) as SSCSM as compromised and untrusted However, the API already allows exiting to the menu - there's a disconnect/shutdown method iirc |
I don't understand, is this a GUI specific PR or a generic input and rendering one? Tying this to GUI in any way might be missing the forest for the trees. |
The events API will not be tied to the GUI in any way, but it will be used by it. On the other hand, I'm still considering whether the drawing API should be tied to the GUI simply because of technical difficulties with sharing drawing between separate mods, specifically the last mod to register a drawing function will draw above everything else, whereas a custom drawn element can specify position in relation to other elements. I don't particularly want them tied together, but we'll see. Regardless of whether the drawing API is tied to the UI API, since the engine -> Lua code is globally exposed to Lua, there's nothing preventing CPCSMs or SSCSMs from overwriting the API completely to make its own drawing code. Not recommended for the majority of mods as it would break everything, but a specialized game that doesn't need a normal GUI could potentially do that (assuming we get SSCSMs, of course). But really, custom elements should be enough for most of these use-cases. |
d8fff01
to
5709910
Compare
4dde038
to
984af85
Compare
e8ea515
to
41a7c29
Compare
I've been experimenting with the idea of custom textures: https://www.youtube.com/watch?v=87iyesNooFI I think the idea has a lot of potential, especially if we had SSCSM. Really, it came about as a by-product of my working on a highly capable drawing API. Making the texture available in-game was a small extension. It has multiple bugs, such as the impossibility of texture modifiers (it's pure ITexture, no associated IImage to allow it to be dynamic) making Regardless, it's not PR ready by a long shot. Nor will I fix the bugs and make it a PR yet because GUIs are higher priority, although somewhat less exciting. Custom textures will make it into the GUI PR for e.g. CSS |
Closing; I don't expect to work directly on this branch anymore. The first split PR will hopefully be for events, but I have my doubts as I think it might be better to wait for SDL2 events before formalizing an API. So, part of drawing might come first. |
Cheers v-rob. Excellent work you've done, and I can see you're on the right conceptual path. The demo video has me particularly intrigued. I've been wanting to develop a security camera mod, so the ability to render to a node surface would prove invaluable. You've opened a world of new possibilities for modding in Minetest. |
I would say that the biggest problem is that of when to update textures. If there are 5 security cameras, that's 5 to update every frame. If there are 500, then that's a lot of textures per frame, all of them rendering a 3D scene. The question is how to keep this number down given that any number of security cameras can be placed? If one were to limit by proximity to the player, there could still be a few hundred within range of the player. Or, in another example, if signs use custom textures for text, should all the text for every sign be generated at the start of the game? That could be a lot of textures to generate on a big world. |
If we get SDL2 input, I worked out a possible API. It's a near exact reflection of SDL's relevant API (although somewhat Lua-ified) meaning it's very large but has all the input functionality anyone could ever wish for. API
|
Edit: This draft has since lost a sense of direction, and not everything pertains to the UI. See this post:
Accordingly, I'm removing the TODO list as the one I have is too long to keep updated here. To look at what's going on, look at
client_lua_api.txt
which has many things that have been added and others that will be added.Old direction (and will be the direction for some of the UI PRs this will be split up into):
This draft PR adds a CSM API for drawing to the screen and receiving events from input devices like keyboard and mouse. This is the essential prerequisite to a Lua-side GUI which is what I intend to replace formspecs with (see #6527 (comment) and onwards and https://forum.minetest.net/viewtopic.php?f=7&p=382529#p382529). The main intent is for GUI and HUD replacement, not CSM, and so it may be isolated in some other part of builtin so that it can be loaded regardless of whether CSM is enabled or not. According to my extensive testing (I didn't profile in milliseconds, but watched the framerate closely with a variety of tests), it is performant enough on both LuaJIT and plain Lua because it is practically no slower than equivalent formspecs.
To do
This PR is a playground Work in Progress. It will be split up into many PRs and closed after I've gotten enough finished.
Things I would like help with:
driver->draw2DLine
does not clip to a rect and I'm not certain if there's a good way to set the thickness. Same goes for drawing other shapes, like circles or irregular filled polygons.I don't want to explain every single design choice here because that would take forever, but don't hesitate to ask questions about anything.
I'm quite open to suggestions and criticism.