Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Formspec replacement #6527

Open
rubenwardy opened this issue Oct 13, 2017 · 79 comments

Comments

@rubenwardy
Copy link
Member

commented Oct 13, 2017

MuSCo (must have, should have, could have)

Actual GUI libraries / logic

Must have

  • Good layouting
    • Consistent co-ordinates and scale.
    • Adapting containers for varying window size (table, linear, stack)
  • OOP design in C++ to allow extending, and decouple elements.

Should have

  • Should use an existing solution - avoid reinventing the wheel. It's fine to define our own Lua API for it, however.
  • Theming
    • Ability to change borders/backgrounds/etc
    • CSS like syntax?

Server API

Must have

  • Sane format
    • OOP style, which is then converted to JSON?

Should have

  • Auto-validation
    • reject invalid submissions

Could have

  • Server-side formspec to new API converter

CSM API

Could have

  • Define custom elements, with draw(). Draw is only called when required, which may be every draw step.

Other notes

@sapier
As discussion on minetest-dev showed up we first need to define what a possible formspec replacement has to do, a suggestion:

Completely specified
Modifyable (change existing formspecs)
Current formspecs have to be convertable to it
Errors have to be detectable (no typos, no logical errors)
Container support (tabs/ribbons/whatever you wanna add here)
Existing format is preferred
Support for optional parameters (not interpreted by GUI, but explicitly defined)
Pixel based AND relative positioning of elements
API versioning 
How UI elements are drawn
How UI is defined
How user input is linked to UI definition

See sfan5's post for an example API

I'm willing to work on this if we can have a consensus on how it should work.

Related: #1399, #5810 )

@rubenwardy

This comment has been minimized.

Copy link
Member Author

commented Oct 13, 2017

Using HTML/CSS, QT, or GTK would probably be too much for a game.

I suggest something like tgui or SFGUI as a backing library. We could reflect their API in Lua, and serialize and send to the client using JSON.

@ThomasMonroe314

This comment has been minimized.

Copy link
Contributor

commented Oct 13, 2017

i agree with that, those aren't exactly lightweight GUI's
however wxwidgets might be a possibility as it is automatically cross-platform.
but idk how much it is supported

@nerzhul

This comment has been minimized.

Copy link
Member

commented Oct 13, 2017

i have a partially working XML format in a branch: https://gitlab.com/project-demeter/core/tree/form_xml
The only missing patch is catching events from XML to core, other parts are managed and just need more and more components, it's XML with HTML-like attributes)
It's using libxml2 which is portable and very performant, used by many softwares

@ThomasMonroe314

This comment has been minimized.

Copy link
Contributor

commented Oct 13, 2017

what about this: http://www.clutter-project.org/
it seems to be very lightweight and runs well on Android

@weqqr

This comment has been minimized.

Copy link
Contributor

commented Oct 13, 2017

nuklear looks good, supports theming and it's very lightweight compared to Qt/GTK.

@ThomasMonroe314

This comment has been minimized.

Copy link
Contributor

commented Oct 13, 2017

@weqqr that looks like exactly what openarena uses, and its GUIs look pretty nice
but does it support android?

@weqqr

This comment has been minimized.

Copy link
Contributor

commented Oct 13, 2017

@ThomasMonroe314 nuklear itself only receives user input and generates a list of shapes to draw, so in theory it can run on everything from a supercomputer to an electric kettle. You need to write a frontend (like this) to make it work with Irrlicht though.

@sfan5

This comment has been minimized.

Copy link
Member

commented Oct 13, 2017

Wanting a CSS themeable, extendable (by CSM) GUI using a completely new GUI toolkit is aiming way too high IMO.
It pretty much guarantees that it won't ever be fully implemented and we'll be stuck on formspecs for a longer time.

Also having something like minetest.register_window removes quite a lot of dynamicness from the gui system.

(The rest of this post is just throwing my ideas in here.)

Instead I would fix "just" these issues:

  • coordinate system
  • proper layouting, meaning: containers with children
  • use nested tables to represent these (not exposed to Lua, should have object-oriented API)
  • proper event system: wide variety of events for each element, all optional

This retains the following from formspecs:

  • Toolkit-independent
  • dynamic ("made up on the spot", no pre-registered layouts)
  • events are produced in certain conditions, these can be individually listened for
  • uses Irrlicht GUI (probably)

There is likely no existing solution for this, but that's fine.
Since this is quite hard to put into words, some example code:

local window_name = minetest.get_current_modname() .. ":respawn"
local pad = 50

local function show_respawn_dialog(playername)
	local root = gui.Window:new(window_name, 100 + 2*pad, 200 + 2*pad)
	root:listenForEvent("close")
	
	local stack = gui.Stack:new()
	stack.padding = {pad, pad, pad, pad}
	root:add(stack)

	local button1 = gui.Button:new("Respawn", 100, 100)
	button1.id = "b_respawn" -- must be unique
	button1:listenForEvent("click")
	stack:add(button1)

	if creative.is_enabled_for(playername) then
		local button2 = gui.Button:new("Respawn at same position", 100, 100)
		button2.id = "b_respawn2"
		button2:listenForEvent("click")
		stack:add(button2)
	end
	
	minetest.show_gui_dialog(playername, root)
end

local function respawn_normally(player, state)
	assert(state["b_respawn"].pressed == true) -- just an example for state
end

minetest.register_gui_event(window_name, "b_respawn/click", respawn_normally)
minetest.register_gui_event(window_name, "close", respawn_normally)

minetest.register_gui_event(window_name, "b_respawn2/click", function(player, state)
	-- do something else
	return true
end)

internally/when sent via network it would look like this (no modder would ever see this though):

{
	"type": "window",
	"id": "testmod:respawn",
	"events": {"exit": true},
	"children": [
		{"type": "stack", "children": [
			{"type": "button", "id": "b_respawn", "events": {"click": true}},
			{"type": "button", "id": "b_respawn2", "events": {"click": true}}
		]}
	]
}

To achieve backward compatibility you would then need to:

  • Convert the formspec string to tables (duh)
  • Translate the stupid coordinate system to the new one during conversion
  • Give each element a default set of listened events (e.g. buttons always respond to clicks)
  • Convert the new events to the old system and call minetest.register_on_player_receive_fields callbacks
@ThomasMonroe314

This comment has been minimized.

Copy link
Contributor

commented Oct 13, 2017

nuklear has lua bindings, a thing that i haven't seen anywhere else, and something that is straightforward, simple to use and should be simple to implement

@rubenwardy

This comment has been minimized.

Copy link
Member Author

commented Oct 13, 2017

nuklear has lua bindings, a thing that i haven't seen anywhere else, and something that is straightforward, simple to use and should be simple to implement

Doesn't help when we need SSM to be able to make windows/dialogs


@sfan5, all of that sounds good to me. I actually prefer it to mine, so updated first post

It's fine to leave stuff out for an initial implementation, but it needs to be thought about to allow future additions. Yours does however, you could add theming without breaking

@ThomasMonroe314

This comment has been minimized.

Copy link
Contributor

commented Oct 13, 2017

pardon my ignorance, but what is SSM?

@rubenwardy

This comment has been minimized.

Copy link
Member Author

commented Oct 13, 2017

pardon my ignorance, but what is SSM?

Sorry, server-side mods or server-side modding.

@ThomasMonroe314

This comment has been minimized.

Copy link
Contributor

commented Oct 13, 2017

ah ok thanks, but why is that a problem? cant it be used for that?

@rubenwardy

This comment has been minimized.

Copy link
Member Author

commented Oct 13, 2017

ah ok thanks, but why is that a problem? cant it be used for that?

Their API will likely be directly manipulating elements, where as we need to send commands over the network to the client.


I suggest keeping both the new API and formspecs in parallel whilst developing, then making a server-side formspec to new GUI converter when stable enough

@ThomasMonroe314

This comment has been minimized.

Copy link
Contributor

commented Oct 13, 2017

hmm true, but as weqqr pointed out, we would need to write a frontend, so couldnt we add some that functionality to the front-end?

@weqqr

This comment has been minimized.

Copy link
Contributor

commented Oct 13, 2017

I really like sfan5's idea, but not the API, so here's my concept.

local function say_hello(window_state, player)
	print("Hello, " .. window_state.name.text)
end

local function show_window(player_name)
	local window = minetest.window("mymod:window", function(w) w
		:size(200, 200)
		:field("name", function(f) f
			:default_text("someone")
			:position(50, 50)
		end)
		:container("my_container", function(c) c
			:position(60, 60)
			:padding(10, 10, 10, 10)
			:button("hello", function(b) b
				:text("Say hello!")
				:position(50, 50)
				:dimensions(100, 100)
				-- Callbacks can be set in-place
				:on_press(say_hello)
			end)
		end)
	end)

	minetest.show_window(player_name, window)
end

-- Complex callbacks can be handled outside of the window definition
minetest.register_on_gui_event("mymod:window/my_container/hello", "press", function(window_state, player)
	print("Your name is " .. player:get_player_name())
end)

minetest.register_on_gui_event("mymod:window", "close", function(window_state, player)
	print("Window closed")
end)
@paramat

This comment has been minimized.

Copy link
Member

commented Oct 13, 2017

I feel 'themeable' is low priority and should not count against a good, simple, practical solution.
MTs neutral grey boxes are visually fine and no problem.

@rubenwardy

This comment has been minimized.

Copy link
Member Author

commented Oct 13, 2017

MTs neutral grey boxes are visually fine and no problem.

MT's current GUI is ugly. The new GUI can be tweaked to look better without breaking things, and margins will be better as you would be able to specify exact margins. Theming is relatively low priority, and should not be in a first implementation - but it should be considered to make sure any first implementation could support it in the future. I think any good consistent and DRY solution would be able to, anyway

@paramat

This comment has been minimized.

Copy link
Member

commented Oct 13, 2017

Hmm my statement there was too strong, even i like to have colourable boxes instead of only grey / black. Being able to design the background graphic is a good feature to have.

@numberZero

This comment has been minimized.

Copy link
Contributor

commented Oct 13, 2017

@weqqr That jQuery-like syntax is awful IMO.
@sfan5 What’s the problem of needing to register windows before showing them? That would be handy IMO, as long as the registration may be done at any time and not only on server load: you could setup the whole huge multi-tab window layout, all the callbacks, etc., and only set some content right before showing it.
Also, that way nodes could store window name in the meta instead of its full specification, thus easing updates.

@raymoo

This comment has been minimized.

Copy link
Contributor

commented Oct 14, 2017

My thoughts (Some of these have been mentioned already in the thread):

  • There should only be a CSM GUI API (maybe with a compatibility layer for old-style formspecs). This would need to wait for servers being able to send client mods.
  • The API should have some way of drawing every frame so that mods can animate GUI components, making it possible for mods to create new interactive GUI elements, like selection wheels, without hardcoding engine code to handle them.
  • Layout should be automatic, i.e. you don't need to manually specify coordinates for everything if you don't want to.
  • The same API could be used for client-mod provided HUD elements.
  • (Optional) I like the idea of imgui's API. I've never actually used it though so it might not be as good as I imagine.
@tobyplowy

This comment has been minimized.

Copy link
Contributor

commented Oct 14, 2017

MTs neutral grey boxes are visually fine and no problem.

Now your really pissing me off

Hmm my statement there was too strong,

Ooook then nevermind

But all jokes aside those "neutral grey boxes" look awful and they have to go they look unprofessional, they are ugly, they make minetest look like test software (the word test in minetest doesn't help with that) and they are NOT CUSTOMIZABLE WITH TEXTUREPACKS!!!!!!

@ghost

This comment has been minimized.

Copy link

commented Oct 20, 2017

Using HTML/CSS, QT, or GTK would probably be too much for a game.

I'm in with QT and GTK but imagine how awesome it would be having HTML-based dialogs and menus and stuff. Of course not full HTML5 and CSS3, but a base set of HTML syntax (img, input, section, div, and some more) plus some Minetest-specific additions (for the inventories for example) would be friggin awesome!

@numberZero

This comment has been minimized.

Copy link
Contributor

commented Oct 20, 2017

@dsohler IMO some XML-based language would be way better: it is easier to parse and more flexible. E.g. mods could be able to define their own elements (using XML namespaces... oh yeah, it uses the same namespace:element naming convention as MT does...)

@rubenwardy

This comment has been minimized.

Copy link
Member Author

commented Feb 6, 2018

@numberZero

Dear ImGui is designed to enable fast iteration and empower programmers to create content creation tools and visualization/ debug tools (as opposed to UI for the average end-user)

from their read me

@Ferk

This comment has been minimized.

Copy link
Contributor

commented Feb 9, 2018

The container tag is essentially implemented as a preprocessor. When the formspec is parsed, it adds the offset to all elements between the tags. The container element doesn't exist after then. It wouldn't be appropriate for a multipage widget.

Even if it's implemented as a preprocessor, it is still possible to expand on that to implement a paging system.

Example:

  • Before pre-processing:
container[pos=1,1;id=pets]
  page[]
     image[pos=0,0;size=1,1;texture=cats.png]
  page_end[]
  page[]
     image[pos=0,0;size=1,1;texture=dogs1.png]
     image[pos=0,1;size=2,2;texture=dogs2.png]
  page_end[]
  page[id=whatever]
     image[pos=0,0;size=1,1;texture=nyan1.png]
  page_end[]
container_end[]
pager_button[pos=0,0;size=1,1,label=cats;activate=pets,1]
pager_button[pos=1,0;size=1,1,label=dogs;activate=pets,2]
pager_button[pos=2,0;size=1,1,label=huh?;activate=pets,whatever]

After pre-processing:

image[pos=1,1;size=1,1;texture=cats.png;page=pets,1]
image[pos=1,1;size=1,1;texture=dogs1.png;page=pets,2]
image[pos=1,2;size=2,2;texture=dogs2.png;page=pets,2]
image[pos=1,1;size=1,1;texture=nyan1.png;page=pets,whatever]
pager_button[pos=0,0;size=1,1,label=cats;activate=pets,1]
pager_button[pos=1,0;size=1,1,label=dogs;activate=pets,2]
pager_button[pos=2,0;size=1,1,label=huh?;activate=pets,whatever]

Then, client side, hide any formspec element that has a "page" field different than the first found in each section of the same "page" set of values.

When the user clicks one of the "pager_button", check (client-side) the value of the "activate" field and make it be the current page, unhiding/showing all items marked with that page and hiding the rest. Just as a client-side visual effect, without really sending any packets to the server.

You can even chain this to have several layers of pagination.

I'm not saying this is the best way to implement it, just giving an example that shows it's not syntax what's blocking this.
If reimplementing the entire formspec system, adding new dependencies, stitching them to the engine and trying to maintain a compatibility layer is faster, easier and more maintainable than improving the current formspec system, then awesome, do that instead.

@v-rob

This comment has been minimized.

Copy link
Contributor

commented Feb 13, 2018

Formspecs need theming. With the current level of theming, I can't make formspecs look very good. Personally, I think that image elements are the best answer to this. They're likely easier than colorizing or making custom gradients for the buttons. Here's what I think they should include:

Note: I'm using images of Windows XP stuff for my examples because I have them on hand for no good reason.

  1. Normal element image. For buttons, this would be the normal image of the button.
    Ex: image -- Normal button

  2. Image when mouse is hovering over the element.
    Ex: image -- Highlighted button

  3. Image when the button is being pressed.
    Ex: image -- Pressed button

  4. If the element is something like a field, then it needs a background image. The image would overlay the field.
    Ex: image -- Field background

These image elements should apply to all elements possible: Dropdowns, checkboxes, textlists, etc.

Unfortunately, this can lead itself to some elements having many images. Take, for instance, scrollbars. Scrollbars have a button for up, a button for down, a button for the actual scrollbar, and the background behind the bar. The buttons all need hovering and pressed images besides the normal images. That ends up with 10 images for just a scrollbar. I don't see a way to fix this problem, though.
Ex: image -- Top button, bottom button, bar, and background.

Also, assuming an element has an image, then the element should be transparent as to not show any grey gradient should the image be partly transparent.

EDIT: Well, a possible solution to the problem stated above is tilesheets, like the ones used for nodes with animated textures. First is the normal, below it is the hover, and on the bottom is the pressed.

@Wuzzy2

This comment has been minimized.

Copy link
Contributor

commented Feb 14, 2018

I'm not sure if it is a good idea to require an image for every widget.

At least make it optional? Quick and easy colorization of widgets is also important. With baked-in images, that's not so simple.

@v-rob

This comment has been minimized.

Copy link
Contributor

commented Feb 15, 2018

The easy way to make an image button act like a normal button is leave the image part of it blank or set it to blank.png. I think making duplicates would be unneccesary.

@Calinou

This comment has been minimized.

Copy link
Member

commented Feb 15, 2018

It's better to go with vector-based stuff rather than bitmaps, since those support scaling and hiDPI out of the box.

@nerzhul nerzhul added this to Feature requests in Minetest 5.0.0 blockers Mar 24, 2018
@v-rob

This comment has been minimized.

Copy link
Contributor

commented May 9, 2018

I recently was experimenting with other programming languages than Lua and some of the GUI's that come with them, and it really showed me to the problems with the current formspec layout. Of the things I tried, this is what I think would be best:

This is how I think the layout should be for registering a formspec:

-- This registers the initial window

minetest.register_window("mod:random_formspec", {
     size = {x = 5, y = 4},
     location = {x = -1, y = -2},
     anchor = {x = 0.5, y = 0.5}
})

-- Add a label

minetest.register_label("mod:label", {
     window = "mod:random_formspec",
     pos = {x = 0.5, y = 1.5},
     image = "text_bg.png",
     text = "This is an example formspec."
})

-- Add a button

minetest.register_button("mod:ok_button", {
     window = "mod:random_formspec",
     pos = {x = 2, y = 3},
     image = "button.png",
     hover_image = "button_hover.png",
     pressed_image = "button_pressed.png",
     text = "Ok",
     func = function(player)
          minetest.close_formspec(player:get_player_name, "mod:random_formspec")
     end
})

minetest.register_window(formspec_name, formspec_definition):
Makes a formspec window and defines the size, location, and anchor.

minetest.register_*(element_name, ) Where * is any formspec element:
Defines the specified element and adds it to the specified window.

To open a formspec, use minetest.show_formspec in the node/entity on_rightclick function.

Also, there should be bindings, like on_enter, on_exit(both for when the mouse hover/stops hovering over an element) on_rightclick, on_rightclick_release, on_leftclick, on_leftclick_release, etc. For these bindings, there need to be a way to tell if another binding is happening at the same time.

Anyway, these are my thoughts on the format of formspecs.

@TMcSquared

This comment has been minimized.

Copy link

commented May 10, 2018

I was poking around and found a neat GUI library that is:

  • themeable
  • works with Irrlicht
  • should be fairly easy to implement into the modding API
  • can be scripted using Lua
  • cross platform
@nerzhul

This comment has been minimized.

Copy link
Member

commented May 10, 2018

i like cegui and i like the products listed on the site. It seems to be mature, but it's a new dep.

@rubenwardy

This comment has been minimized.

Copy link
Member Author

commented May 10, 2018

A new dep is perfectly fine for something as important as a GUI system

@sfan5

This comment has been minimized.

Copy link
Member

commented May 10, 2018

While "can be scripted using Lua" is nice, it's not what we need.
We need a GUI library that can work "over" the network, since formspecs are shown on the client but the code that handles formspecs runs on the server.

Just a reminder, since this is not the first time someone has suggested a GUI library because it has Lua bindings.

@nerzhul

This comment has been minimized.

Copy link
Member

commented May 10, 2018

@sfan5 i don't think GUI library over network exists somewhere. What i note here it's cegui permits to drop many shitty irrlicht old and crappy code on formspecs, and we can create a new formspec protocol for this new UI systems in parallel and drop formspecs at a point. (or keep the compat but the formspec format is... anoying)

@sfan5

This comment has been minimized.

Copy link
Member

commented May 10, 2018

A new GUI library can be used when writing a replacement, yes.
It's just doesn't solve the problem this issue is about: a replacement for Formspecs

(That doesn't mean it doesn't belong here, GUI libraries are still useful to know.)

@rubenwardy

This comment has been minimized.

Copy link
Member Author

commented May 10, 2018

The only useful part of the Lua binding is if it has the ability to make custom elements using it already. Otherwise it's not a benefit of the library for us

@nerzhul

This comment has been minimized.

Copy link
Member

commented May 10, 2018

formspecs are bound to irrlicht, we can make formspecs cegui aware, and it can be nice :)

@rubenwardy

This comment has been minimized.

Copy link
Member Author

commented May 10, 2018

No, formspecs need to be killed completely.

@TMcSquared

This comment has been minimized.

Copy link

commented May 10, 2018

The only useful part of the Lua binding is if it has the ability to make custom elements using it already. Otherwise it's not a benefit of the library for us

I believe the Lua might be only for animating the GUI, not sure on that though. But for the record, that is very cool nonetheless.

@rubenwardy rubenwardy removed this from Feature requests in Minetest 5.0.0 blockers May 10, 2018
@numberZero

This comment has been minimized.

Copy link
Contributor

commented May 11, 2018

@appgurueu

This comment has been minimized.

Copy link

commented Feb 9, 2019

I would rather implement GUI in Lua. Also I think we should try to keep backwards compat.
Formspecs should probably work pretty much like HTML forms - you can do some preeval with lua, client side. In the end, the submission goes to the server.

For everything dynamic, requests have to be made explicitly. In general, MT needs such a request system. Probably worth another issue.

@yquemener

This comment has been minimized.

Copy link

commented Aug 10, 2019

Do you think it would be possible to add IME (Input Method Editor) support in the features a GUI replacement should have? If you ever want to have support for Asian language inputs, you will need that at one point. Now that's a string of yarn that I understand most people don't want to pull, but that's why having IME support in the GUI framework (and making sure unicode works there but I think most recent UI framework wikk have good support) allows for some peace of mind.

@rubenwardy

This comment has been minimized.

Copy link
Member Author

commented Aug 10, 2019

That feature is irrelevant to the GUI system used. It could already be added to our current GUI system, problem is that it needs to be added to irrlicht (apparently?)

@yquemener

This comment has been minimized.

Copy link

commented Aug 11, 2019

It can be added to any UI system (that supports unicode correctly) but it is a pain to do so. Some framework (I know Qt has) have done all the hardwork of taking into account the shortcuts, additional keys, unusual keyboard layouts and presenting a working input. That's definitely a "nice to have" feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.