The future of Nuklear - brainstorming ideas

[dumblob]

@BieHDC wrote on 2021-12-20:

This should be the discussion place of what should be done and etc as the discussion started in this issue.

This is also my first time directly working with others together and figuring out this new interface.

[dumblob]

@dumblob wrote on 2021-12-20:

Will reply later (hopefully at the latest in a few days but ... anything can happen).

Btw. could this be made public? I think we shouldn't hide such discussions 😉.

[dumblob]

@BieHDC wrote on 2021-12-20:

I dont know, i am literally using this interface first time too. But i am busy the next days anyway, so...

[dumblob]

@RobLoach wrote on 2021-12-25:

One thing I'd love to see revamped is the gamepad/keyboard-only support. While you can accomplish it, it takes some work. Would be nice to have it just work.

[Hejsil]

With my null-termination-be-gone branch, it's pretty clear that I'm a big fan of having an API that is friendly and efficient to bindings from other languages. This would include:

• API that does not use null terminated strings
• libc is optional (nuklear already forfills this)
• Mark none nullable pointers as such with some annotation (there are compiler attributes for this)
  • This is useful for tools that generates bindings based on header files

One other thing I have as a wishlist item is a better allocator interface. The current one has the same interface as malloc/free, but this kind of interface requires the allocator keeps track of the allocation length, which can be extra unnecessary overhead for some allocation schemes.

[Hejsil]

Another one: A very low level API for nk_label and other text widgets that provides some sort of string builder, which can then be used to implement printf formatting and the like. Idk the exact API, but something like this is what I'm thinking.

struct nk_command_text_builder builder = nk_label_builder_begin(ctx);
nk_command_text_builder_append(&builder, "prefix - ");
nk_command_text_builder_append(&builder, some_string_variable);
nk_command_text_builder_append(&builder, " - suffix");
nk_label_builder_end(&builder);

The idea here being that the string will be built in place on the nk_command_text that will be emitted. This would allow other languages to wrap this API and use their standard formatting APIs.

[dumblob]

A very low level API for nk_label and other text widgets that provides some sort of string builder, which can then be used to implement printf formatting and the like...

Just saw this and find it interesting - especially if extended to allow for custom string "type" - e.g. https://github.com/josephg/librope (or its "successor" https://github.com/josephg/jumprope-rs ).

[iperov]

Hi guys.
Is it possible to create brand new UI concept, something between [Immediate mode UI] and [Full state UI].
Keep simplicity of UI declaration, but provide more flexibility, such as animations?

[dumblob]

@iperov yes, that's one of the few things I myself had in mind - but didn't have time to write a concise description of what I think. I'll get to it at some point.

Feel free to take a look at traffaillac/traffaillac.github.io#1 and Pauan/rust-signals#31 in the meantime.

If you have any ideas, speak out - the more feedback the better 😉.

[iperov]

I have an idea, but I don't know how relevant it is because I don't design GUI engines.
For example when we declare a sequence of layout and widgets
gui.vertical_layout_start()
gui.label()
gui.horizontal_layout_start()
gui.label()
gui.button()

the engine should automatically build internal layout/widget objects/structures?
and they won't reset on the next pass if the widget declarations match the internal representation?

also can user get access to such structures to define additional logic?
for example some pseudocode

button_obj = gui.button()
if button_obj.clicked():
   ... instant click process right here

button_obj.call_on_hover( on_button_hover ) - call function if mouse is hovering button

func on_button_hover (button_obj.):
   time = button_object.mouse_hovering_elapsed_time()
   button_obj.set_color(  RGB()*time  )
   if time >= 1.0:
     button_obj.set_icon ( icon_ready )
   if time < 1.0:
     button_obj.disable_click()

also 

gui.button.set_default_on_button_hover (on_button_hover) - to set handler for all buttons

so it is ok when some widget sequences are changed and do not match internal representation, they will just recreated, and all animation will reset.

also I would like to have a library for python as well. Because machine learning requires visualization, and QT is too redundant for that purpose.

[threedslider]

Hi,

Nuklear is very cool project, I will use it surely. My request is : will be it as 100% full customization with 100% visual for my C++ project ?

[dumblob]

My request is : will be it as 100% full customization with 100% visual for my C++ project ?

The current Nuklear is very customizable. If you can specify technical shortcomings you experience, then please describe them in detail here. Otherwise please use the Nuklear issue tracker to discuss everything else. Thanks!

[dumblob]

To keep in mind the other side of the spectrum: https://wiki.csswg.org/ideas/mistakes (it's missing some - e.g. ::first-line which is subject to font-size thus creating a loop...).

[threedslider]

The current Nuklear is very customizable. If you can specify technical shortcomings you experience, then please describe them in detail here. Otherwise please use the Nuklear issue tracker to discuss everything else. Thanks!

Ok cool ! Thank you too and keep it up ^^

[dumblob]

Some very interesting thoughts about "command" drawing (Nuklear currently also uses commands) and different kinds of such commands etc. from a highly experienced person: red/REP#103 (comment) .

[BieHDC]

Sorry for the long silence, but i finally got time again. Here are some of the keypoints i would like to have:

• Only 2 layouting systems, a simple one like nk_layout_dynamic and one for complex ones. Consider the default chat application: left sidebar spanning from top to bottom 64px wide, then you have a subspace with a top bar holding channel description. Then you have another subspace with input bar at the bottom and a send button, and the rest in the middle for text. Something like this should be "easily" doable, but right now we lack the "take the rest of the remaining space and treat it as new window" kind of command (which is kinda what my subspace pr is trying to do). This also means that instead of being top left to buttom down only layouting, you should be able to specify the anchor point (i think thats what its called). In this example for the text input and send button you would anchor to bottom left.
• I would also make much more use of caching. So you still stay immediate mode but first time code path it initialises memory from heap. This makes saving and accessing the state much easier and should remove some "hacky" things as annotated in current nuklear.
• Having simple modular widgets to build larger more complex widgets. For example being able to connect a bunch of text input widgets to build a bigger excel sheet. This was something that was personally causing me issues since some widgets have a mind of their own. Basically have a nice way to make complex widgets.

Thats what i remember right now, please give feedback.

[dumblob]

Images (incl. icons, buttons, etc.) shall be better dealt with - see e.g. Immediate-Mode-UI/Nuklear#444 .

[dumblob]

Transparency - masks etc. could be useful...

[dumblob]

Tiling images (as background etc.) could be desirable for some use cases (neo)Nuklear tries to cover. See Immediate-Mode-UI/Nuklear#444 .

[iperov]

@dumblob do you have any resources to read about internal UI implentations and architectures?

[dumblob]

@iperov not much as of now. But if you have anything potentially related, please speak up - if not for me, then it's guaranteed valuable for others.

[ryuukk]

My suggestions:

• animation/tweening/easing support
• better theming support (per control/id style)
• better font styling support (outline, shadow)

[BieHDC]

So over the last months i tried to get something to show multiple times, but eventually it becomes overwhelming. IRL work probably did not help ether dealing with that.
Did one of you get any examples done? Because to be honest, i feel like writing something new from scratch that aligns more with my mindset.

[dumblob]

I for myself did not code any examples yet - I am still in the brainstorming phase looking for proper abstractions. I am especially focused on dynamic behavior as over the years I started to realize that having solved the dynamic behavior (i.e. anything coming and changing over time), everything else is suddenly much easier to find abstractions for (layouting, styling, customizing, etc.).

But if you have anything (even non-functional), feel free to post a link to your repo so that we can take a look at your ideas to distil your thoughts 😉.

[iperov]

have you seen any new UI related projects?

[HackerDaGreat57]

Add an Apple macOS backend

[dumblob]

@iperov I am trying to incrementally keep up with UI & UX development all over the world. So yes and no to your question 😉. I have seen new projects/repos but unfortunately no new approaches.

[iperov]

@dumblob What new approaches are you suggesting?

[iperov]

Inside any Immediate-mode-GUI (IMGUI), a Retained-GUI (RGUI) is implemented in order to store the widget states.
But as we see in the example of DearIMGUI - the implementation is just awful.
Therefore why implement RGUI when you can use a ready-made RGUI framework, like QT/wx/Tk?

Technically IMGUI is just marking up RGUI widgets and getting the events of each widget directly into the markup function.
If the markup of a widget doesn't match a specific widget, it just gets recreated to the right widget and the event fetches will already be in the next markup.

In that case, instead of redrawing the GUI screen at 30-60FPS , the layout will only be rebuilt, and the RGUI framework itself won't cause costly redrawing every frame.

At this point you can create completely new lib.

But I've gone further and am building an experimental GUI in Python that doesn't need to invoke a rebuild of the markup 60 times per second.

Instead of getting the events in the markup function, I made "abstract backend controls" that implement primitive data management and work without a connected GUI to the backend.
Examples of such controls: Number (for SpinBox), Path (for file dialog), Signal (for button), SingleChoice (for ComboBox).
In the markup function we mark the widgets referring to these Controls.
We do not manipulate with widgets, but with controls. For example Signal.emit() is controlled by the Pushbutton widget. In the backend class I subscribe to the Signal.on_signal event and handle the pressing of the button, changing the state of the class and optionally call the GUI markup again, which can change the widget structure.

Example

2022-11-30_14-20-50.mp4

example of video player buttons markup:

...
with gui.hstack() as hst, gui.size(XGUI.ESize.S3), gui.color(XGUI.EColor.red):
    gui.spacer().expand(h=True)
    
    gui.pushbutton(self._con_seek_begin_signal, image=gui.get_ionicon_image(XGUI.EIonIcon.play_skip_back_circle_outline))
    gui.pushbutton(self._con_seek_backward_signal, image=gui.get_ionicon_image(XGUI.EIonIcon.play_back_circle_outline))
    
    play_btn = gui.pushbutton(self._con_play_signal, image=gui.get_ionicon_image(XGUI.EIonIcon.play_circle_outline), tooltip='Play')
    stop_btn = gui.pushbutton(self._con_stop_signal, image=gui.get_ionicon_image(XGUI.EIonIcon.pause_circle_outline), tooltip='Stop' )
    if fp.is_playing:
        play_btn.disable()
    else:
        stop_btn.disable()
        
    gui.pushbutton(self._con_seek_forward_signal, image=gui.get_ionicon_image(XGUI.EIonIcon.play_forward_circle_outline))
    gui.pushbutton(self._con_seek_end_signal, image=gui.get_ionicon_image(XGUI.EIonIcon.play_skip_forward_circle_outline))
        
    gui.spacer().expand(h=True)
    
with gui.hstack() as hst:
    gui.spacer()
    with gui.hstack() as hst:
        hst.compact()
        gui.spinbox(self._con_frame_index)
        gui.spinbox(self._con_frame_count, read_only=True)
    gui.spacer()
...

example of backend controls definition

self._con_seek_begin_signal.on_signal.subscribe(self._on_con_seek_begin_signal)

self._con_seek_backward_signal = boxcon.Signal()
self._con_seek_backward_signal.on_signal.subscribe(self._on_con_seek_backward_signal)

self._con_play_signal = boxcon.Signal()
self._con_play_signal.on_signal.subscribe(self._on_con_play_signal)

self._con_stop_signal = boxcon.Signal()
self._con_stop_signal.on_signal.subscribe(self._on_con_stop_signal)

self._con_seek_forward_signal = boxcon.Signal()
self._con_seek_forward_signal.on_signal.subscribe(self._on_con_seek_forward_signal)

self._con_seek_end_signal = boxcon.Signal()
self._con_seek_end_signal.on_signal.subscribe(self._on_con_seek_end_signal)

self._con_frame_index = boxcon.Number(range=(0,fp.frame_count-1), control_func=self._on_con_frame_index)
self._con_frame_index.set_number(fp.frame_idx, bypass_control=True)

self._con_frame_count = boxcon.Number(range=(fp.frame_count-1, fp.frame_count-1))
self._con_frame_count.set_number(fp.frame_count-1, bypass_control=True)

example of Play signal handler

def _on_con_play_signal(self):
    ... # some work
    self._con_error_stream.emit('Play start')
    self._update_gui() # - request to rebuild GUI

I think the DearIMGUI era is coming to an end, and we can create something better.

[dumblob]

@iperov yeah, good old signals (slots, events, ... however you call that). Unfortunately that concept is usually implemented too low level and thus highly error prone and becomes cumbersome (already in mid-complex UIs).

Therefore streams/live_variables/data_flows/... concept (one can view it as "high level" signals/slots/...) shall to be introduced as described e.g. in vlang/ui#7 (comment) (I briefly touched this in the discussion linked above in #1 (comment) ).

[iperov]

highly error prone

not in my lib.

[dumblob]

@iperov where is your lib hosted? Let us move the discussion about "error prone" etc. of signals/slots/events APIs to your repository.

[iperov]

But you started this discussion

[iperov]

I was a bit surprised that you were talking about errors at all, discussing abstract concepts.
It seems that programming in C seriously changes your perception and openness to new paradigms.
Good thing that after 15 years of C/C++ programming I don't go into that filth.

[dumblob]

How far shall neonuklear go with its API abstractions? There is https://github.com/linebender/glazier and similar projects which strive to make it easier to focus only on one thing in UI libs.

[ryuukk]

how is glazier relevant?

nuklear should remain non bloatware, keep it simple, fast, extensible and portable renderer agnostic IMGUI library, nothing else

[dumblob]

@ryuukk sure, Nuklear will stay as it is. But here we discuss neonuklear - i.e. some form of a successor (if any).

If we find out there is no need for more disruptive APIs/ideas than the current Nuklear offers, then neonuklear will not take off and thus will not exist at all 😉.

[Googler1]

Something else.

[dumblob]

@Googler1 could you elaborate?

Nuklear and Dear ImGui are most probably the only most portable (supporting most platforms) medium-capable (feature-wise) GUI libs in the world. Do you mean neonuklear shall continue in this league or do you mean something else?

[MichaelKv]

Brainstorming ideas? On a GUI library? In 21st century?
This is really sad if you really need them. But OK. Let it be so.

1. It shall not be just a library. It shall be at least a framework – and infrastructure with widgets and primitives for scientific, CAD (architecture, electronics, microelectronics, IT, etc.), hardware (medical, houses, IoT, etc.) and so on primitives developed by independent companies. And it must not be a monolithic one.
2. It shall not be C. C++ is the must. Starting a new big ambitious project in C is just unlikely. And it is enormously expensive. C++ means exceptions, multithreading, coroutines, etc. The framework might probably not use all these directly but shall at least be aware or them.
3. You will need to invest into its architecture and design a lot. I.e., it shall not have graphic primitives’ IDs generated from programmer’s provided strings and things like that as in Dear ImGui. All the things must be simple, intuitive and direct as visiting a crocodile farm.

[iperov]

@MichaelKv

On a GUI library? In 21st century?

Especially in the 21st century!

I am currently writing a layer on top of Qt, because there is NO gui library that meets my requirements for new software.

1. disagree. Most "infrastructure" implemented by other libraries such as numpy, opencv, etc. No need to reinvent the wheels.

2. disagree. C++ has gotten dirtier lately. C is the only parent that keeps the language clean. C programs can be ported to other languages in pure form in languages like V, Zig...

3. agree. DearImGui is piece of shit.

[iperov]

By the way, does anyone know if there are GUI projects where the rendering is done entirely on GLSL shaders?
There are SDF functions https://iquilezles.org/articles/distfunctions2d/ , with which you can draw anything in maximum quality, as well as animation.

[dumblob]

@iperov I am only aware of the following:

https://github.com/zauonlok/renderer
https://github.com/nokola/nonaquad
https://github.com/audulus/vger

They use shaders to varying degrees, but I am not sure if for everything.

[Googler1]

What's the idea?

[dumblob]

@Googler1 we are brainstorming them here 😉.

The reason is that we know Nuklear, we know how it is being used, we know its pros, but we are also very much aware of many (most?) of its cons. Here we try to gather potential "requirements" for a modern GUI library which would tackle most (all?) of the Nuklear flaws while not sacrificing many (any?) of Nuklear's major advantages.

Feel free to bring your own ideas here!

[iperov]

I think when you are brainstorming ideas, you are limited by the language you think in.
C has no support for asynchrony, tasks, generators, lambda functions.
Without that, you can't quickly program rich and flexible interfaces "in mind".

[dumblob]

Just to make it clear as sky - this brainstorming has no constraints. Thus there is no limit on which language(s)/notations/... to use for neonuklear's design.

If we will later find out it is difficult to implement the design for lack of features in certain languages or whatsoever, then we will have another separate discussion but for now let us not consider any such constraints.

[ryuukk]

I think when you are brainstorming ideas, you are limited by the language you think in. C has no support for asynchrony, tasks, generators, lambda functions. Without that, you can't quickly program rich and flexible interfaces "in mind".

that's bullshit and so wrong in many levels..
https://lvgl.io/ this is 100% C, https://github.com/lvgl/lvgl

If you guys go with a different language, then it's gonna be the death of Nuklear

The advantage of NK is: portability and efficiency, C is king at portability and is very fast

[iperov]

that's bullshit and so wrong in many levels..

it is obvious that you have not programmed in other languages.

death of Nuklear

what is dead cannot die

[ryuukk]

that's bullshit and so wrong in many levels..

it is obvious that you have not programmed in other languages.

I did, you just fail to understand the scope and purpose of Nuklear

[iperov]

What's your problem friend?

you can keep using nuklear.

Obviously the author has outgrown his project and is ready to move on.

This is a normal process of self-development.

[zecruel]

iperov commented 2 days ago

I think when you are brainstorming ideas, you are limited by the language you think in.
C has no support for asynchrony, tasks, generators, lambda functions.
Without that, you can't quickly program rich and flexible interfaces "in mind".

I disagree, C is definitely not limiting at all. It is the most flexible language I know. But I understand your perception: it takes a lot of work to implement the most sophisticated features.

ryuukk commented 2 days ago

If you guys go with a different language, then it's gonna be the death of Nuklear

The advantage of NK is: portability and efficiency, C is king at portability and is very fast

I disagree. Let me explain: I've been using the Lua language in my main project (with Nuklear) and I'm loving it.

• Lua is not a replacement for C, on the contrary, the two complement each other. Lua is the pure philosophy of C
• its performance is great, practically the same
• portability is also the same as C

I'm more and more convinced that Nuklear and Lua were made for each other, because their philosophies are the same.

What I propose by using the Lua platform is not merely a binding or a language migration, but a renewal of the Nuklear API.

For example, the following functionality can be incorporated into Nuklear:

• garbage collector, improving and simplifying memory management for using dynamic resources like loading images and fonts

• passing parameters by stack or table with named variables, which can join function calls almost the same as
  nk_button_text
  nk_button_label
  nk_button_color
  nk_button_symbol
  nk_button_image
  nk_button_symbol_label
  nk_button_symbol_text
  nk_button_image_label
  nk_button_image_text
  nk_button_text_styled
  nk_button_label_styled
  nk_button_symbol_styled
  nk_button_image_styled
  nk_button_symbol_text_styled
  nk_button_symbol_label_styled
  nk_button_image_label_styled
  nk_button_image_text_styled

• use of meta objects, which can do away with opening and closing calls like
  nk_combo_begin_xx and nk_combo_end (I get mad when I forget the closing)

• built-in parser for configuration and script files, which makes it easy to customize themes for example

This is all using C! And with the option to use Lua too, together and mixed. This paves the way for rapid interface development without the need for traditional code compilation.

Finally, the name "neonuklear" reminds me of a project that is following this path: it is NeoVim, a reimplementation of the classic Vim editor, incorporating Lua.

[iperov]

@zecruel

C is definitely not limiting at all. It is the most flexible language I know

It is flexible in terms of direct data manipulation on CPU.
I am talking about language. In C you cannot implement async coroutines / anonymous functions in one line. You cannot implement a method that will stop execution at some point, then transfer this execution to other thread and then continue.
And I'm not even talking about such trivial things, such as all memory has to be freed manually, leaving a lot of bugs for the user. C is dead like dinosaurs.

[iperov]

@zecruel

I guess these terms are all new to you, because you are as old as the C language.
You're like pagans who don't accept the new. (by the way I am 37)
You don't understand that modern software needs to be developed quickly without making bugs. The high performance parts are derived in separate blocks and are not a problem, but even for them you don't need to use C :)) such as the numba package for python.

[dumblob]

I feel the discussion is getting heated.

I think all the raised points here in our discussion are relevant.

Please just do not point fingers at each other, but rather at the specific features/ideas/proposals. We need less "human context" (like I guess these terms are all new to you) and more technical observations.

Btw. @zecruel you are one of the top users of Nuklear when measured by app complexity and size. Thank you for taking some of your precious time and chiming in - I welcome your input!

[iperov]

what do you think about the concept of a GUI engine where it would be easy for the user to create new widgets of any complexity

[TahirSpark]

Hi all, please dont kill the project, help could be on its way if theories comes together.

I also encourage people here to have more out of the box thinking from most classical design and architecture you are used to. In one case, Ive come up with a new potential graphics standard called UIP (user inter phase) and UXP (user experiential design) which can fit more better in immediate design. If its not self-explanatory, I apologize because the spec is in early stages and requires me to delve deep in many topics which is overwhelming.

If you can, it would be nice to shift focus on more to Vulkan. As for C's limits, its limited by the classical paradigm we are in and it is the fastest classical language. The last great thing we've had in software was C99 and the worst thing to happen to software was JS.

If there's a chat channel for this project, please put it here.