Create modular actions

[dodgyrabbit]

Problem statements

• The current user interface for button actions does not allow for easily adding new actions.
• The same area is used to define various items, some are quite uncommon (like changing brightness).
• The UI code becomes more and more complex whenever anyone adds something new.
• Ideally it should be easy for the community to create new functionality and not have to worry too much about anything else.

Proposed solution

Main configuration

• All the standard actions are removed from the UI and replaced with a list of actions for the selected button.
• The text and image could stay, as this is common among all actions. However, it may make sense to move this into the Action definition itself. This will allow for custom Actions to provide their own default image, text etc more easily. Note that multiple actions should be possible - which need to be considered (i.e. the first action provides the image?)
  

Action configuration

• A three step Wizard is provided. Two of the steps would be the same for all Actions and standard with streamdeck-ui
• The third step is the UI that will define the details of the action.
• Users will be able to create their own actions (for example, an OBS plugin, sound plugins etc).
• Simple example/template actions created to make it easy to get started.
  

Downsides

• The current implementation makes it quite quick to get started. I.e. select a button, give it a heading and action.
• The proposed solution is way more flexible but will require more interactions (add action, go through 3 steps) to achieve some basic results.
• A solution to this may be to provide some "Quick actions". I.e. Add Volume-Up. This automatically adds an icon, required action bound to the correct trigger.
• Some convenience features like "copy/paste" of button actions may make sense. I.e. you spent time defining one - then you can duplicate and just tweak the part you want.

Considerations

• The application must dynamically load actions
• How does it 'find' installed actions? Do we traverse everything on the python path, so you can install it via pip? i.e. pip install streamdeck-ui-obs?
• Do we create a directory somewhere and specifically look for it there?
• An abstract base class is defined and any class that implements it will be loaded.

Alternatives

• Support for native Stream Deck plugins would be possible (some). In particular, ones written in JavaScript can be hosted in a process and communicated to via their JSON api.
• This would be somewhat tricky as streamdeck_ui implementation would have to match their API exactly.
• While doable, the time commitment to do this may simply be outside the scope (unless someone wants to take it on).

[attilakv]

I am not sure about the wizard. It adds a lot of additional steps, and it's benefits would only be felt when there are a large number of trigger and action types.

Here is a version without the wizard:

You also proposed, that the text and image could move into the action definition. I disagree.
The text and image is property of the button display itself, and shouldn't be initialized by an action.
Although actions should be able to change them, that would be really cool.

[dodgyrabbit]

I am not sure about the wizard. It adds a lot of additional steps

Yea, shared concern as I noted. I put something together and played with it and it's not really that cumbersome in practice.

when there are a large number of trigger and action types.

Definitely the case. There would be a few trigger but hopefully a very large number of action types.

I considered the layout but the screen becomes too wide. Remember with the streamdeck xl has 8 buttons across and will take up space on the left. Also, more complex actions needs a decent amount of space for their config.

I think the wizard can be reduced to 2 steps for now (only key down anyways) and the Add Action can have a dropdown that selects additional triggers once they exist. In my mind even triggers could become complex (like key combinations for example).

Agreed that having custom actions manipulate the button image and other aspects is the goal. At a very basic level level, even setting the image itself could be an action and the trigger is "startup". Maybe overkill but basically I'm just thinking what would be the most flexible/hackable approach.

Appreciate the input.

[attilakv]

basically I'm just thinking what would be the most flexible/hackable approach.

that is my goal as well. I see more potential in this version, than the official Elgato driver. But it's a long way from it's full potential.

the screen becomes too wide. [...] the streamdeck xl has 8 buttons across and will take up space on the left

I agree, but it would make the actions settings glanceable, and that is a serious advantage.
My layout was more of an example on how to do it without a wizard. It is not set in stone yet, so here is a better layout:

Note, that although I reduced button sizes significantly in this diagram, a scrollbar may be a better solution to make room for the extra panel, if the panels will be resizable.

[dodgyrabbit]

OK I've taken another stab at a wizard-less layout.

• I've experimented with the stream deck UI buttons a bit and we can save some space by reducing the gaps between the button.
  

Although the space underneath could be used, I find it quite awkward. I think the official version uses it for the label/image config but it doesn't flow very well visually IMHO. I haven't looked at the official version in a bit (on purpose).

Missing from your layout is available Action list. Although there are only 5 or 6 today, if we get this "right" and the community (probably just me :) ) builds more, there should in theory be a very large list and it will be important to properly categorize, with an icon and description. A simple dropdown won't do it. In my design below, I also leave out the trigger type (for now). Since there is only one at the moment, it's implied. I think one could select that at the point where you click Add action.

There are some things we could move around to make room and still have the "at a glance" view of everything.
I'll experiment with an actual layout today, something like this:

Ultimately this is a compromise. Limited to to spend on this hobby project.

[dodgyrabbit]

Looking at this it may make more sense that the configured actions are on the left, and the action selection on the right as this is likely what you'll be looking at most - and being right underneath the label makes it more cohesive.

[attilakv]

the official version uses it for the label/image config but it doesn't flow very well visually IMHO

Agreed, it works in their case, because they have 1 action / button, and their actions are less atomic in general

Missing from your layout is available Action list

I planned to keep your window from the wizard as a form of popup selection, but your new solution is 100 times better 😄

I also leave out the trigger type

In my design, the trigger type is implied at the list of actions. You reorder the actions sequentially, and you can move the action into a different trigger type
This limits the number of triggers to a handful, but personally I can't think of more than onPress onHold and onRelease.
(perhaps multiple breakpoints for onHold?)

IMO this design is excellent, but it should also be manually resizable. This is because the importance of each section changes depending on what you're doing:

• setting up a new button? action-select & action-list > action-configure
• adding a new action to a preexisting button? action-list > action-select > action-configure
• configuring a complex action? action-configure >>>> action-select & action-action-list

Note, that these are not mutually exclusive, and it's really up to the user on what they want to see best.

[Data-007]

**Originaly posted earlyier in thread

• How does it 'find' installed actions? Do we traverse everything on the python path, so you can install it via pip? i.e. pip install streamdeck-ui-obs?**
  

Could It be made as part of the Export/Import feature.

• start initially with a blank canvas(Events wise)
• Let the user configure the actions
• Then if the user uninstalls run make the export feature

Just a sugestion I dont know if it would work

[sftim]

How I'd like to see it: the tool emits D-Bus events for all buttons (I'm thinking page, x, y as data), and also listens to D-Bus for events like “switch to page” or “set Stream Deck brightness”.

The options for a button action then become:

• run script
• call system D-Bus action (eg: lock screen; play/pause)
• switch page
• emit D-Bus value

Maybe also:

• press key

(yes: I've imagined a design that means that if you want to change the brightness with a button, you need a helper script or helper D-Bus listener).

Using D-Bus would let you split out the UI / manager part of the tool from the bit that talks USB and understands D-Bus. I could launch the main tool from systemd and only start the desktop UI if I want to reconfigure it.

This approach would perhaps let you switch out the key-press and keyboard typing tool(s) for your own tooling, maybe. Handy for people who use Wayland, if so.

I know this is a lot of change. What I want to encourage is small steps that make getting there more feasible, even if the final thing is a long way off yet.

If you had a magic wand to summon coders who can help, how about this: different integration plugins (Twitch? OBS?) could register via D-BUS and make themselves available for configuration.

[sftim]

A simpler way for plugins to register would be that they each drop some config inside ~/.config/streamdeck-ui/plugin.d

[Danie10]

Some excellent ideas coming through here, and I also support with the comment it should to be phased ideally, maybe with each new idea spun out into its own discussion thread where it makes logical sense. Also, maybe a todo list of ideas to be voted on, so they can be prioritised.

I'd still like to see the relative brightness per button still being in (not sure it is working properly right now) as it is useful to adjust some buttons brighter or dimmer (not essential though).

I do like the layout with all buttons shown, and being able to click on each button, to see what it does, and then drag it elsewhere (in future hopefully copy/move to a different page). But on a dropdown list of available actions, we must also not forget about multi-actions. We could also take inspiration I suppose from the official app as many users also work on Linux as well as Windows, for a familiar use but a bit different.

[coolapso]

hi, this project seems to be stale and a new fork as been made with the intention of replacing this one.

if this still an issue please reopen it at: https://github.com/streamdeck-linux-gui/streamdeck-linux-gui