Scene Organization Tools

[dodgepong]

Scene Organization Tools

Feature Summary

Allow users to organize scenes in such a way that they can easily find the scenes that they need to switch to or edit during both preproduction and live production.

Background and Motivation

Most OBS users never need more than a small handful of scenes with simple, static layouts. However, it's not uncommon for a user's scene count to grow rather quickly as more demands are placed on their production capabilities and requirements.

In many cases, this functionality can be solved through the use of scene collections such that separate shows can use separate scene collections, but sometimes a single production can still be unmanagably complicated, necessitating organizational tools within a single scene collection. Furthermore, it's often impractical to change scene collections in the middle of a broadcast due to flickering and unprofessional visuals generated by the live loading of soruces, as well as the lack of an ability to select the scene that is activated upon switching to a new scene collection.

Oftentimes, the largest culprit adding complexity to a production is the OBS feature that allows users to add scenes as a source inside of other scenes. This leads to a usage pattern whereby many scenes are simply "component" subscenes and used to construct other scenes, but are never actually needed for the purposes of switching during a live production.

Users would like a way to organize their scene list to make it easier to find the right scene at the right time whether they are editing their scene collection before a broadcast, or trying to switch to the correct scene during a live broadcast.

Request For Proposal

Submit a proposal explaining how you will implement this feature such that it meets the following requirements:

• New functionality is added to the Scenes list that allow users to organize their list of scenes. Functionality includes:
  • The ability to group or associate related scenes together
  • The ability to conditionally hide or show groups of scenes or scenes that are otherwise associated with one another
• The new functionality follows or closely resembles existing established usability conventions such that users are more likely understand intuitively how these organizational tools can and should be used.
• The new functionality is easy to employ or delete, and scenes behave in a predictable manner when the functionality is deleted.
• The new function behaves reasonably in context of how the multiview is currently populated.
• The file format should be as backward compatible as possible, allowing people to safely downgrade to older versions. Note that this does not necessarily mean you can't use a new format, just that if someone downgrades to an older version, all scenes will be present and the program will still function fine. You could implement a new json object/array alongside the old json/array object array and update both, or figure out a way to use the original json object/array in a way that still makes it backward compatible somehow or another. (-Jim)
• To ensure backward API compatibility, the APIs used for enumerating scenes should be unchanged, and act as though the grouping just isn't there. If APIs are to be added, it should probably be in the frontend API, and probably should be something like obs_frontend_get_scenes_v2() if anything. Something super simple if possible. (-Jim)
• The feature should work on Windows, macOS, and Linux.*

* Multiple contributors may assist with implementation on different platforms and may split the bounty according to the level of contribution they made to the project.

Please read Tips for Writing a Good Proposal for guidance on how to author your proposal.

The OBS development team will consider all submitted proposals and select one submission to be implemented by the developer who submitted it. That developer will be eligible to collect the bounty upon completion of the project according to the criteria listed below.

Bounty

The bounty for implementing this feature is $2500.

You can learn more about the OBS Project Bounty Program here.

If you wish to contribute funds toward increasing the amount on this bounty, contribute to the OBS Project Bounty Fund and select the "Scene Organization Tools" option (or just click here).

Completion Criteria

You may collect the bounty when the following are true:

• Your proposal addresses all items listed in the "Request For Proposal" section.
• You have submitted one more more pull requests that fully implement the feature as you proposed.
• Your pull request(s) are reviewed, and you have responded to review comments within a reasonable time frame.
• Your pull request(s) are merged into the master branch of the appropriate obsproject repositories.

Additional Information

Related ideas from the OBS Ideas Page:

• https://ideas.obsproject.com/posts/739/group-scenes
• https://ideas.obsproject.com/posts/233/add-color-labels-to-scenes-as-well
• https://ideas.obsproject.com/posts/385/create-folders-for-scenes-too

Related posts on the OBS Forums:

• https://obsproject.com/forum/threads/categories-for-scenes.111584/
• https://obsproject.com/forum/threads/ability-to-create-sub-folders-among-sources.139506/
• https://obsproject.com/forum/threads/folders-in-scenes.82593/

[Programatic]

I think there are several great features to implement, so here is my updated proposal.

Proposal of Implementation

This proposal is overwhelmingly going to be UI changes. While it certainly could integrated further with libobs (similar to how sources are), I don't find that there is any particular reason to be. It is for ease of use, clarity, and better management, all of which are UI/UX focused ideals.

Frontend (SceneTree)

Currently, SceneTree is simply a subclass from QListWidget. This is fine on its own, but it should be expanded (very similar to SourceTree), such that it can support groups. It doesn't need to be particularly fancy, but it should be able to support 2 main things: Groups and Filtering

SceneTree "API"

Being able to interact with the SceneTree will be extremely helpful for plugins. This ranges from being able to directly modify / read the scenetree, to other components being devolped being integrated with the scenetree (for instance a searchbar). Given that this would most make sense organizationally in obs-frontends-api, the core datastructure be done in C. This just makes it more portable and slot into the existing codebase. Then, we can use an QAbstractListModel to actually be the backbone of the list and handle the state, and simply interact between the 2 using signals. In total, for this to be complete, it should at least satisfy the following capabilities:

• Create Group
• Delete Group
• Add Scene to Group
• Remove Scene from Group
• Expand Group
• Contract Group

Translating this to actual code, it can work something like this:

class SceneTreeModel : public QAbstractListModel {

    // Standard Class Constructs to facilitate model

    void CreateGroup(std::string &name);
    void DeleteGroup(OBSSceneGroup group);
    
    void AddSceneToGroup(OBSSceneGroup group, OBSSource source);
    void RemoveSceneFromGroup(OBSSceneGroup group, OBSSource source);
    
    void ExpandGroup(OBSSceneGroup group);
    void ContractGroup(OBSSceneGroup group);

    OBSSceneGroup findGroupByName(std::string &name); 
    void FilterScenes(std::string &search);

}

Which could in turn be utilized in the api as such:

// in obs-fontend-api.h

EXPORT void obs_frontend_scenelist_add(OBSSceneGroup group, obs_source_t *source);
EXPORT void obs_frontend_scenelist_remove(OBSSceneGroup group, obs_source_t *source);
EXPORT void obs_frontend_scenelist_create(const char *);
EXPORT void obs_frontend_scenelist_delete(OBSSceneGroup group);
EXPORT void obs_frontend_scenelist_filter(const char *);
EXPORT void obs_frontend_scenelist_expand(OBSSceneGroup group);
EXPORT void obs_frontend_scenelist_contract(OBSSceneGroup group);
EXPORT obs_data_array_t * obs_frontend_scenelist_group_get_scenes(OBSSceneGroup group);
EXPORT OBSSceneGroup obs_frontend_scenelist_find_group_by_name(const char *);

OBSSceneGroup simply contains a vector that contains all the sources it holds. Then, when this is modified in any way, this will get reflected the frontend, handled by the SceneTreeModel.

I am not an expert on using the Lists, so this is subject to change given how flexible the lists actually are, and whether this adds too much technical debt, but this should allow for easy control over the list for both the frontend and plugins. This is also kind of modeled after the source-tree which already has some of this functionality built in, so I also imagine if they have simliar APIs, it will make changes easier in the future and reduce programmer overhead. However, due to the core differences in how it is stored and represented, this will naturally lead to differences.

Potential Issues

There is definitely a concern about adding technical debt. With the above api version, it would add 2 new classes, as well as modifying the current scene tree implementation. This could potentialy be a bit more complex than needed, which is why I suggest a potential change in that case be switching to querying the abstractmodel directly through function pointers (which is also definitely not ideal). This would reduce the need for OBSSceneGroup as a kind of intermediary, but I believe the current way is more ergonomic to use. As is, it allows better access to modifying groups. Consider the following example:

OBSSceneGroup games = obs_frontend_scenelist_find_group_by_name("TestGroup");
OBSSceneGroup editing = obs_frontend_scenelist_create("Editing");
if (games_only) {
    obs_frontend_scenelist_add(games, scene1);
    // ...
    obs_frontend_scenelist_add(games, scene4);
}

if (maybe_csgo)
    obs_frontend_scenelist_add(games, csgoscene);
    
if (random_item_is_game)
    obs_frontend_scenelist_add(games, randomscene);
else
    obs_frontend_scenelist_add(editing, randomscene);

obs_frontend_scenelist_expand(games);
obs_frontend_scenelist_contract(editing);

I personally believe that it is just more intuitive and fits my mental model to have OBSSceneGroup. But either will probably work fine, one can just use const char *, but having OBSSceneGroup ensures better consistency.

Storage

I propose that it should be stored in its own file. This would also be stored in json and follow the following format:

{
"groups": [],
"tree-order": [],
}

This is very simple, easy to integrate, and easy to edit. It could certainly be modified in the future should any considerations arise, but as this is, it should get the job done very simply. This doesn't necessarily need to be its own file, and could very easily be moved to the main file. However, I think there is so much going on in that file, splitting them up into more distinct seperate chunks makes it more manageable.

Filtering Considerations

Filters usually modify the view in place, allowing for individuals to clearly see all scenes that satisfy the search, as well as where it is located (for instance, expanding groups where items are). OBS should function no differently. I believe that vast majority of users will greatly appreciate the minor details that make usage so much more seamless. This in turn would allow for improved productivity and just overall enjoyment of the program. I share the concern of the UI being too cluttered especially for new users, so it overall needs to be non-invasive and just work as someone intuitevely would think.

SceneTree Visuals

Since the scenetree has been a core component of OBS, it should stay relatively unchanged. Under this proposal, it would remain nearly identical. The only change should be the addition of a searchbar.

---

ORIGINAL THOUGHTS

I think this is a great thing to implement, and I have a couple ideas for my proposal:

• First, grouping scenes would certainly help reduce clutter, and allow individuals to have better control over what is presenting. I believe this should be implemented similarly to how source groups present. However, I think it would be much preferable to have the groups on the frontend only, and instead have some sort of internal DS that keeps tracks of the groups such that plugins and w/e comes down the line can utilize it (through helper methods that I would implement).

• Second, I think it would also be helpful, with the groups, to have a search bar that allows individuals to filter the current view. This would go in hand with the helper methods described before, so the integration I believe would work fairly well. This would allow much easier searching for specific, known scenes. I imagine that it would hide all other scenes not described (potentially even with regex for more technical users), and if they are nested in groups, expand to clearly show where they are nested in. This should be consistent with many other applications that have similar behavior. It could look something similar as follows.
  

• Third, on the part about having clear, consistent, and expected behavior, I believe it is relatively straightforward to follow how other popular applications use it. If it uses an internal DS to keep track of the frontend organization, then it would involve revamping interactions with the scene list, but it could easily be disabled in settings, as if it was never there. It would be entirely optional and current users could keep on going as if it was never made.

[jp9000]

The problem I see with your proposal is potential backward compatibility issues. People might want to downgrade, and thus using a new scene format or using separate files might cause a problem. You're allowed to do that of course, but just make sure that whatever you do still modifies the old json object/array so that it's backward compatible. Might cause a bit of redundancy but it's really important to me. I've added it as a requirement in the RFP.

[jp9000]

Also, I'm not sure about how I feel about an API in libobs. I know that plugins might want it, but I think we could probably just add a simple new frontend API for it instead. I'm just not sure. I'd prefer some sort of simpler approach somehow or another. I don't think scene grouping should necessarily be a libobs thing.

The normal functions that return the scene list should be unaltered; they should return all scenes with the same order that they have in the scenes tree widget; just minus the groups. If we want to allow plugins to see groups, then we could add a v2 frontend API function for enumerating or returning scenes. But it should be really simple, and we really shouldn't need to modify libobs in my humble opinion.

[Programatic]

The problem I see with your proposal is potential backward compatibility issues

I could definitely be incorrect about this, but the plan in my had would prevent that. The actual scenes will be loaded and stored as they currently are. The only extra information that would be stored and split off are the information specific to groups. Thus, when loaded onto an earlier version that does not support scene grouping, it just loads the scenes as it normally would. The main issue I see, is that ordering could be messed up, but could be fixed by mirroring that information to how it is usually stored.

I'm not sure about how I feel about an API in libobs

I'll defer whether or not that is useful to someone like @exeldro, but I would imagine it could allow some cool plugins. If you dont want an API for it, would creating a callback for it that can be called suffice?

[jp9000]

I could definitely be incorrect about this, but the plan in my had would prevent that. The actual scenes will be loaded and stored as they currently are. The only extra information that would be stored and split off are the information specific to groups. Thus, when loaded onto an earlier version that does not support scene grouping, it just loads the scenes as it normally would. The main issue I see, is that ordering could be messed up, but could be fixed by mirroring that information to how it is usually stored.

It's possible I misunderstood, but yea, I just want to make sure that anyone going back to an old version won't have broken or incorrect data. I know it sucks to deal with but it's important for users in case they want to downgrade for whatever reason.

I'll defer whether or not that is useful to someone like @exeldro, but I would imagine it could allow some cool plugins. If you dont want an API for it, would creating a callback for it that can be called suffice?

I don't mind if you create a new API or something for it, I guess the issue is I just want it to be as simple and compact as possible. I kind of wish the grouping was just metadata stored for each scene or something. Something super simple like that, but I'm not sure how well something like that would work out. What do you think?

Another thing is that the existing frontend API for scenes should also treat the scenes as though they weren't grouped so it's backward compatible that way as well. If the grouping was just metadata then you wouldn't need to add new API for that, though I'm not really sure how well that would go.

I'm not sure. Just in general, grouping feels like the biggest problem. What do you think though?

[Programatic]

That's completely fair. Why not have it as metadata at first and should the need arise it can be extended later? That way it can be designed with the intent to later extend it, but the basic design can be perfected to be minimal and compact.

[Azeirah]

Proposal

UI

Most OBS users don't need powerful scene organization tools, so for those who do, I think we can match most demanding users' needs with only two additions.

First, I saw the addition for a filter field proposed by @Programatic. This is a common and recognizable way to search for one or a couple of specific items in a list. I agree that it's a good method for scene organization that would be a good fit for OBS.

Secondly, I'm thinking about a visual addition to scene organization. Many tools (including file browsers, music players, and... Pokémon PC storage!) let you tag or color items to make them stand out. I propose to add a right click option for scenes to label them with a color of choice.

These are my mockups for the filter as well as the labels. Apologies for the mockup, I'm not an artist, I'm an engineer :). Of course all this would be implemented with standard built-in Qt components, not with paint

Quick find scene

Colored labeling

The add color QAction would feature a QColorDialog popup.

Code

The filter

In my opinion, the role the filter field should have is to quickly find a single specific scene you're looking for and nothing else. Having looked at the code, I would prefer not to mess with the QListItems populating the SceneTree, as this would be my first contribution to OBS as a whole, I fear it would be a risk for unintended interactions with other parts of the code accessing the SceneTree items. Instead, I propose a little popup with results where you can select the scene you were looking for. This would be a simple floating QListView whose result will be selected in the SceneTree after you found what you were looking for.

For all this, we shouldn't need any custom data structures or changes to configuration whatsoever as it can be implemented entirely in the frontend. I'll add a QLineEdit to underneath the SceneTree, initializing and configuring the component and connecting the necessary slots/signals in around ui_OBSBasic.h:741.

The filter field should behave as you would expect from other well-coded programs, IE, pressing escape should clear the field as well as the result popup, up/down keys for selecting your desired result, enter as well as lclick on a result item should select it in the SceneTree. Additionally, the popup should never clip off-screen. (IE, if your obs window is near the bottom of your monitor, the pop-up should appear at the top-right instead of the default bottom to prevent clipping off-screen).
Lastly, since this is a new feature, it should feature a simple descriptive pop-up when hovering over the feature to explain what it does. (in html that'd be the title attribute, not sure what it is in Qt)

Because I'm intending to implement this as a search popup, it will function equally for both the list view as well as the graph view.

The labels

The colors of the labels will be saved in your selected configuration file. In its current state, the configuration file has no single organized construct for all scene related configuration, everything is added to the global scope. I would prefer to reorganize everything under a single scope like so, but that would require a lot of unrelated refactoring.

scene: {
	"current_program_scene": "",
	"current_scene": "",
	"scene_order": ["..."],
	"scene_labels": [
		{"name": "", "color": "#aabbcc"},
		"..."]
}

So instead, I'll add it to the global scope of the configuration file instead, with the same inner structure as shown above.

As the complexity of scene configuration increases I would personally suggest refactoring the configuration though, maybe I'll take some time to do that after implementing this feature itself. The refactoring is out of scope for this feature if you ask me.

As for the implementation itself, I'll add a QMenu with QActions to QMenu *sceneCollectionMenu around ui_OBSBasic_h:1130. It will list a couple of preset colors as well as a custom color.. option which will open a color picker. Any custom colors will be saved as well. Instead of altering the config file with yet another list of colors, it would be better to populate the selectable colors with the colors found in scene_labels: ... to prevent overcrowding that context menu with random colors. Although that being said, I'll implement that first and see if it feels right, otherwise I'll just save the custom colors to the configuration file.

Coloring the items themselves will be implemented separately for both the graph and the list item views, as those are separated in the code right now too.

Where will the code go?

As for organization, I've looked through the OBS ui codebase and found that it's coded mostly in two files ui_OBSBasic.h and window-basic-main.cpp and .hpp. I will add any UI initialization to whatever header file makes most sense, and add any new UI references to their public slots.

As for the implementation of the labels and the filter, I'll add two new c++ files that implement the methods necessary to implement my proposal. scene_organization_quick_filter.cpp and scene_organization_abeling.cpp.

Cross-platform considerations

As far as I can think of, since this should be implementable with exclusively OBS primitives as well as Qt, I don't foresee any need for custom code per platform.

As testing is concerned, I have access to a windows machine, a Linux machine, as well as an M1 mac. Once I've arrived at the cross-platform testing phase at the end, I can hopefully find someone to compile it for MacOS (M1 can't compile OBS yet :<) so I can perform the necessary testing.

[Programatic]

@Programatic Respectfully, being that you are someone who is making your own proposal, it's probably not appropriate for you to be making commentary on other people's proposals. Ultimately we'll make that sort of determination for ourselves. I am grateful that you want to help but it's probably not a good thing for candidates to be interacting too much with one another when they are in effect competing with one another for approval. Hopefully you understand the potential conflict of interest there.

That is completely understandable and I didn't even think of that. My apologies to the author, I have removed the comment.

[Azeirah]

@jp9000 @dodgepong Hi, I was wondering if there are any updates on the bounty program? I'm very interested in investing more time into getting to know the OBS codebase as well as the bounties, but I'm hesitant to invest more time as long as I don't have more info to go on.

Specifically, I'd really like to know what kind of time-scale you're thinking of for considering and accepting a proposal for a bounty, as well as if you have any comments on my proposal.

I very much understand that OBS is an open-source project and that the bounty program is new for all of you, so it's understandable that not everything is going smoothly from day one, but I'm eager to get started and am hoping to get some more info and certainty so I can decide on where to focus my time.

Thanks a lot in advance!

[jp9000]

This reply might end up being be a bit long so I'm going to try to be as concise as I can.

I noticed that your proposal calls for simplification: implement a search filter and add color coding. I personally liked that idea and I brought it up with the team, but there was a bit of resistance; the consensus seems to be that we really need collapsible groups. A lot of users have a very large amount of scenes, sometimes hundreds, and often a lot of those scenes will only be used for specific purposes or under specific circumstances, so they don't always need to be visible in the list. Even though a search filter and color coding would be very helpful, they still wouldn't be sufficient for some users. So it'd be nice to have collapsible groups, especially with easy reordering. I don't know if we can avoid modifying the SceneTree class. It's probably going to need to be reworked to some extent.

At the moment I'm not sure if anything would work better for organizing hundreds of scenes. It's possible that there might be a better idea, but I'm not sure what that would be. I understand that for a first-time contributor, reworking something like SceneTree is probably going to be pretty intimidating. I honestly don't blame you there, I'd feel the same way, especially considering that some of the code is... unpleasant at times. There are probably a lot of things need to be reworked sadly.

Regarding the RFP process, I admit that we're still kind of getting a feel for it. Annoyingly, our release deployment process for 27.1 started happening around the same exact time that we started the bounty program, so it's been a bit of a juggling act, and time management is a skill that I still need to work on apparently. I would at least like to state for the record that we consider RFPs very high priority, so I was going to respond sooner than later anyway. But I'd say that we're still learning to some extent, so my apologies in advance.

For the question about time scales, if you mean how long it takes for us to review and accept proposals, I think that giving each RFP a few weeks is probably a safe bet, although I think that if we get a particularly fitting proposal we might accept some proposals right away. I don't see that happening very often though, probably depends on the RFP; some future RFPs are probably going to be easy. Generally I think giving it a few weeks is probably a safe bet. Sometimes we might accept proposals immediately, sometimes it might take a bit longer.

I also think that it makes sense to give an RFP more time if there's a lot of back and forth. For example, I'll probably give this RFP more time just simply because of the amount of interaction going on.

Just as a note, sometimes I might also get a bit busy or overwhelmed and might not respond very fast (as you probably already noticed). It annoys everyone unfortunately. It's easy for me to get overwhelmed, and I'm trying to delegate more of my work. So I want to apologize for that in advance as well.

Feel free to let me know your thoughts. The good thing about proposals is that they're relatively cheap in terms of time investment. But they are a bit of a time investment, so we definitely don't want people to feel like they're wasting their time or anything.

[Azeirah]

@jp9000 Thank you very much for your elaborate reply. I personally believe that an RFP process like this relies on back and forth discussion from proposer to team so I'm glad to hear your thoughts on the ideas. I wasn't aware that there were people out there with hundreds of scenes, a grouping/folder kind of feature would definitely make a lot of sense there.

I'd also like to mention that I completely understand the struggle related to managing and creating a new system, I've been working on my own projects and products for years now and it's very different experience compared to working under a boss telling you what's important and what isn't.

I'm going to take the feedback and revise my proposal and look into the codebase a bit deeper to see what would make sense.

My intuition right now goes out to dropping the colouring feature and adding folders. I think the search still makes a lot of sense for anyone with more than 10 scenes, whether it is 20 or 300.

And as an FYI, while reworking something like a SceneTree might be overwhelming for a first-time contributor, this is in the end a paid job, and not a simple hobby project. If both the OBS team as well as myself decide that that's what's necessary to get this to work in its entirety then I as an implementor should and will take that seriously. I don't see bounty implementors (for lack of an alternative word) as open-source contributors, but as a professional who just need to get the job done. I can use the money, I have the time, and I have relevant experience. Which is why I'm taking this seriously.

[jp9000]

I appreciate that. I just need to give the disclaimer that although I'm giving feedback and trying to be encouraging towards everyone, I can't necessarily guarantee that I might pick a proposal. I also think I should be completely transparent and say that Programatic seems to have a pretty good plan so far, and we've had pretty positive results with him in the past. He wrote our Undo/Redo system which was a pretty complicated thing to write, so I admit you've got pretty tough competition on this bounty just simply because we have experience with him. Just wanted to make sure that you're aware of that and that I'm being as transparent as possible.

Worst case scenario, if we don't accept it, don't let that discourage you from continuing to submit proposals for bounties. Also, although Programatic might be favored, I wouldn't let that discourage you from continuing to alter your proposal; nothing is off the table yet. I just felt it was an important thing to be transparent about if anything.

I definitely want to make sure that no one's discouraged when there are vying proposals for bounties. I think what we'll tend to see is it'll probably end up being first-come-first-serve in most cases.

[Fenrirthviti]

Just to add a comment here from my perspective as someone who receives a lot of the user feedback, one thing that is asked for a lot is the ability to sort scenes into collapsible folders. One could argue that the ability to search might supersede this, but from an organizational standpoint, having a folder that items can be placed in is one of the most common requests we receive. This would have no functional aspects outside sorting inside the Scenes list (i.e. it would not be the same as Source Grouping), and would allow the folder to be opened/closed accordion-style.

[jp9000]

I have added extra requirements to this RFC:

• The file format should be as backward compatible as possible, allowing people to safely downgrade to older versions. Note that this does not necessarily mean you can't use a new format, just that if someone downgrades to an older version, all scenes will be present and the program will still function fine. You could implement a new json object/array alongside the old json/array object array and update both, or figure out a way to use the original json object/array in a way that still makes it backward compatible somehow or another. (-Jim)
• To ensure backward API compatibility, the APIs used for enumerating scenes should be unchanged, and act as though the grouping just isn't there. If APIs are to be added, it should probably be in the frontend API, and probably should be something like obs_frontend_get_scenes_v2() if anything. Something super simple if possible. (-Jim)

Due to my additional requirements, and given the fact that they might be a significant extra annoyance, I've also increased the bounty to $2500.

[pedjas]

It would be helpful to have an option to set snapshot of a scene to the Scenes list.

Grid mode, as it is now is not very helpful, as it shows only scene names, and those names are usually longer than space available in grid buttons.

With scene snapshots shown along the names, grid mode would became extremely useful.