Add support for embedding the game window in the editor

[reduz]

Describe the project you are working on

Godot

Describe the problem or limitation you are having in your project

Several users who want to switch to Godot (or switched to Godot already) miss the functionality provided by engines such as Unity for "exploring" the running game.

While Godot can show you all the running game contents in the remote scene tree and its sort of possible to override the remote camera, it still leaves a lot to be desired.

Engines like Unity or Unreal let you run the game within the editor window. They achieve this by running inside the same editor process, which lets you explore the game while you run it.

This approach has the clear advantage of allow you to explore the running game more intuitively, but it also has several disadvantages:

• Non-destructive editing of the running game is impossible or in a very limited fashion.
• The game has to be saved, the cleared before playing.
• The game has to be reloaded after playing.
• The editor stability can be easily compromised if the running game crashes.

None of these are problems with Godot however. Godot runs the game in a separate process and lets you explore its contents via IPC, but as the game window is not embedded, exploring the content becomes more unintuitive and annoying.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

The idea is to have an embedded game mode in Godot. This proposal describes specifically how this could be done in a way that is feasible and should allow most of the expected behavior.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

The main goal is to have a new mode in the top bar called "Game", where the game can run embedded in the editor. It would look something like this:

(Note: Pretend that the icon in game is a joystick icon and I am a very skilled graphic designer).

The idea is that the game runs in this window, at either native resolution or letting you stretch the window to fit it (zooming options can be discussed further, but you get the idea).

What functions are supported and what are those icons?

• Pause: Ability to pause the running game
• Next Frame: Ability to move to the next frame while paused.

• ---

• 2D/3D selection mode: This lets you select which kind of nodes and view will be interacted with while exploring the scene.

• ---

• Game input: When this icon is toggled and the game is focused (click on it to focus), all input is captured and goes to the game until user clicks elsewhere. (Of course if the game captures the mouse, we will need some way to un-capture it with some special keybinding).
• Selection Mode: This mode no longer sends input to the game and it lets you select 2D/3D nodes (depending on the selection mode described before) by clicking on them. They will appear on the remote scene tree and inspector so you can see their content.
• List Selection Mode: Same as the mode above, but it presents a list of nodes you can choose to select instead of selecting the topmost one.

• ---

• Camera override: This lets you override the game camera and pan/zoom (if 2D is selected) or WASD/similar (if 3D is selected).

Implementation notes:

The main idea of this feature is that Godot will remain running the game as a sub process, with the main difference being that the game window is embedded into the editor. This can be done with relevant Vulkan extensions (and worse case if a platform does not support this, it just does not let you embed the game window, but most should support it).

FAQ:

Q: Are there not other 10 proposals about this?
A: Yes, but this explains how it would be done in a way that is feasible technically.

Q: Will the 2D and 3D editors show the game objects moving around?
A: No, this is not possible and probably never will. Exploring will have to happen from the game window itself. We can add tools to make this as comfortable as possible.

Q: Will I still be able to edit the scene while the game runs in a non-destructive way?
A: Yes, unlike other engines, you will not lose any change made to the scenes you are editing and the changes will also replicate to the running game.

Q: Will the editor stability be compromised by the game?
A: No, it should not be.

Q: Is this feature optional?
A: This feature is not only optional but probably only present for platforms and rendering drivers where it is possible to do this kind of embedding. I suppose users will be able to configure whether they want to use it by default or not in the editor settings.

If this enhancement will not be used often, can it be worked around with a few lines of script?

This is core

Is there a reason why this should be core and not an add-on in the asset library?

N/A

[aaronfranke]

This would significantly increase the usability of the editor camera override. Currently you need to have both the editor and the game visible on screen, then enable the camera override, and you need to have the editor focused to move the camera in the game. This proposal significantly improves the situation so that you only need the editor visible.

[yosoyfreeman]

Could be an option to not save the changes made while running the game in this mode? I personally miss it to quickly test things, spawn objects or change values being sure it will revert when i stop it.

[aaronfranke]

@yosoyfreeman You would not be spawning objects into the running game directly. You can only spawn objects in the saved scenes, and in many cases Godot can sync that to the running game. This proposal does not change this behavior, it only improves the usability by allowing the game window to embed into the editor.

[bitbrain]

I really like this proposal! How would this behave if I were to set a breakpoint somewhere? I suspect since it still runs in a sub-process, the game would be set to paused if it reached a breakpoint?

[Bastienre4]

As a professional game developer currently working with Unity and wanting to switch to Godot, this is one of the feature I miss the most.

If I have an issue about the UI, I need to be able to inspect (both in terms of data AND visually, in the scene/game) the node/gizmos to understand what's wrong. Debugging without this is a pain 😅.

[BoneDaddy64]

Coming from Unity, all I really want is to have a scene view (probably should be embedded) that has a separate camera from the game view but renders the same scene state, as in same objects (transforms, models, animation state etc. Unity allows for different shading which is good) that I can fly around with, and to be able to pick (as in click in that view on the rendered object's model) and have the Inspector show that node (and be able to change it, like when in Remote in the Scene list currently). Whether the game view is embedded or not doesn't matter that much to me.

Basically I just want to be able to quickly fly around, select objects, have those objects appear in the Inspector, be able to real-time modify the objects (which Godot has the functionality for now). That's what's really lacking right now for me in Godot, makes it really cumbersome to "touch" my game as it's running

[aaronfranke]

@BoneDaddy64 Depending on the details of how the implementation of copying the texture to the editor works, it might be possible to have the editor camera override add a new viewport + camera and read the texture from there, instead of overriding the existing camera's transform, so the game view could keep the same camera. That would be a separate proposal made after this one, but it would help if this feature was designed with that use case in mind.

[allenwp]

As a professional game developer currently working with Unity and wanting to switch to Godot, this is one of the feature I miss the most.

If I have an issue about the UI, I need to be able to inspect (both in terms of data AND visually, in the scene/game) the node/gizmos to understand what's wrong. Debugging without this is a pain 😅.

This comment about gizmos brings up the question of whether gizmos could be drawn over top of this new Game view mode to visually show what is selected. Further to this, there are questions about whether these gizmos could modify the remote node properties, such as using the transform gizmos to translate, rotate, or scale remote nodes.

Personally, I suspect that this is something that should be kept in mind when implementing this new feature so that full gizmo support could be added at a later time. I think it would be great to get something like this implemented without gizmo overlays sooner, rather than delaying the feature to have full-featured gizmo overlays.

I also think that just being able to see gizmos in the scene would be immensely helpful to visually distinguish what node is selected and shown in the inspector panel. Being able to modify the node's properties using gizmos is a less valuable feature because this can be done through the inspector panel.

[sosasees]

i need to test window transformations often — like moving and resizing — especially when they are not triggered in the OS but in the game.

so the current way of running the game in its own window saves me much time.

so if people want the game window embedded, this should be an option and not the only way.

[allenwp]

I previously started a discussion regarding "Spacial Editor - Focus Selection should focus on remote nodes". I think this new proposal should address this feature: When in the new Game view mode in "Selection Mode", pressing the F key to Focus Selection should focus the camera override on the currently selected remote node.

This is especially helpful when you have nodes that are far away from your game or override camera, such as an off-screen world object. With this Focus Selection feature implemented for the Game view mode, it would allow the user to select the node in the remote scene tree, Focus Selection, and have the override camera focus on this node.

I also noticed that the original post mockup screenshot does not show a remote scene tree view at all, and it seems that the proposal suggests a "List Selection Mode" instead. How does this List Selection Mode compare to the existing remote scene tree? I'm curious to know more about this, especially in the context of how it will interact with node selection and the Focus Selection action.

[m4gr3d]

This would be a welcome improvement for the Android editor, given the limitations on tablets and phones to have multiple windows open side by side.

[robbertzzz]

I'd personally really like if the UI would allow for switching between different performance views. I sometimes want to see wireframes or overdraw in my game, but can currently only do it by adding additional code to the game. Some UI for it like Unreal has would be amazing!

[NathanLovato]

Overall, it sounds great, and it's been requested time and time again, as you know. This is especially big for professionals coming from Unity and Unreal.

UX-wise, do you plan to unify the play scene functionality (the transport controls in the top-right) with this? Here are some thoughts to complement the proposal regarding making this view work with the existing transport control, UX-wise.

Unifying the transport controls with this new Game main screen:

• By default, playing the scene or game with F5 or F6 shows the running game in this new embedded Game main screen.
• Pressing F8 stops the game view and takes you back to the last main screen you were working with (2D or 3D mainly).
• An icon similar to the one to pop out the script editor is present on the Game main screen and lets you float the Game main screen view to another screen.
• When the game window is floating, the toolbar stays attached to it (I understand this may not be feasible, just an idea to have the pause and stop and other controls right by the game)
• An icon on the Game view allows you to toggle it full-screen (to have this game view full-screen on another display, e.g. for two-people test sessions where one person plays the game with a gamepad and a developer or level design makes changes with hot reload).

These are just quick ideas to spark discussion, to see if that's possible or the intention at all.

The idea would be to unify the current play scene functionality with this new view, so that users have one consistent and powerful experience when running their scenes or games in the editor.

If we can do anything to help, please let me know. We now have an experienced UX/UI designer specialist in the team, we can help with mockups, icons, and focused user testing moving forward if the team needs extra hands.

[reduz]

@NathanLovato yeah it does make sense to do this if you are using the embedded mode.

[yosoyfreeman]

* By default, playing the scene or game with F5 or F6 shows the running game in this new embedded Game main screen.

* Pressing F8 stops the game view and takes you back to the last main screen you were working with (2D or 3D mainly).

* An icon similar to the one to pop out the script editor is present on the Game main screen and lets you float the Game main screen view to another screen.

* When the game window is floating, the toolbar stays attached to it (I understand this may not be feasible, just an idea to have the pause and stop and other controls right by the game)

* An icon on the Game view allows you to toggle it full-screen (to have this game view full-screen on another display, e.g. for two-people test sessions where one person plays the game with a gamepad and a developer or level design makes changes with hot reload).

These are just quick ideas to spark discussion, to see if that's possible or the intention at all.

The idea would be to unify the current play scene functionality with this new view, so that users have one consistent and powerful experience when running their scenes or games in the editor.

As @sosasees said, the current workflow is solid and clean and should be maintained. I don't think is a good idea to default to this new mode and needing extra clicks to get to the floating window that was default. I think specific shortcuts for this mode would make more sense from a usability perspective, so if its not possible to disable it, at least do not interfere.

I think that the toolbar should remain where it is. Is the editor the one who play or stop scenes, so for me it makes more sense that is there, always in the same place.

I think this is good, specially for new users, but should not replace what is already there.

[Bastienre4]

The best answer to this "Should it be default or not" is to add an editor setting to define this. So that everyone is happy :)

[sosasees]

• When the game window is floating, the toolbar stays attached to it (I understand this may not be feasible, just an idea to have the pause and stop and other controls right by the game)

there are multiple ways this could be done:

toolbar inside window title bar

• ⭕️ it's very clear that the toolbar belongs to the window

but it has several issues that make it a bad fit for Godot:

• ❌️ if users want to add their own buttons to their game's window title bar (if that ever becomes a feature), they can't see and use them in editor builds
• ❌️ borderless windows have no window title bar, so the toolbar disappears in borderless windows
• ❌️ if the window is too narrow, the toolbar has to be truncated with an overflow menu (and with extreme small window sizes like 64×64px, even the overflow menu becomes inaccessible because it's overlapped by the minimise/maximise/close buttons)
• ❌️ Windows (and probably iMac, i don't know enough about iMac to know for sure) have good support for additional window title buttons like this, but not Linux (at least not without hacks)

toolbar under window title bar

this seems similar to toolbar inside window title bar, but it's much better like this:

• ⭕️ much more room for window title
• ⭕️ window title bar stays as-is, so on Linux we don't need to use hacks to change it
• ⭕️ toolbar is outside the window title bar, so it doesn't disappear in borderless window mode

of course, it still has some of the same advantages and disadvantages of toolbar inside window title bar:

• ⭕️ it's very clear that the toolbar belongs to the window
• ❌️ if the window is too narrow, the toolbar has to be truncated with an overflow menu (but this time the overflow menu stays accessible in extreme small window sizes like 64×64px because nothing overlaps)

and we need to be cautious of

• ❌️ Godot needs to subtract the toolbar height from the window height (just like it subtracts the window border) if we want to get the real game viewport height value

toolbar in own window

• ⭕️ toolbar does not truncate if the game window has an extreme small size like 64×64px
• ⭕️ window title bar stays as-is, so on Linux we don't need to use hacks to change it
• ❌️ connection to the game window is more abstract
• ❌️ we need extra window management programming so that the toolbar window moves as expected:
  • if it has not been moved by the user since the game started, it should move together with the game window
  • if it automatically moves to a position where it goes (at least partially) off-screen, it should move down until it's fully visible
  • if it has been moved by the user, it should not move automatically anymore until the game is restarted
  • it should be always on top if the game window is focused

toolbar under window title bar, but as its own borderless window

¿why didn't i think of this much earlier? 🤦‍♂️ (i added it when editing the comment 3 days later)

this combines the best of toolbar under window title bar and toolbar in own window:

• ⭕️ it's very clear that the toolbar belongs to the window
• ⭕️ toolbar does not truncate if the game window has an extreme small size like 64×64px
• ⭕️ window title bar stays as-is, so on Linux we don't need to use hacks to change it
• ⭕️ toolbar is outside the window title bar, so it doesn't disappear in borderless window mode
• ⭕️ we don't need much window management programming to keep the toolbar in place: just always move it to the same position relative to the window
• ❌️ Godot needs to subtract the toolbar height from the window height (just like it subtracts the window border) if we want to get the real game viewport height value

[NathanLovato]

An editor setting is good, but the default experience will still be everyone's first experience even if you add a setting, and this has a massive impact on discoverability if not handled carefully.

It should at least be thought about seriously what should be people's default experience, weigh the pros and cons. The current pop out window is seen as one of Godot's weaknesses by many users coming from Unity and Unreal, and having worked professionally with both, I understand why.

Godot's pop out window is amazing for two people, with one playtesting on one display, and the other working on the project live in the editor on another display.

However, for debugging, the ability to move your camera in Unity to a given place and switch immediately to the game view in-place, test the bug instantly, and tap out to the editor mode, is something you end up using constantly in production.

---

Note that I personally don't mind either, with what we do at GDQuest, we're not too concerned by discoverability problems, because we always keep up with the latest development in Godot. This point is about taking the time to consider the wider user-base, current and future, and thinking about what's the best experience for them.

[Bastienre4]

An editor setting is good, but the default experience will still be everyone's first experience even if you add a setting, and this has a massive impact on discoverability if not handled carefully.

It should at least be thought about seriously what should be people's default experience, weigh the pros and cons. The current pop out window is seen as one of Godot's weaknesses by many users coming from Unity and Unreal, and having worked professionally with both, I understand why.

Godot's pop out window is amazing for two people, with one playtesting on one display, and the other working on the project live in the editor on another display.

However, for debugging, the ability to move your camera in Unity to a given place and switch immediately to the game view in-place, test the bug instantly, and tap out to the editor mode, is something you end up using constantly in production.

Agreed. Also, I'm currently working on a single 21:9 display, and having the floating player window everytime I want to test the game is a pain if, as you said, I need to go back and forth in the editor.

[NathanLovato]

Bringing a good point/suggestion from a Twitter user: for mobile game developers, but also app developers, it's useful to be able to quickly test the view at different sizes matching popular mobile devices, to see if the game/UI scales properly on different displays.

So they pointed out that Unity, but we could add web browsers, offer a dropdown to quickly change the preview size to these devices' resolution.

This may be out of scope for this proposal, but seeing that the game view would be resizable, would it be possible to have a dropdown or something to select common presets? For various mobile phones, tablets, and typical desktop display ratios (16:9, 16:10, and the likes).

Please let me know if it should go into a separate proposal.

[yosoyfreeman]

An editor setting is good, but the default experience will still be everyone's first experience even if you add a setting, and this has a massive impact on discoverability if not handled carefully.

It should at least be thought about seriously what should be people's default experience, weigh the pros and cons. The current pop out window is seen as one of Godot's weaknesses by many users coming from Unity and Unreal, and having worked professionally with both, I understand why.

Godot's pop out window is amazing for two people, with one playtesting on one display, and the other working on the project live in the editor on another display.

However, for debugging, the ability to move your camera in Unity to a given place and switch immediately to the game view in-place, test the bug instantly, and tap out to the editor mode, is something you end up using constantly in production.

Note that I personally don't mind either, with what we do at GDQuest, we're not too concerned by discoverability problems, because we always keep up with the latest development in Godot. This point is about taking the time to consider the wider user-base, current and future, and thinking about what's the best experience for them.

I understand the importance of the first experience and fully agree with you. I may be failing to see the benefits tho. I think that what users miss the most is the fully editable and reversible playback and this that Reduzio pointed out:

Q: Will the 2D and 3D editors show the game objects moving around?
A: No, this is not possible and probably never will. Exploring will have to happen from the game window itself. We can add tools to make this as comfortable as possible.

We would still missing the two more famous features of those engines. So in the end, there is still a learning curve towards a different approach.

Just to understand better, you talked about quickly moving the camera to the point and moving things and seeing it in game. Is this not that?

Grabacion.de.pantalla.desde.2023-07-07.11-28-51.webm

I do agree that some way to quickly test screen ratios would be great.

PS: Please, remember that i lack social skills and i apology if my tone is not the right one. I'm just trying to give the feedback you asked in an organized way.

[sosasees]

a dropdown to quickly change the preview size to these devices' resolution

some way to quickly test screen ratios would be great.

i just got a new add-on idea:
window size panel

i will most likely not make this add-on because i only make add-ons that i need. otherwise i would make too many add-ons once and never update them to newer Godot versions.

[robbertzzz]

An editor setting is good, but the default experience will still be everyone's first experience even if you add a setting, and this has a massive impact on discoverability if not handled carefully.

Unreal has a drop-down menu next to the play button which allows you to pick how the play button behaves. I really like it because it's immediately clear to the user yet works as a project setting.

[lunarcloud]

As long as the current behavior (run it like for real) remains. It's actually something I really like.

[geekley]

To me, the most important feature here is this:

Selection Mode: This mode no longer sends input to the game and it lets you select 2D/3D nodes (depending on the selection mode described before) by clicking on them. They will appear on the remote scene tree and inspector so you can see their content.

Godot really needs a way to select nodes (instantiated at runtime) in the Game View by clicking, even if it's in the current implementation with separate window. In fact, I hope that when this is implemented, we can still use all those new features regardless of whether the game view is embedded or detached / separate window like now.
If you're gonna implement this step-by-step, this would be the best thing to have first IMO, and if you wanted to back-port just 1 thing from here to 3.x, this would be the most helpful (and the one feature that has no workarounds). (I'm not asking to back-port btw, just saying that to say it's the most important feature)

List Selection Mode: Same as the mode above, but it presents a list of nodes you can choose to select instead of selecting the topmost one.

Hopefully it will also unify with previous feature without requiring the separate mode/button, with alt+right-click.

[frkntlr]

@reduz If embedding the game engine will cause such problems, it is acceptable. It makes it harder to work on a single screen, but it's acceptable as long as it prevents the proliferation of crashes rather than interface bugs.

For mobile devices, the resolution test is an important element. We have to go into the settings every time to adjust the resolution. Even for testing purposes, it is mandatory to go into the interface and there is no chance to change settings on one screen and make other changes. It's a waste of focus to exit after adjusting every time. It's a waste of time and focus to get a build for each test and test it virtually or in real life. This should be improved even if the game window is not embedded in the editor.

[sosasees]

i also work with only 1 average-sized screen (16:9, 33 cm tall) but i don't mind moving the window around — i prefer testing my games in the real window
rather than having to export the project finished-style if i want to test window managament things like dragging the window in borderless mode

[reduz]

@RobProductions Reimplementing some behaviors that already exist in the editor in the embedded game window is far easier to do than running the game in the editor process. This is why this road is preferred.

To you as user it would be more or less the same, ultimately, but for us its much more easy to implement.

[strich]

Just to be clear, does this proposal include the out of the box ability to:

• Replicate the node hierarchy from the live game into the editor and in a mutable fashion such that nodes and their properties can be modified on the live game.
• On top of the above, implement a "scene camera" with free move to inspect the game scene.
• Modify or replace assets with immediate effect in the live game - Update textures, shaders, audio, etc.

For us, these are the actual features we would hope this whole proposal is designed to enable if not outright implement from the beginning.

[ryevdokimov]

Edit: I'm just going to make a separate proposal for this: #9142

Are the downsides of running it in the editor that bad?

With a couple tool scripts you can take control of the camera in the editor scene and reset the transform of all the nodes in the scene. I can't see it being much worse to allow the engine to do something like that natively, compared to bringing editor features to the external runtime. Combine this with split viewports, PhysicsServer set_active(true), maybe a check-in/check-out system for nodes you want to modify while the scene is running, and some other features for editor stability and user convivence, it doesn't seem too far off and could allow for a system that's pretty flexible.

In fact, in certain situations I want the reverse of this proposal to be true and be able to detach the editor viewports from the editor so that they're floating like I can do with the docks, but that's a different discussion.

2024-02-12.22-10-53.mp4

Edit: Here is a plugin that adds start/pause/stop buttons for a project I've been developing, that builds on this thought.

2024-02-13.09-11-47.mp4

[Kajinor]

I hope its ok if I put my opinion in here as well and sorry if my question got already answered as I didn't read every post.

The main issue for me was(I moved to Unreal now), that I worked with 3D with a bit more high-poly models and mainly C# in Godot. In the beginning, the playtest was very fast but the bigger the project became, the longer the playtest needed to show up. I ended up with 10-15 seconds of wait time. But having to wait 10-15 seconds or even more everytime just to see how a change reflects in-game made me move away from Godot for now (as well as some other issues with it). I know Godot has some sort of Live-Update already but its very limited and causes bugs, at least for me.

Especially after switching from Mac back to a High-End Windows PC (64 GB RAM, Core i9) Godot runs even slower than than on my M1 Macbook Air. Its hilarious. Booting Unreal Editor from VS2022 and then hit Play-In-Editor is sometimes faster than having Godot already opened and hit Play. This might be related to the still not fixed issue here but not sure: godotengine/godot#20566
As it also appears to happen with empty projects/scenes that it takes 5-7 seconds sometimes.

The main idea of this feature is that Godot will remain running the game as a sub process, with the main difference being that the game window is embedded into the editor. This can be done with relevant Vulkan extensions (and worse case if a platform does not support this, it just does not let you embed the game window, but most should support it).

If the Play-In-Editor feature runs in a sub-process does it mean that it cannot share resources with the Editor? The last thing is very important to make play-test instant I think so that the game doesnt need to load all assets, etc. again everytime you wanna see a change. Sure the Editor might crash if the game has a hard crash, happens in Unreal all the time. But since Godot uses GDScript/C# most of the time, I think crashes there can be catched more easily without crashing the entire Editor.

While I am of course not a AAA Studio or anything close, what do you think they will do? Wait 30-60 seconds or longer to show up a playtest? Switching hardware on their 100+ workstations because Godot uses DirectInput with a bug since 6 years which is still not fixed?(correct me if I am wrong). No they will simply not use it.

I really like Godot and want to come back to it with another Project. Seeing how changes reflect in the game quickly is important I think. So I hope this gets implemented in a way that is much much faster than doing a real playtest.

Thanks to everyone working on Godot making this Engine better and better!

[ltecheroffical]

The editor should be a little stable since I can think of a user having the editor freeze while having an unsaved scene and running the game Never mind, Godot saves while the game is running.

[chocola-mint]

I've been looking into this again, specifically the "sharing frames between game instance and editor" part, in an effort to actually push this proposal closer to fruition.

On Windows, there's the Spout2 library and on macOS, there's the Syphon framework, both open source. These two projects enable sharing of GPU resources and would mean there's no need to stream frames from the game to the editor and the performance impact would be minimal. Unfortunately I haven't found a Linux equivalent so far.

We could inject a hidden Camera2D/Camera3D+SubViewport ingame and share the SubViewport's viewport texture with the editor, allowing us to have what is basically a debug camera similar to Unity's Scene View. The only inter-process communication needed would be to send commands to remotely move the debug camera from the editor GUI.

I've looked into using Spout2 through a GDExtension, but critically it's impossible to get a Texture2D's OpenGL texture ID from the available API (of course, for good reasons), so there's no way but to implement this feature in core.

[JonathanDotCel]

I've been looking into this again, specifically the "sharing frames between game instance and editor" part, in an effort to actually push this proposal closer to fruition.

On Windows, there's the Spout2 library and on macOS, there's the Syphon framework, both open source. These two projects enable sharing of GPU resources and would mean there's no need to stream frames from the game to the editor and the performance impact would be minimal. Unfortunately I haven't found a Linux equivalent so far.

We could inject a hidden Camera2D/Camera3D+SubViewport ingame and share the SubViewport's viewport texture with the editor, allowing us to have what is basically a debug camera similar to Unity's Scene View. The only inter-process communication needed would be to send commands to remotely move the debug camera from the editor GUI.

I've looked into using Spout2 through a GDExtension, but critically it's impossible to get a Texture2D's OpenGL texture ID from the available API (of course, for good reasons), so there's no way but to implement this feature in core.

Maybe overkill?
It's possible to just parent the game process's window to the editor window's panel.
(not just place it always on top and in front, actually set the game window's parent)
No weird overlay issues, no double taskbar issues, no floaty overlay nonsense.

I posted a working windows proof of concept a few months back.

[mrussogit]

To me the real problem is the lack of ability to properly inspect objects when the game is running so we know, visually what we are selecting in Remote Tree. If I instantiate a number of unit objects, say for a RTS game, the first problem is that Godot gives them fancy names. Second, Remote tree isn't selected when the game is running automatically and game window will go to background when I click Remote. So every time I'm have to move the game window to a second monitor so I can click Remote without sending the running game window to background. I imagine the pain for people using one monitor only. Then if I want to select a unit to inspect an odd behavior, I can select its name in Remote but I have no idea which unit it is. There is no highlight, no outline, nothing indicating which unit I just selected.

It doesn't matter much if it runs in editor or outside, as long as we can select an object in a Remote tree (that is automatically selected when a game is running) and we can have some feedback to what is being selected while the game is running. If the game is not running in editor, to have an option to have the game running window stuck and not send to background would be huge plus as well. -> (edit: this was already implemented and I didn't know. Thanks ydeltastar it's a setting on display/window/size/always_on_top but must be on advanced to see it.

[ydeltastar]

to have an option to have the game running window stuck and not send to background would be huge plus as well.

There is project setting display/window/size/always_on_top, editor setting run/window_placement/screen to make it start on another screen, and other window configurations. As I see it, window placement is not much of an issue compared to the lack of feedback and intercommunication between the editor and the running game.

[mrussogit]

to have an option to have the game running window stuck and not send to background would be huge plus as well.

There is project setting display/window/size/always_on_top, editor setting run/window_placement/screen to make it start on another screen, and other window configurations. As I see it, window placement is not much of an issue compared to the lack of feedback and intercommunication between the editor and the running game.

Thanks a LOT. It was on advanced settings and turned off by default. Huge help! Much appreciated!

[bbbscarter]

@reduz

So, overall, its better (and likely a lot less complex) to figure out how to do what you need to do visually debug your game even if as a separate process, and make it as useful, or nearly as useful as Unity despite that.

Just to follow up on this, as posted in #9142.

I agree, and looked into the approach of pushing some general debugging features into the runtime, while improving communication back to the editor, and created a proof-of-concept addon:

https://github.com/bbbscarter/GodotRuntimeDebugTools

For me, it closes the gap with Unity quite significantly:

• A runtime debug camera, with editor controls in game.
• Clicking on things in game selects them in the editor.
• Clicking on things in the editor selects them in the game.
• Selected objects are highlighted in game.
• Runtime toggling of various debug views.

Although it's an add-on, it's mostly intended as a proof of concept for a something that could be implemented in mainline Godot in the future.

[ltecheroffical]

@reduz

So, overall, its better (and likely a lot less complex) to figure out how to do what you need to do visually debug your game even if as a separate process, and make it as useful, or nearly as useful as Unity despite that.

Just to follow up on this, as posted in #9142.

I agree, and looked into the approach of pushing some general debugging features into the runtime, while improving communication back to the editor, and created a proof-of-concept addon:

https://github.com/bbbscarter/GodotRuntimeDebugTools

For me, it closes the gap with Unity quite significantly:

* A runtime debug camera, with editor controls in game.

* Clicking on things in game selects them in the editor.

* Clicking on things in the editor selects them in the game.

* Selected objects are highlighted in game.

* Runtime toggling of various debug views.

Although it's an add-on, it's mostly intended as a proof of concept for a something that could be implemented in mainline Godot in the future.

I can see something like this being helpful if integrated with the editor directly, however in a addon it can't be done like that, so it looks like a direct change to the editor is necessary.

[bbbscarter]

I can see something like this being helpful if integrated with the editor directly

Checking - what additional functionality did you have in mind, that can't be done via an addon?

I very much see a benefit to something this being integrated directly into the editor, but I was thinking in terms of availability and UI integration polish, rather than anything more fundamental. Did you have something else in mind?

[ltecheroffical]

Ideally it should have it all within the editor window (starting to require modifying the editor’s code), while allowing the objects to move within the editor when selecting the remote scene and what would help is also being able to put it side by side with the game so you might be able to see for example why a camera is looking crazy.

…

On Sep 13, 2024, at 2:47 AM, Simon Carter ***@***.***> wrote: I can see something like this being helpful if integrated with the editor directly Checking - what additional functionality did you have in mind, that can't be done via an addon? I very much see a benefit to something this being integrated directly into the editor, but I was thinking in terms of availability and UI integration polish, rather than anything more fundamental. Did you have something else in mind? — Reply to this email directly, view it on GitHub <#7213 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ATXERVG5FJFEM5R6KX2RGJDZWKC6VAVCNFSM6AAAAAA2AKJDY2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBYGE2TMOBYG4>. You are receiving this because you were mentioned.

[bbbscarter]

I see - thanks for the explanation! And agreed, your crazy camera debugging example is exactly the sort of thing I'd be keen on targeting.

My thinking - since Godot has a clear separation between the game editor processes, the mental model this encourages is the IPC-based model discussed above. e.g.

• The editor can read from the game process, and display UI based on the game state (e.g. the remote scene tree).
• The editor can send some commands to the game process.
• Any modifications to game state happen in the game process.

Given that IPC model, I suspect you could probably get quite a long way without necessarily making changes to the editor - largely by rebuilding parts of the editor in the game process.

For example, using the existing add-on above you can already:

• Select an object in the game, and see it in the remote scene tree in the editor.
• Edit properties of that object in the editor in the remote scene tree - e.g. move the camera around. This updates in the game view.
• You can see a transform gizmo on the selected object.

Obvious extensions you could make to this would include:

• In the game process, make 'invisible' object visible and clickable with icons. This way you could select the camera by directly clicking in the game window.
• In the game process, allow the transform handles of the selected node to be dragged around. e.g. basic scene editing controls in the game process.

At which point, your crazy camera example should be fairly easily debuggable.

If you combined such an add-on with the original proposal ("embedding the game window in the editor"), you would probably have something very close to a 'live editable game running in the editor'.

All of which is to say:

• Yes, I think this functionality should be baked in, rather than an add-on.
• The original proposal of embedding the game window will obviously require editor changes.
• But outside of that, you can probably get a very long way without any editor changes.

[ltecheroffical]

I see - thanks for the explanation! And agreed, your crazy camera debugging example is exactly the sort of thing I'd be keen on targeting.

My thinking - since Godot has a clear separation between the game editor processes, the mental model this encourages is the IPC-based model discussed above. e.g.

* The editor can read from the game process, and display UI based on the game state (e.g. the remote scene tree).

* The editor can send some commands to the game process.

* Any modifications to game state happen in the game process.

Given that IPC model, I suspect you could probably get quite a long way without necessarily making changes to the editor - largely by rebuilding parts of the editor in the game process.

For example, using the existing add-on above you can already:

* Select an object in the game, and see it in the remote scene tree in the editor.

* Edit properties of that object in the editor in the remote scene tree - e.g. move the camera around. This updates in the game view.

* You can see a transform gizmo on the selected object.

Obvious extensions you could make to this would include:

* In the game process, make 'invisible' object visible and clickable with icons. This way you could select the camera by directly clicking in the game window.

* In the game process, allow the transform handles of the selected node to be dragged around. e.g. basic scene editing controls in the game process.

At which point, your crazy camera example should be fairly easily debuggable.

If you combined such an add-on with the original proposal ("embedding the game window in the editor"), you would probably have something very close to a 'live editable game running in the editor'.

All of which is to say:

* Yes, I think this functionality should be baked in, rather than an add-on.

* The original proposal of embedding the game window will obviously require editor changes.

* But outside of that, you can probably get a very long way without any editor changes.

Your plugin is very close to the best we can do with just plugins. I mean, while you could stream the nodes over IPC, it will most likely be slow. In fact, while I was making the plugin referenced in this comment, then one of the things that stopped me was the speed, but if we were to take this to C++, then we could get a lot more speed. But while I was writing this, I remembered that if I can stream nodes back from the game, then something similar can be implemented in C++. While I can try my hand at this myself, I'm not yet that familiar with Godot's codebase that much but I can revisit that plugin again.

[Cronos87]

Huge thanks to @YeldhamDev for the PR 🙏 That's amazing!

I wonder, as this kind of work is a rework from the ground up, is it expected to be merged for 4.5 or 4.6?

No pression, simple curiosity 😊

[timothyqiu]

I wonder whether features like "pause frame", "next frame", and "object selection in the game window" really related to "embedding the game window"?

[Calinou]

I wonder whether features like "pause frame", "next frame", and "object selection in the game window" really related to "embedding the game window"?

I have a dedicated proposal for frame-by-frame stepping: #3105

I agree this shouldn't be tied to embedding the game window, i.e. you should be able to do it even if the window runs separately from the editor (and even when running on a remote device, when using remote debug).

[davifoox]

Someone recently released a plugin that embeds the game window:

https://fabimakesgames.itch.io/embedgame

[ltecheroffical]

I wonder whether features like "pause frame", "next frame", and "object selection in the game window" really related to "embedding the game window"?

I'm pretty sure that at least some of us got way to excited and drove this discussion way offtopic.

[allenwp]

When game window embedding is implemented, I believe that a "Make the game editor floating" button should be added as follows:

I have detailed this a bit more in godotengine/godot#99010 (comment) to include a full screenshot of what this might look like.