Decos: Window decoration flags, shadow improvements

[vaxerski]

Motivation

Mostly code cleanup + this:

Notice how the shadow includes the top bar

Breaking Stuff

IHyprWindowDecoration::allowsInput yeeten

RFC

@MightyPlaza (deco MRs)

Numero uno

enum eDecorationFlags {
    DECORATION_ALLOWS_MOUSE_INPUT  = 1 << 0, /* this decoration accepts mouse input */
    DECORATION_PART_OF_MAIN_WINDOW = 1 << 1, /* this decoration is a *seamless* part of the main window, so stuff like shadows will include it */
};

Is this enough? Do we want any other flags? We have space for 64.

Numero dos

Another thing is how we stencil the shadow.

Hyprland/src/render/decorations/CHyprDropShadowDecoration.cpp

Lines 71 to 74 in 66b8b80

	// +1 +1 -2 -2 is to avoid artifacts with AA. TODO: figure out a better method. Alpha blending? This same shit will happen on hyprbars.
	m_bLastWindowBox = {(int)(m_vLastWindowPos.x - maxExtents.topLeft.x - BORDER + 1), (int)(m_vLastWindowPos.y - maxExtents.topLeft.y - BORDER + 1),
	(int)(m_vLastWindowSize.x + maxExtents.topLeft.x + maxExtents.bottomRight.x + 2 * BORDER - 2),
	(int)(m_vLastWindowSize.y + maxExtents.topLeft.y + maxExtents.bottomRight.y + 2 * BORDER - 2)};

Currently, I have to do this, because the stencil is very 0/1. Even if we do a != 1, there still will be an issue if the top bar is transparent:

There is no way to avoid this with the current stencil solution. Are we able to make some sort of an alpha blending mask here? Any opengl wizards with some khronos docs / ideas?

[vaxerski]

Could we perhaps plop another 2D sampler to a fragment shader... and use that as a mask? hm.

[MightyPlaza]

Numero uno

enum eDecorationFlags {
    DECORATION_ALLOWS_MOUSE_INPUT  = 1 << 0, /* this decoration accepts mouse input */
    DECORATION_PART_OF_MAIN_WINDOW = 1 << 1, /* this decoration is a *seamless* part of the main window, so stuff like shadows will include it */
};

Is this enough? Do we want any other flags? We have space for 64.

I think we should try to get a more robust decoration placement system
Maybe something like a decoration order, so that the user could easily set it's value in the config and avoid any decorations overlapping each other

We could have a decoration manager to help manage, place and handle the decorations
Decoration should call it instead of getting window position and size directly. This would mean decoration type could also be a flag, allowing 1 sided decorations for stuff like bars or panels, 2 and 3 sided for directional shadows and 4 sided for conventional shadows and borders

Some decorations like groupbars/hyprbars could also benefit from a opening/closing slide animation, which would be another possible flag

I'll see if I can think of other possible use cases

[vaxerski]

I'll push the alpha matte fix for the shadows thing when github stops dying. It looks pretty alright imo.

[vaxerski]

I think we should try to get a more robust decoration placement system
Maybe something like a decoration order, so that the user could easily set it's value in the config and avoid any decorations overlapping each other

We could have a decoration manager to help manage, place and handle the decorations
Decoration should call it instead of getting window position and size directly. This would mean decoration type could also be a flag, allowing 1 sided decorations for stuff like bars or panels, 2 and 3 sided for directional shadows and 4 sided for conventional shadows and borders

That's a great idea and something I've also thought about, but it would be for another MR.

The current deco placement system is... bad.

[MightyPlaza]

I'll push the alpha matte fix for the shadows thing when github stops dying. It looks pretty alright imo.

if this has been fixed, should the shadows start from the end of the border instead of the end of the window?

also, seem like there is some delay with resizing the window and the border and shadows following the same size. Not sure it comes from here

[vaxerski]

if this has been fixed, should the shadows start from the end of the border instead of the end of the window?

IMO yes. The previous behavior (no borders) sucked.

also, seem like there is some delay with resizing the window and the border and shadows following the same size. Not sure it comes from here

huh, can't see any.

[MightyPlaza]

video link: https://filebin.net/xwfcnj6fhrrv26pc

[vaxerski]

ah, that's normal. Not this mr's fault.

[vaxerski]

regarding flags, I am talking about general flags. WD positioner MR will have a separate flag enum for positioning-related ones most likely

[MightyPlaza]

regarding flags, I am talking about general flags. WD positioner MR will have a separate flag enum for positioning-related ones most likely

possibly one for animating like I said
we could also separate the different types of input into different flags

DECORATION_PART_OF_MAIN_WINDOW would probably be moved inside positioning enum then as well, since it controls position relative to the shadows (and others)

[vaxerski]

possibly one for animating like I said

elaborate? How would that work?

DECORATION_PART_OF_MAIN_WINDOW would probably be moved inside positioning enum then as well, since it controls position relative to the shadows (and others)

No, this is only a rendering hint.

[MightyPlaza]

possibly one for animating like I said

elaborate? How would that work?

I kinda attempted this on the original groupbars PR with 6013a3c
to fix some damaging issues when drawing decorations inside the bar I turned the decoration extents into CAnimatedVariable
which just for 1 window decoration, would animate it if using that value as the decoration height when rendering it.
didn't finish this because it would obviously require more vars or better control over decorations

on a closer look the resizing issue definitely seems worse on this pr

[vaxerski]

to fix some damaging issues when drawing decorations inside the bar I turned the decoration extents into CAnimatedVariable

hm. For that I'd just add custom onChange callbacks for damage control (which I've been planning to do anyways...) and that would be solved. Changing extents all the time is a bad idea.

on a closer look the resizing issue definitely seems worse on this pr

idk how

[MightyPlaza]

hm. For that I'd just add custom onChange callbacks for damage control (which I've been planning to do anyways...) and that would be solved. Changing extents all the time is a bad idea.

the damage issues were when the total extents were changed but for some reason the border would not get damaged
my point was that we could animate the decoration real render size on creation and destruction of the decoration
not sure how if it would look bad without animating the extents, but probably will look fine

also shadows don't seem to be damaged properly

[vaxerski]

my point was that we could animate the decoration real render size on creation and destruction of the decoration
not sure how if it would look bad without animating the extents, but probably will look fine

bad idea. That would necessitate a layout recalc. No No No.

[MightyPlaza]

my point was that we could animate the decoration real render size on creation and destruction of the decoration
not sure how if it would look bad without animating the extents, but probably will look fine

bad idea. That would necessitate a layout recalc. No No No.

I don't think so, layout already gets recalc'd if the extents change, just render the decoration increasing at the right rate to match the window size decreasing rate animation. This way the total window outer box size would change only once, since window_height(t) + decoration_height(t) = initial_window_height

but since that wouldn't require external control, since the deco could calculate current size from the initial window size and the current window size, maybe there is no point in adding a flag for that

[vaxerski]

damage tracking should be fixed

[its-darsh]

does there is a way to write decorations with GTK+?

[vaxerski]

yes

[vaxerski]

though worth mentioning you'd have to write your own gdk backend most likely. IIRC gtk can work with cairo as a backend. If that's the case, you can use cairo surfaces easily to draw stuff in hyprland. Hyprbars use cairo extensively.

Would be an interesting plugin.

[vaxerski]

Since no further comments have arisen, I'll merge. Flags can always be added in le future