Blur-Behind / Glassy Windows implementation #568

haudan · 2018-06-15T18:51:00Z

Implements #538.

Backend progress:

macOS
Windows
X
Wayland

Tested on all platforms changed
Added an entry to CHANGELOG.md if knowledge of this change could be valuable to users
Updated documentation to reflect any user-facing changes, including notes of platform-specific behavior
Created an example program if it would help users understand this functionality

haudan · 2018-06-15T18:56:36Z

src/os/macos.rs

@@ -16,6 +16,9 @@ pub trait WindowExt {
    ///
    /// The pointer will become invalid when the `Window` is destroyed.
    fn get_nsview(&self) -> *mut c_void;
+
+    /// Just for testing purposes.


Should probably change that comment.

haudan · 2018-06-15T19:20:53Z

Once the changes are merged, I will submit a PR to glutin for compability with these changes. For the time being, in order to follow along, clone glutin and apply these changes. Also check for newer commits.

francesca64

I won't get around to actually testing this until tomorrow, but I've left you plenty of super fun nitpicks.

francesca64 · 2018-06-15T18:53:50Z

src/platform/macos/view.rs

-        IdRef::new(msg_send![view, initWithWinit:state_ptr])
+        let view_class = if win_attribs.blur { VISUAL_EFFECT_VIEW_CLASS.0 } else { VIEW_CLASS.0 };
+        let view: id = msg_send![view_class, alloc];
+        msg_send![view, initWithWinit:state_ptr];


Proper Obj-C style dictates that we rebind view to the return of the init method. While it's typically the same pointer, it's not idiomatic to assume that.

francesca64 · 2018-06-15T18:54:59Z

src/os/macos.rs

+#[repr(i64)]
+// Applies to MacOS Mojave.
+pub enum BlurMaterial {
+    AppearanceBased = 0, // Deperecated


Since this enum is public, the deprecation notices should be doc comments above the variant.

francesca64 · 2018-06-15T18:56:00Z

src/platform/macos/view.rs

@@ -13,11 +13,12 @@ use cocoa::foundation::{NSPoint, NSRect, NSSize, NSString, NSUInteger};
 use objc::declare::ClassDecl;
 use objc::runtime::{Class, Object, Protocol, Sel, BOOL};

-use {ElementState, Event, KeyboardInput, MouseButton, WindowEvent, WindowId};
+use {ElementState, Event, KeyboardInput, MouseButton, WindowEvent, WindowId, WindowAttributes};


I've been trying to keep imports sorted alphabetically, so WindowAttributes should go before WindowEvent.

francesca64 · 2018-06-15T18:56:24Z

src/platform/macos/view.rs

 use platform::platform::events_loop::{DEVICE_ID, event_mods, Shared, to_virtual_key_code};
 use platform::platform::util;
 use platform::platform::ffi::*;
 use platform::platform::window::{get_window_id, IdRef};
+use os::macos::BlurMaterial;


likewise, os imports go above platform imports.

Though, I might be more comfortable with you defining the BlurMaterial within platform, as IIRC no other types define themselves in os.

I tried defining BlurMaterial inside platform, but that creates a problem with module visibility. os needs access to this enum, but platform is inaccessible from os.

I don't know if we even should keep the enum, see my other comments.

francesca64 · 2018-06-15T19:00:12Z

src/os/macos.rs

@@ -157,3 +165,27 @@ impl MonitorIdExt for MonitorId {
        self.inner.get_nsscreen().map(|s| s as *mut c_void)
    }
 }
+
+#[repr(i64)]


Are these actually supposed to be i64, or are they an architecture-dependent size? I'd appreciate a comment including the NSWhatever name of the enum.

It's an i64 based on the current NSVisualEffectView.setMaterial method signature. The method takes a c-style enum, which is represented as an i64. I don't think this is set in stone but I doubt it will ever change.

I have added some valuable documentation in my new commit.

francesca64 · 2018-06-15T19:01:57Z

src/platform/macos/view.rs

+        let view: id = msg_send![view_class, alloc];
+        msg_send![view, initWithWinit:state_ptr];
+        if win_attribs.blur {
+            msg_send![view, setMaterial: BlurMaterial::Light as i64];


You have to do let _: () = msg_send!, as otherwise segsegv/sigill will be triggered at runtime when the ! (never) type is stabilized.

francesca64 · 2018-06-15T19:03:15Z

src/platform/macos/window.rs

@@ -3,6 +3,7 @@ use std::ops::Deref;
 use std::os::raw::c_void;
 use std::sync::Weak;
 use std::cell::{Cell, RefCell};
+use os::macos::BlurMaterial;


I've been grouping imports into blocks, ordered:

std

Crates

Internal

With a newline between each block.

(I know this isn't consistently applied throughout the repo yet, but we're getting there.)

francesca64 · 2018-06-15T19:06:46Z

src/platform/macos/window.rs

+    #[inline]
+    fn set_blur_material(&self, material: BlurMaterial) {
+        let view = self.get_nsview() as *mut objc::runtime::Object;
+        unsafe { msg_send![view, setMaterial: material as i64]; }


In every other usage, we call msg_send! on *self.view. Doing that is much cleaner (and has less potential overhead) than calling get_nsview like this.

Additionally, with setMaterial: material, from what I've seen Obj-C style has us writing setMaterial:material. This applies to the other occurrences as well, naturally.

francesca64 · 2018-06-15T19:15:33Z

src/lib.rs

@@ -482,6 +482,11 @@ pub struct WindowAttributes {
    /// [iOS only] Enable multitouch,
    /// see [multipleTouchEnabled](https://developer.apple.com/documentation/uikit/uiview/1622519-multipletouchenabled)
    pub multitouch: bool,
+
+    /// Whether the window should be blurry. This is similar to transparency.


"blurry" has a pretty strong negative connotation. "blurred" sounds more neutral, but that has associations with input focus... "the window should have a blur effect" probably sounds the coolest, and then the second sentence should go on to explain things more.

francesca64 · 2018-06-15T19:20:40Z

src/window.rs

@@ -96,6 +96,15 @@ impl WindowBuilder {
    #[inline]
    pub fn with_transparency(mut self, transparent: bool) -> WindowBuilder {
        self.window.transparent = transparent;
+        self.window.blur = !transparent;


What's the intention with the negation here?

Transparency and blur are mutually exclusive. The negation ensures that transprency and blur are always set to the opposite of each other, so that both can never be true at the same time. This avoids running code paths for transparency in backends when blur is enabled (and vice versa), which in turn avoids potential bugs and improves performance.

Writing this comment, I noticed that this obviously creates a bug. When for example with_blur(false) is called, transparency will be enabled, which is wrong. I'm going to fix this.

elinorbgr · 2018-06-16T09:23:31Z

For the record, there is currently no way to have this implemented for Wayland, and I doubt there will be in any short-term timescale.

- Improve documentation for BlurMaterial, methods and members. - Reorganize imports - Annotate return type of msg_send! usages - Rebind view to the return value of init, as is idiomatic in Objective-C - Mark set_blur_material as unsafe - Change some code styling for consistency

haudan

I checked the changes and implemented them accordingly. Commits are already pushed. Please review again!

haudan · 2018-06-15T19:00:45Z

src/os/macos.rs

+
+#[repr(i64)]
+// Applies to MacOS Mojave.
+pub enum BlurMaterial {


So this enum is problematic. I created it based on the documentation Apple updated a while ago, based on the upcoming SDK version for macOS Mojave. Most of these variants are not supported on High Sierra, and will crash the application when attempted to be used. I'm unsure how to go about this. Should we remove the enum entirely and make the set_blur_material method take an i64 instead? That would more or less remove the OS version dependency from winit.

haudan · 2018-06-17T12:43:10Z

src/platform/macos/view.rs

 use platform::platform::events_loop::{DEVICE_ID, event_mods, Shared, to_virtual_key_code};
 use platform::platform::util;
 use platform::platform::ffi::*;
 use platform::platform::window::{get_window_id, IdRef};
+use os::macos::BlurMaterial;


I tried defining BlurMaterial inside platform, but that creates a problem with module visibility. os needs access to this enum, but platform is inaccessible from os.

I don't know if we even should keep the enum, see my other comments.

haudan · 2018-06-17T14:01:54Z

src/os/macos.rs

@@ -157,3 +165,27 @@ impl MonitorIdExt for MonitorId {
        self.inner.get_nsscreen().map(|s| s as *mut c_void)
    }
 }
+
+#[repr(i64)]


It's an i64 based on the current NSVisualEffectView.setMaterial method signature. The method takes a c-style enum, which is represented as an i64. I don't think this is set in stone but I doubt it will ever change.

I have added some valuable documentation in my new commit.

haudan · 2018-06-17T14:24:35Z

src/window.rs

@@ -96,6 +96,15 @@ impl WindowBuilder {
    #[inline]
    pub fn with_transparency(mut self, transparent: bool) -> WindowBuilder {
        self.window.transparent = transparent;
+        self.window.blur = !transparent;


Transparency and blur are mutually exclusive. The negation ensures that transprency and blur are always set to the opposite of each other, so that both can never be true at the same time. This avoids running code paths for transparency in backends when blur is enabled (and vice versa), which in turn avoids potential bugs and improves performance.

Writing this comment, I noticed that this obviously creates a bug. When for example with_blur(false) is called, transparency will be enabled, which is wrong. I'm going to fix this.

francesca64 · 2018-06-17T14:43:36Z

src/os/macos.rs

+    /// this controls the appearance of the blur effect.
+    /// 
+    /// Marked as unsafe because depending on the version of macOS and the `BlurMaterial` variant passed,
+    /// this might cause a crash.


Is there a lowest common denominator of supported enum variants? I don't think people will be comfortable using a method that can crash based on criteria that are only vaguely explained.

The lowest common denominator appears to be AppearanceBased, Light, Dark, MediumLight and UltraDark (source), but those are now marked as deprecated. There are some marked as "modified", I think that means their underlying integer value has changed (validation needed), which would disqualify them. This is all quite problematic since this swift enum is our only source of truth and it appears Apple is comfortable changing the order of constants willy nilly - they only keep the swift code consistent, but apparently not the enum's binary representation.

Honestly I don't know what to do here. This is why I was originally unsure about the BlurMaterial enum in the first place.
The only alternative I see is exposing the raw i64 and removing the enum, but that won't solve the crash / macOS-version dependency problem either.

How impractical would it be for us to translate the values depending on the version of macOS?

Also, shouldn't it be isize instead of i64?

I don't think that would be too much work, definitely doable.

Yeah you could be right with isize over i64. The right size is whatever the size of the swift enum is. isize should be a good bet.

OvermindDL1 · 2018-06-18T15:58:43Z

Just for note, KDE5's wayland client supports blue-behind/glassy windows as long as the proper transparency is set for any given pixel.

elinorbgr · 2018-06-18T16:07:12Z

Does it actually blur the contents under it ? This PR is not about windows with a (partially) transparent background, but about asking the display server to blur the contents behind the transparent window.

OvermindDL1 · 2018-06-18T16:19:32Z

It is proper blur, came out in Plasma 5.13, example video (this is apparently an early version of it, bit more refined in the release version):
https://www.youtube.com/watch?v=xroL6ZWg2Gs

The settings are global, but any properly transparent window (or parts there-of) will have the system-set blur. I'm unsure if there are per-window settings that are controllable, though knowing KDE Plasma that wouldn't surprise me if there is.

OvermindDL1 · 2018-06-18T16:21:45Z

Ah, correction, apparently it's enabled on a window when the window has the _KDE_NET_WM_BLUR_BEHIND_REGION property enabled on it (potentially other options?).

elinorbgr · 2018-06-18T16:27:59Z

Ah I see. Though, it seems that this _KDE_NET_WM_BLUR_BEHIND_REGION property is an x11 mechanism, not a wayland one. So it would not be applicable to native wayland clients.

OvermindDL1 · 2018-06-18T16:29:14Z

Using the API itself blur is enabled by KWindowEffects::enableBlurBehind(winId, true);, and this would enable it on both X11 and Wayland.

For note, KDE is only performing bug fixes for X11 currently, new development is exclusively Wayland. The Blur uses the existing compositor functionality so it works on both.

OvermindDL1 · 2018-06-18T16:31:11Z

It seems you can even force blur on when it is default disabled via setting the WindowForceBlurRole attribute to true via the API.

OvermindDL1 · 2018-06-18T16:33:16Z

Documention I found this at starts with:
https://api.kde.org/4.x-api/kde-workspace-apidocs/kwin/libkwineffects/html/group__kwineffects.html
and
https://api.kde.org/4.x-api/kde-workspace-apidocs/kwin/libkwineffects/html/namespaceKWin.html

So at least KDE should be covered it seems once the calls are made. Just remember to feature detect for Plasma API 5.13 or higher to use it.

elinorbgr · 2018-06-18T16:48:39Z

Ah so this would need to link winit against Qt/KDE... Not the kind of solution I was hoping for :/

OvermindDL1 · 2018-06-18T17:03:20Z

Dynamically at most, though KDE does have a habit of exposing most things over DBus as well, so it might be available there.

What is winit linking to as it stands? Is it just direct X11/Wayland interfaces? If so then I'm unsure how you would be able to control effects such as blur as those are handled by the compositor on each, of which Plasma is KDE's compositor (I think there is another DE or two that uses plasma as well) so compositor interfaces would have to be called regardless. Plasma's Compositor is significantly the most complete on Wayland (it also has backwards compat with X11) and it even has a testing framework that does not require an active framebuffer (it has a virtual framebuffer, along the lines xvfb for X11 but one that works on Wayland as well) that is fantastic for performing automated testing (of which KDE has a substantial amount of), useful if you don't already have automated wayland testing.

elinorbgr · 2018-06-18T18:10:51Z

What is winit linking to as it stands? Is it just direct X11/Wayland interfaces?

Currently yes. Ideally we would like to keep it that way as much as possible I believe.

For wayland, the ideal situation would be if Plasma exposed some KDE-specific protocol extension, I think. Though if not we'll need to work around that.

I think it may be worth opening an other issue to discuss how we should integrate WM-specific features in winit in general for X11/Wayland. This is getting out of scope for this PR, with only touches MacOS specific code.

OvermindDL1 · 2018-06-18T18:27:49Z

For wayland, the ideal situation would be if Plasma exposed some KDE-specific protocol extension, I think. Though if not we'll need to work around that.

I'm like 90% certain it does as I'm pretty sure I ran across it a few months ago on their Wayland interfaces...

I think it may be worth opening an other issue to discuss how we should integrate WM-specific features in winit in general for X11/Wayland. This is getting out of scope for this PR, with only touches MacOS specific code.

Perhaps the title should be changed to reflect it then? The title implies nothing about Apple-specific interfaces at all? ^.^;

👍

haudan · 2018-06-18T20:32:57Z

I think it may be worth opening an other issue to discuss how we should integrate WM-specific features
in winit in general for X11/Wayland. This is getting out of scope for this PR, with only touches MacOS
specific code.

Perhaps the title should be changed to reflect it then? The title implies nothing about Apple-specific
Interfaces at all? ^.^;

This PR is for the implementation on all platforms. I'm currently working on the windows backend. The code is written, but it doesn't work... doh!

Thanks for the discussions, by the way. That will definitely help me on the Linux side.

OvermindDL1 · 2018-06-18T21:16:31Z

Thanks for the discussions, by the way. That will definitely help me on the Linux side.

Sure. :-)

I'm mostly in the KDE world so that is what I primarily know (it's the only setup that does things like HiDPI and such 'mostly' right and so forth), but yeah it is the compositor that handles all that, and as far as I know there are 2 such compositors out, KDE's KWin is one, and Compiz is the other. I'm unsure if there are others but those are the two big ones, and only KWin has full production support for Wayland at the current time.

haudan · 2018-06-18T21:53:02Z

I just pushed a raw Windows implementation. If anyone want's to try it, just remember to resize the window. The client are needs to be redrawn for the blur effect to become visible.

By the way I accidentally rustfmted some files, which might have killed some manual formatting.

francesca64 · 2018-06-18T21:59:03Z

By the way I accidentally rustfmted some files, which might have killed some manual formatting.

I won't review this until you revert that.

elinorbgr · 2018-06-19T09:57:11Z

@Lisoph

This PR is for the implementation on all platforms. [...]
Thanks for the discussions, by the way. That will definitely help me on the Linux side.

Alright, my bad.

If you plan to do the wayland implementation as well, I expect this will require some digging and possible PRs to wayland-protocols and/or sctk. I right now am quite overwhelmed but a lot of stuff and don't have the time to do all this digging around, but if someone manages to fond out what needs to be done, actually doing it should be pretty quick and straightforward.

sp3d · 2018-06-20T08:44:13Z

@vberger, digging suggests that https://github.com/KDE/kwayland/blob/master/src/client/protocols/blur.xml is the source of truth for the KDE blur region protocol, so it should just be a matter of binding it.

fschutt · 2018-11-11T04:15:18Z

Regarding KDE on X11 at least, _KDE_NET_WM_BLUR_BEHIND_REGION might be worth a look. As far as I am aware, it is a regular X11 Atom property like so many others.

Using xprop:

xprop -f _KDE_NET_WM_BLUR_BEHIND_REGION 32c -set _KDE_NET_WM_BLUR_BEHIND_REGION 0 -id $x11_window_id

friktor · 2019-01-28T13:41:38Z

Hey. Can it be better to accept the ready changes for blur in mac os? I don`t think that in the near future anyone would want to be engaged in the implementation of blur for windows. But if you do not actualized this PR and do not accept it - over time, this work will depreciate, and someone will have to write it from scratch. @francesca64 @Lisoph

haudan · 2019-01-28T13:59:14Z

@friktor That's fine with me. Waiting for @francesca64's opinion.

friktor · 2019-01-28T16:13:10Z

@Lisoph Good) It's just that this PR has been active for almost a year now, and would it be better to decide the fate of this feature.

jansol · 2019-03-24T22:06:15Z

@Lisoph did you ever revert the formatting you mentioned? GitHub isn't very good at showing forced pushes sometimes...

Osspial · 2019-04-24T20:47:49Z

@Lisoph just fyi - @francesca64 is no longer working on Winit, I should be able to test this out on Windows, though.

haudan · 2019-05-06T18:19:40Z

src/platform/windows/events_loop.rs

@@ -473,7 +473,8 @@ pub unsafe extern "system" fn callback(
                window_id: SuperWindowId(WindowId(window)),
                event: Refresh,
            });
-            winuser::DefWindowProcW(window, msg, wparam, lparam)
+            //winuser::DefWindowProcW(window, msg, wparam, lparam) // TODO: Why was this here?


Like the comment says - DefWindowProc doesn't appear to make much sense for WM_PAINT. In fact it broke the redrawing code in the example. Can we remove this without breaking anything or am I missing something?

This line is also present in the most recent commit from upstream.

haudan · 2019-05-06T18:21:38Z

@Lisoph just fyi - @francesca64 is no longer working on Winit, I should be able to test this out on Windows, though.

@Osspial Thanks for the info! I'm still actively working on this (just fixed the modern Windows 10 implementation). Can we change to a different reviewer if @francesca64 is no longer working on winit? I'll try to merge with master and then take a look at the formatting.

goddessfreya · 2019-05-07T00:59:29Z

I can test this for linux, once it's impl'd. Can this be rebased against eventloops v2?

goddessfreya · 2019-07-26T07:21:38Z

Can someone rebase this PR against master and reopen it? It seams to have not aged well, so I'm closing it. Thanks.

haudan added 6 commits June 2, 2018 00:13

Blur: WIP/experimental MacOS implementation

2cb2826

MacOS: Basic blur implementation

03e7eb2

Merge branch 'master' of https://github.com/tomaka/winit

d0e7a2f

Add macOS related comment to blur example

a0e9484

Comment blur member

e7686aa

Merge branch 'master' of https://github.com/tomaka/winit

ea070f0

francesca64 added the DS - macos label Jun 15, 2018

haudan commented Jun 15, 2018

View reviewed changes

francesca64 suggested changes Jun 15, 2018

View reviewed changes

haudan added 2 commits June 17, 2018 16:08

Fix buggy mutual exclusion between trans. and blur

79d1f94

haudan commented Jun 17, 2018

View reviewed changes

francesca64 approved these changes Jun 17, 2018

View reviewed changes

francesca64 reviewed Jun 17, 2018

View reviewed changes

haudan mentioned this pull request Jun 18, 2018

Window transparency doesn't work on Windows 10 #260

Closed

WIP (broken) Windows blur implementation

ea119ff

haudan added 2 commits May 6, 2019 19:53

Working Windows implementation including example

bc53f54

Delete vim swap files

4eb989c

haudan commented May 6, 2019

View reviewed changes

Osspial changed the base branch from master to winit-legacy June 13, 2019 05:36

goddessfreya closed this Jul 26, 2019

goddessfreya mentioned this pull request Feb 1, 2020

Blur-Behind / Glassy Windows #538

Open

kchibisov mentioned this pull request Aug 20, 2020

Add option to blur background on macOS #1667

Draft

8 tasks

sravioli mentioned this pull request Oct 12, 2023

[request] Acrylic window blur neovide/neovide#2078

Open

Blur-Behind / Glassy Windows implementation #568

Blur-Behind / Glassy Windows implementation #568

Conversation

haudan commented Jun 15, 2018

Backend progress:

Choose a reason for hiding this comment

haudan commented Jun 15, 2018

francesca64 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elinorbgr commented Jun 16, 2018

haudan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

haudan Jun 17, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

OvermindDL1 commented Jun 18, 2018

elinorbgr commented Jun 18, 2018

OvermindDL1 commented Jun 18, 2018

OvermindDL1 commented Jun 18, 2018

elinorbgr commented Jun 18, 2018

OvermindDL1 commented Jun 18, 2018 • edited

OvermindDL1 commented Jun 18, 2018

OvermindDL1 commented Jun 18, 2018

elinorbgr commented Jun 18, 2018

OvermindDL1 commented Jun 18, 2018

elinorbgr commented Jun 18, 2018

OvermindDL1 commented Jun 18, 2018

haudan commented Jun 18, 2018 • edited

OvermindDL1 commented Jun 18, 2018 • edited

haudan commented Jun 18, 2018

francesca64 commented Jun 18, 2018

elinorbgr commented Jun 19, 2018

sp3d commented Jun 20, 2018

fschutt commented Nov 11, 2018

friktor commented Jan 28, 2019

haudan commented Jan 28, 2019

friktor commented Jan 28, 2019

jansol commented Mar 24, 2019

Osspial commented Apr 24, 2019

Choose a reason for hiding this comment

haudan commented May 6, 2019

goddessfreya commented May 7, 2019

goddessfreya commented Jul 26, 2019

haudan Jun 17, 2018 •

edited

OvermindDL1 commented Jun 18, 2018 •

edited

haudan commented Jun 18, 2018 •

edited

OvermindDL1 commented Jun 18, 2018 •

edited