Support click-through of transparency

[tommoor]

As commented here, #949 (comment) - transparent windows unfortunately are of limited use unless you can click through the transparent area.

I noticed that in the documentation it is stated that this is blocked by an upstream bug. The nw.js project DOES have support for this, but i'm not sure how different their implementation is...

I think this is worth the issue to track discussion and implementation, so here it is.

[thomasjo]

While nw.js does support it, it comes with some pretty hefty drawbacks. First of all it requires disabling GPU acceleration, and secondly it only works on OS X and Windows (based on their documentation).

[CharlieHess]

+1, this one's plaguing Slack too, since we display all of our notifications in a transparent window.

[YourCyborg]

+1, this is critical for the project I'm working on.

[marcfowler]

+1, this would be excellent.

[geelen]

Interested in hearing the latest on this. For my purposes, I'd love if we could set a transparent window to have no click area. So you could have an always-on-top full screen overlay that didn't catch clicks.

[TimvdEijnden]

I'm interested too, i'm building an app with a semi transparent window that's on top of other applications but mouseevents need to get through to the other applications.

[yohaia]

+1

Any news about it?

[lukeapage]

the linked chromium bug is marked wont-fix
https://code.google.com/p/chromium/issues/detail?id=387234

the last comment is:

#11 NekR.Fab...@gmail.com
You know, just btw, people in Atom team can do it by themselves if their users need it, right?

so nothing is currently going to happen until someone on the atom team decides to do it or persuades the chrome team it is worth putting on their backlog (and they said it was low priority).

[molster]

Just ran into this today, It has put a -massive- issues into my plans of using a new window has an overlay. (in fact, it is now impossible)

I would love a fix for this!

[devployment]

Is there any work in progress for this? Will there be a chance for the Electron team to make this possible without being added to Chromium?

[stanier]

+1

Assuming this won't be resolved through the Chromium code, is there a plausible way to have the application listen for a click event and then trigger the proper counterpart in GTK or similar?

[kutu]

i use ahkscript

#c::
    WinSet, ExStyle, ^0x20, A
    WinSet, Transparent, 255, A
    return

press Win+C on active window, and its click-through now

[xeoneux]

@zcbenz Any update on this? 😄

[zcbenz]

To people watching this issue: if you simply want to make the whole window click-though, you can use the new win.setIgnoreMouseEvents API introduced in v1.2.2.

[zcbenz]

Also on making specified regions transparent, I have come up with some ideas on the implementation (but do not assume it would be done in near future):

For Linux, it is actually very easy by using the XShapeCombineRectangles API, the implementation of SetIgnoreMouseEvents is just setting only a region of (0, 0, 1, 1) to accept mouse events.

For Windows, it is easy to set the shape of window, but there is no API to control which regions can accept mouse events.

For OS X, either setting window shape or setting click-though regions is impossible if you don't have control to the rendering code.

And since Chromium does complex hardware acceleration, it becomes super hard to implement custom click-through regions. NW.js does it by disabling hardware acceleration, and patching the rendering code, this is an approach that I definitely don't want to take.

My idea for the implementation on OS X/Windows is, we just make the whole window click-through, and then create a transparent window as child window, and use it to capture all mouse events. Since we have full control of the child window, we can set it to the shape of the regions that users do not want to be click-though, and then send all mouse and keyboard events received in the child window to the parent window.

In this way we don't have to patch Chromium, so it is easy to maintain and hardware acceleration works. The downside is we have to very carefully redirect mouse and keyboard events, to make the window behave as if it there is no child window.

[StickyCube]

If, like me, your use case for this feature is fairly simple this gist might be of interest. It is a basic implementation of what @zcbenz is describing, creating one clickable area which redirects mouse events to the parent.

[ButzYung]

During the development of my Windows gadget project which requires click-thru, I have unintentionally come up with a somewhat fancy but practical solution which works to define a clickcable area of truly arbitrary shape.

In the renderer process, use "screen.getCursorScreenPoint()" to keep tracking the cursor position, and then send the coordinates to the main process regularly. On the main process, use "webContents.capturePage()" to capture the image of that exact 1x1 pixel on the renderer window. Since it's just 1 pixel, the performance cost is ignorable, and then you can define whether that pixel is clickable or not by looking at the opacity value. This way, you can practically define a truly pixel-correct clickable area. The only issue left here is a function to detect the mousedown state regardless of the cursor position.

[amhoho]

Electron click through transparency example
https://gist.github.com/StickyCube/ed79421bc53cba38f5b74b060d3f15fa

[varavut]

@ButzYung Could you please tell me that how can I get opacity value from NativeImage (webContents.capturePage() return NativeImage)? Thanks.

[ButzYung]

@varavut Capturing a 1x1 image as an example, it would be something like this.

webContents.capturePage({x:0,y:0,width:1,height:1}, function(image) {
  var buffer = image.getBitmap()

  // send the data back to renderer for processing, with buffer[3] being the alpha channel
  webContents.send('capturePage', 'RESULT|' + [buffer[0],buffer[1],buffer[2],buffer[3]])
});

[cecilpeng]

@ButzYung Thanks for your solution. I tried it, the key is the interval of tracking. 100ms for my computer, looks well. If I used webContents.capturePage, CPU cost high. I would define the area which is click-through, just check the point if in the area.

[ButzYung]

@cecilpeng In my case I give a 200ms delay after the current webContents.capturePage has finished, instead of doing another capture immediately. That's roughly 4-5 captures/second, enough for the purpose of checking transparency, and the CPU usage is ignorable.

[cecilpeng]

@ButzYung yes, it is better to do another capture after last has finished. I'm still worried about the timeout solution, it's not smart enough. We can't know what happens on the users' computer. I recommend that javascript is a good languague to observe instead of inspect.

[sancarn]

I'm very much a HTML noob but is it not possible to simply:

1. setup a 'onMouseLeave' event on HTML elements which you want to be clickable.
   --> this mouse leave event executes win.setIgnoreEvents(true)
2. setup a 'onMouseEnter' event on HTML elements which you want to be clickable.
   --> this mouse enter event executes win.setIgnoreEvents(false)

Due to my lack of HTML knowledge there may be an issue with this in certain scenarios... But in my head it seems to be the, currently, ideal workaround. Except for the extra setup time required of course?

[DDinenno]

@nikkwong Your example was the only one that mostly worked.

The problem is that, if you rest you mouse for more than the timeout amount, you are unable to click through.

I made another example (See below), which is a modified version of yours.
What this achieves, is it makes sure that the element is not the document because setting ignore to false.
It's done by adding a interval which inside will get the current screen position and using document.elementFromPoint, it will grab the top most element, if there's no elements it will return the documentElement.

main.js

ipcMain.handle("get-screen-point", () => screen.getCursorScreenPoint());

preload.js added a bridge function

 getScreenPoint: () => ipcRenderer.invoke("get-screen-point"),

handle mouse move

 const handleMouseMove = (e) => {
      if (e.target === overlayWindow.document.documentElement) {
        window.api.menuOverlaySetIgnoreMouseEvents(true);

        if (t) clearInterval(t);
        t = setInterval(async () => {
          const point = window.api.getScreenPoint();

          const elem = overlayWindow.document.elementFromPoint(
            Math.floor(point.x),
            Math.floor(point.y)
          );

          if (elem !== overlayWindow.document.documentElement) {
            window.api.menuOverlaySetIgnoreMouseEvents(false);
            clearInterval(t);
          }
        }, 150);
      } else {
        window.api.menuOverlaySetIgnoreMouseEvents(false);
      }
    };

[whosnorman]

I was able to use @LauraAubin's approach with a couple tweaks!

• Instead of using mainWindow.hide() I use Menu.sendActionToFirstResponder('hide:'). I found it here cause when I was trying to CMD+Tab back to the app there was no event firing to be able to use mainWindow.show(). This approach also gives up control to the application you click on below your app, so no need for double clicking. You can get the Menu variable via const { ..., ipcMain, Menu } = require('electron');

• Also, instead of adding an id to all elements that should be clicked, I just listen for if the event target is the document body e.target == document.body, and put the rest of my app in an element that has a mask and its overflow is hidden.

[aarowman]

Any updates on this?

I'd also be interested to see the hover behavior go through transparency (in addition to clicks).

My use case is to overlay a game, and the only way the click through would work is by disabling GPU compositing or acceleration, but the click/hover goes through everything - even non-transparent elements.

[gdbroman]

I'd also be interested to see the hover behavior go through transparency (in addition to clicks).

+1 on adding functionality for propagating hover!

My use case is rendering a custom animated cursor in a transparent overlaid window, and the cursor has different states depending on what element it is hovering.

[toonvanvr]

I think that if you combine setIgnoreMouseEvents(true, {forward: true}) with pointer-events: none and a body class for example click-through, you can define your css as such:

:not(.click-through) .my-elem:hover {
  background-color: pink;
}

I had an npm lib for it, but it's outdated and won't get an update soon. Someone got my v2 draft version working recently though. The trick it uses is that if your pointer event target (and every element below) has pointer-events: none, it would emit an event with the <html> element as event.target. The pointerenter/over/leave/.. event toggled setIgnoreMouseEvents(target == <html> element, {forward: true}). The bugs it caused were OS version and electron version dependent around the time Electron 6 was current, making it hardly worth it to maintain the lib, but the setIgnoreMouseEvents implementation may be more robust now.

[LZQCN]

During the development of my Windows gadget project which requires click-thru, I have unintentionally come up with a somewhat fancy but practical solution which works to define a clickcable area of truly arbitrary shape.

In the renderer process, use "screen.getCursorScreenPoint()" to keep tracking the cursor position, and then send the coordinates to the main process regularly. On the main process, use "webContents.capturePage()" to capture the image of that exact 1x1 pixel on the renderer window. Since it's just 1 pixel, the performance cost is ignorable, and then you can define whether that pixel is clickable or not by looking at the opacity value. This way, you can practically define a truly pixel-correct clickable area. The only issue left here is a function to detect the mousedown state regardless of the cursor position.

We always stand on the shoulders of giants.
Combining the ideas of geniuses, the following code achieves perfect results.

  setInterval(() => {
    const point = screen.getCursorScreenPoint();
    const [x, y] = win.getPosition();
    const [w, h] = win.getSize();

    if (point.x > x && point.x < x + w && point.y > y && point.y < y + h) {
      updateIgnoreMouseEvents(point.x - x, point.y - y);
    }
  }, 300);

  const updateIgnoreMouseEvents = async (x, y) => {
    console.log("updateIgnoreMouseEvents");

    // capture 1x1 image of mouse position.
    const image = await win.webContents.capturePage({
      x,
      y,
      width: 1,
      height: 1,
    });

    var buffer = image.getBitmap();

    // set ignore mouse events by alpha.
    win.setIgnoreMouseEvents(!buffer[3]);
    console.log("setIgnoreMouseEvents", !buffer[3]);
  };

[raphael10-collab]

@LZQCN I tried to follow your suggestion here: #1335 (comment) but , but I guess I'm making something wrong because nothing happens:

SearchWindow is in my case the main window .

declare const SEARCH_WINDOW_PRELOAD_WEBPACK_ENTRY: string;

let SearchWindow: BrowserWindow

let child_window: BrowserWindow
declare const CHILD_WINDOW_WEBPACK_ENTRY: string;
declare const CHILD_WINDOW_PRELOAD_WEBPACK_ENTRY: string;

const createSearchWindow = (): void => {
console.log("createSearchWindow called")

SearchWindow = new BrowserWindow({
    fullscreen: false,
    height: 900,
    width: 1200,
    webPreferences: {
        preload: SEARCH_WINDOW_PRELOAD_WEBPACK_ENTRY,
        nodeIntegration: false,
        contextIsolation: true,
        nodeIntegrationInWorker: false,
        nodeIntegrationInSubFrames: false,
        webviewTag: true,
        webSecurity: true
    }
});

SearchWindow.loadURL(SEARCH_WINDOW_WEBPACK_ENTRY)

// Open the DevTools
if (isDev) {
    SearchWindow.webContents.openDevTools()
}
SearchWindow.webContents.on("did-navigate", async (event, url) => {
    SearchWindow.webContents.session.cookies.get({name:"_session_id"})
})

let searchWindowHeight: number = SearchWindow.getBounds().height
let searchWindowWidth: number = SearchWindow.getBounds().width
let searchWindow_x: number = SearchWindow.getBounds().x
let searchWindow_y: number = SearchWindow.getBounds().y

child_window = new BrowserWindow({
    //height: 900,
    //width: 1200,
    height: searchWindowHeight,
    width: searchWindowWidth,
    x: searchWindow_x,
    y: searchWindow_y,
    maximizable: false,
    minimizable: false,
    parent: SearchWindow,
    transparent: true,
    skipTaskbar: true,
    movable: false,
    resizable: false,
    alwaysOnTop: true,
    webPreferences: {
        nodeIntegration: false,
        nodeIntegrationInSubFrames: true,
        scrollBounce: true,
        safeDialogs: true,
        safeDialogsMessage: 'Prevent this page from creating additional dialogs',
        preload: CHILD_WINDOW_PRELOAD_WEBPACK_ENTRY,
        contextIsolation: true,
        sandbox: true,
        enableWebSQL: false,
        // match Chrome's default for anti-fingerprinting purposes (Electron defaults to 0)
        minimumFontSize: 6
    }
})

child_window.once('ready-to-show', () => {
    child_window.show()
})

child_window.setFocusable(false)

child_window.setIgnoreMouseEvents(true,{
    forward: true ,
})

setInterval(() => {
    const point = screen.getCursorScreenPoint()
    const [x, y] = SearchWindow.getPosition()
    const [w, h] = SearchWindow.getSize()

    if (point.x > x && point.x < x + w && point.y > y && point.y < y + h) {
        updateIgnoreMouseEvents(point.x - x, point.y - y)
    }
}, 300)

const updateIgnoreMouseEvents = async (x, y) => {
    console.log("updateIgnoreMouseEvents")

    // capture 1x1 image of mouse position.
    const image = await SearchWindow.webContents.capturePage({
        x,
        y,
        width: 1,
        height: 1,
    })

    let buffer = image.getBitmap()

    // set ignore mouse events by alpha.
    SearchWindow.setIgnoreMouseEvents(!buffer[3])
    console.log("setIgnoreMouseEvents", !buffer[3])
}

How to make it work ?

[LZQCN]

@raphael10-collab

I found that in your code SearchWindow does not seem to be a transparent window, but child_window is a transparent window.

[raphael10-collab]

Yes. the transparent window is child_window

I've put in your code the right window :

setInterval(() => {
    const point = screen.getCursorScreenPoint()
    const [x, y] = child_window.getPosition()
    const [w, h] = child_window.getSize()

    if (point.x > x && point.x < x + w && point.y > y && point.y < y + h) {
        updateIgnoreMouseEvents(point.x - x, point.y - y)
    }
}, 300)

const updateIgnoreMouseEvents = async (x, y) => {
    console.log("updateIgnoreMouseEvents")

    // capture 1x1 image of mouse position.
    const image = await child_window.webContents.capturePage({
        x,
        y,
        width: 1,
        height: 1,
    })

    let buffer = image.getBitmap()

    // set ignore mouse events by alpha.
    child_window.setIgnoreMouseEvents(!buffer[3])
    console.log("setIgnoreMouseEvents", !buffer[3])
}

Thechild_window (the transparent one) is on top of the SearchWindow.
Clicking on the child_window, the one which is on top, does not pass the click event to the underneath window

and I'm getting this error :

The error message points to this line:

setInterval(() => {
    const point = electron_1.screen.getCursorScreenPoint();
    const [x, y] = child_window.getPosition();    // <----------------------------------------------------------------------------
    const [w, h] = child_window.getSize();
    if (point.x > x && point.x < x + w && point.y > y && point.y < y + h) {
        updateIgnoreMouseEvents(point.x - x, point.y - y);
    }
}, 300);

[LZQCN]

@raphael10-collab

const timer = setInterval(() => {
    const point = electron_1.screen.getCursorScreenPoint();
    const [x, y] = child_window.getPosition();    // <----------------------------------------------------------------------------
    const [w, h] = child_window.getSize();
    if (point.x > x && point.x < x + w && point.y > y && point.y < y + h) {
        updateIgnoreMouseEvents(point.x - x, point.y - y);
    }
}, 300);

child_window.on("close", () => {
    clearInterval(timer);
});

[raphael10-collab]

@LZQCN With clearInterval(timer) the Timeout Error disappeared

This is the output

The transparent window, the child_window, does not allow me to click the button Click me built in the underneath window (SearchWindow): how to "click-through" the trasparent top window, in order to be able to click the button in the underneath window? Is that possible?

[raphael10-collab]

Following the indications found here: https://stackoverflow.com/questions/53357428/how-to-make-mouse-pass-through-window-in-electron I've put focusable: false for the child_window (the `transparent window) :

child_window = new BrowserWindow({
    //height: 900,
    //width: 1200,
    height: searchWindowHeight,
    width: searchWindowWidth,
    x: searchWindow_x,
    y: searchWindow_y,
    maximizable: false,
    minimizable: false,
    parent: SearchWindow,
    transparent: true,
    skipTaskbar: true,
    movable: false,
    resizable: false,
    focusable: false, // https://stackoverflow.com/questions/53357428/how-to-make-mouse-pass-through-window-in-electron
    frame: false,
    alwaysOnTop: true,
    webPreferences: {
        nodeIntegration: false,
        nodeIntegrationInSubFrames: true,
        scrollBounce: true,
        safeDialogs: true,
        safeDialogsMessage: 'Prevent this page from creating additional dialogs',
        preload: CHILD_WINDOW_PRELOAD_WEBPACK_ENTRY,
        contextIsolation: true,
        sandbox: true,
        enableWebSQL: false,
        // match Chrome's default for anti-fingerprinting purposes (Electron defaults to 0)
        minimumFontSize: 6
    }
})

Now the top transparent window is not visible anymore, the button Click me in the underneath window is clickable and the screen.getCursorScreenPoint() yields its output

But, still, I'm not able to get the correct mouse position with getCursorScreenPoint().... I mean... what I would like to do is to get, in the transparent window, child_window, the position {x, y} of the mouse when clicking on the underneath window

[LZQCN]

@raphael10-collab My code works for this scenario:

[raphael10-collab]

Wow ...... I do not understand how you are actually using the screen.getCursorScreenPoint 's ouput:
How are you passing the cursor position from the transparent window to the underneath window , and viceversa, from the underneath window to the transparent window? Through electron.ipcSend ?

[LZQCN]

@raphael10-collab
If use my code, clicking on the transparent area of the window no longer generates any mouse events for the window. To be precise, the transparent window will be set to ignore any mouse events when the mouse is over the transparent area. only the content below the transparent window will be clicked.

Also, if you want to get the clicked position, you should get the clicked position from the window that was actually clicked.

[raphael10-collab]

@LZQCN I've set :

child_window.setFocusable(false)

child_window.setIgnoreMouseEvents(true,{
    forward: true ,
})

According to what we read here: https://www.electronjs.org/docs/latest/api/browser-window#winsetignoremouseeventsignore-options :

win.setIgnoreMouseEvents(ignore[, options])

    ignore boolean
    options Object (optional)
        forward boolean (optional) macOS Windows - If true, forwards mouse move messages to Chromium, enabling mouse related events such as mouseleave. Only used when ignore is true. If ignore is false, forwarding is always disabled regardless of this value.

Makes the window ignore all mouse events.

All mouse events happened in this window will be passed to the window below this window, but if this window has focus, it will still receive keyboard events

should be fine....

Is this not enough for Linux ?
Why x and y do not change, based on the mouse events? :

[raphael10-collab]

I do not understand why this message sent from the child_window (the transparent window on top of the other window) is not received in the main process:

child_window renderer :

React.useEffect(() => {
window.api.electronIpcSend("hello-from-app-child", "HELLOOOOOOOOO")

}, [])

in main :

ipcMain.on("hello-from-app-child", (event, args) => {
    console.log("IpcMain.on-hello-from-app-child-args: ", args)
})

This is how I defined and created the bottom window (SearchWindow) and the transparent window on top :

declare const SEARCH_WINDOW_WEBPACK_ENTRY: string;
declare const SEARCH_WINDOW_PRELOAD_WEBPACK_ENTRY: string;

let SearchWindow: BrowserWindow

let child_window: BrowserWindow
declare const CHILD_WINDOW_WEBPACK_ENTRY: string;
declare const CHILD_WINDOW_PRELOAD_WEBPACK_ENTRY: string;

const createSearchWindow = (): void => {
    console.log("createSearchWindow called")

SearchWindow = new BrowserWindow({
    fullscreenable: false,
    height: 900,
    width: 1200,
    kiosk: true,
    webPreferences: {
        preload: SEARCH_WINDOW_PRELOAD_WEBPACK_ENTRY,
        nodeIntegration: false,
        contextIsolation: true,
        nodeIntegrationInWorker: false,
        nodeIntegrationInSubFrames: false,
        webviewTag: true,
        webSecurity: true
    }

    let searchWindowHeight: number = SearchWindow.getBounds().height
    let searchWindowWidth: number = SearchWindow.getBounds().width
    let searchWindow_x: number = SearchWindow.getBounds().x
    let searchWindow_y: number = SearchWindow.getBounds().y

    child_window = new BrowserWindow({
        height: searchWindowHeight,
        width: searchWindowWidth,
        fullscreenable: false,
        x: searchWindow_x,
        y: searchWindow_y,
        maximizable: false,
        minimizable: false,
        parent: SearchWindow,
        transparent: true,
        skipTaskbar: true,
        movable: false,
        resizable: false,
        //focusable: false, 
        frame: false,
        alwaysOnTop: true,
        webPreferences: {
            nodeIntegration: false,
            nodeIntegrationInSubFrames: true,
            scrollBounce: true,
            safeDialogs: true,
            safeDialogsMessage: 'Prevent this page from creating additional dialogs',
            preload: CHILD_WINDOW_PRELOAD_WEBPACK_ENTRY,
            contextIsolation: true,
            sandbox: true,
            enableWebSQL: false,
            // match Chrome's default for anti-fingerprinting purposes (Electron defaults to 0)
            minimumFontSize: 6
        }
    })

    child_window.once('ready-to-show', () => {
        child_window.show()
    })

    child_window.setIgnoreMouseEvents(true,{
        forward: true ,
    })

}

[foxxy432]

It is ignored by Microsoft because of incompatibilities BUT you can use my fork: https://github.com/lnxx-kernel/electron_touch

[frankz61]

I've noticed that the Electron API setShape can solve some problems, similar to the Windows setWindowRgn. It allows areas without a set shape to be non-drawable and lets mouse events pass through.

This solved my problem. If your page's displayed content is not complex, you can use this API.

[Nebula-Developer]

win.SetShape is currently only supported (experimentally) on Windows and Linux. Are there any efforts towards implementing this on macOS? (and removing it from its 'experimental' state)