Skip to content

feat(systray): added systray explorer.exe hook mode#802

Merged
amnweb merged 1 commit intoamnweb:mainfrom
Video-Nomad:feat/systray-explorer-hook
Mar 21, 2026
Merged

feat(systray): added systray explorer.exe hook mode#802
amnweb merged 1 commit intoamnweb:mainfrom
Video-Nomad:feat/systray-explorer-hook

Conversation

@Video-Nomad
Copy link
Contributor

With this PR the default mode of icon interception for systray will be hook mode (use_hook: true)

The old method (monitor)

The old method was to create a hidden win32 window with class name Shell_TrayWnd and keep it topmost on a timer to intercept all the messages coming from shell32 to explorer.exe process. After processing, those messages were forwarded to the original taskbar.
This caused plenty of compatibility issues as some apps were discovering YASB systray monitor instead of the native taskbar window when using FindWindow function.

   ┌───────┐                          
   │Shell32│                          
   └───┬───┘                          
       │                              
Taskbar Messages                      
       │                              
┌──────▼───────┐                      
│ YASB Monitor ┼──────────┐           
└──────┬───────┘          │           
       │            Taskbar Messages  
   Icon Data              │           
       │       ┌──────────▼──────────┐
  ┌────▼────┐  │ explorer.exe process│
  │  YASB   │  │ (original Taskbar)  │
  └─────────┘  └─────────────────────┘                                

The new method (hook)

The new hook method takes a completely different approach. It injects a small dll into explorer.exe process and inserts itself in the message loop intercepting WM_COPYDATA messages and sending all the relevant data via named pipe to YASB. After the copy of the message is sent via pipe, the original message is safely forwarded back to the explorer machinery.

Because hIcon (and other handles) is only guaranteed to be valid before it's consumed by the original process, all the icon processing is performed on the DLL side and already processed icon bytes are sent via pipe directly. Without that there will be race conditions where hIcon is invalidated before python has a chance to process it.

     ┌───────┐                       
     │Shell32│                       
     └───┬───┘                       
         │                           
  Taskbar│Messages                   
         │                           
 ┌───────┼──────────────explorer.exe┐
 │ ┌─────▼───────┐ ┌──────────────┐ │
 │ │ Injected    │ │Original Proc │ │
 │ │ DLL         ┼─►              │ │
 │ └──┬───────▲──┘ └──────────────┘ │
 └────┼───────┼─────────────────────┘
      │       │                      
  Pipe│       │Inject                
     ┌▼───────┼┐                     
     │  YASB   │                     
     └─────────┘                     

Failure handling

  • On explorer crash (or restart) the DLL will be automatically re-injected from YASB.
  • If dll is missing for some reason (AV deleted it) the original monitor method will be used automatically.
  • In case of YASB closing or crashing the DLL will be released automatically via named mutex release.

How to compile

CMake and Visual Studio (MSBuild) should be installed.
Running build_dll.bat should place the binary directly where it should be for everything to work.

Alternatively, Ninja+Clang can be used:

cmake -G Ninja -B build
cmake --build build --config Release --clean-first

Changelog

  • added systray explorer.exe hook mode and updated the docs
  • will inject a dll into explorer.exe process to monitor icon changes
  • the hook will automatically detach when YASB is closed (named mutex)
  • added systray_hook.py to handle injection and data processing
  • added hook folder to store the dll and source code
  • added cmake config to build the dll
  • compiled dll is included in the repo
  • some small systray changes

@Video-Nomad Video-Nomad marked this pull request as draft March 19, 2026 16:29
@Video-Nomad Video-Nomad force-pushed the feat/systray-explorer-hook branch 3 times, most recently from 71d7f49 to f0d2d72 Compare March 20, 2026 08:50
@Video-Nomad Video-Nomad marked this pull request as ready for review March 20, 2026 08:51
@Video-Nomad Video-Nomad force-pushed the feat/systray-explorer-hook branch 5 times, most recently from 925868c to 96b4133 Compare March 21, 2026 19:45
- added systray `explorer.exe` hook mode and updated the docs
- will inject a dll into explorer.exe process to monitor icon changes
- the hook will automatically detach when yasb is closed (named mutex)
- added `systray_hook.py` to handle injection and data processing
- added `hook` folder to store the dll and source code
- added cmake config to build the dll for both x64 and arm64
- modified build.py to copy the correct dll to the lib folder
- compiled dlls (x64 and arm64) are included in the repo
- dll cross injection is not supported and will revert to legacy mode
- some systray refactoring
@Video-Nomad Video-Nomad force-pushed the feat/systray-explorer-hook branch from 96b4133 to f0d8e5f Compare March 21, 2026 21:07
@amnweb amnweb merged commit 25d9bad into amnweb:main Mar 21, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants