Window attributes not updated after window move (on windows 10)

[codecat555]

Talon version v0.2.0-357-g3b3e (357)
knausj_talon @14a370f18f248ba19177a2543c49335b415a1464

I've noticed that the 'snap next' command is no longer working as expected on my windows 10 system (and 'snap last', and related commands).

This seems to be due to a failure to update the window attributes after moving the window.

For example, I used the repl to display the attributes of a notepad.exe window on my main screen (DISPLAY1). Then, I ran the 'snap next' command while the notepad window was focused. The notepad window did in fact move to the next screen (DISPLAY3), but the window attributes (including the window screen) reported by the repl were unchanged.

I recorded a video of this test, but it's too big to upload here (112 mb). I also have the talon log and I captured the event log.

[codecat555]

I forgot to copy my repl output yesterday, so I repeated the test this morning -

talon.log

Talon REPL | Python 3.9.10 (tags/v3.9.10:f2f3f53, Jan 17 2022, 15:14:21) [MSC v.1929 64 bit (AMD64)] on win32)
>>> import time
>>> time.sleep(3); w=ui.active_window()
>>> w
Window(app=App(pid=26068, "Notepad"), title="Untitled - Notepad")
>>> w.id
6229688
>>> w.rect
Rect(1604, 399, 455, 365)
>>> w.screen.name
'\\\\.\\DISPLAY1'
>>> w.screen
Screen(x=0.0, y=0.0, size=(2560.0, 1440.0), mm=(600.000, 340.000))
>>>
>>>
>>>
>>> ui.screens()
[Screen(x=0.0, y=0.0, size=(2560.0, 1440.0), mm=(600.000, 340.000)), Screen(x=-2560.0, y=-166.0, size=(2560.0, 1600.0), mm=(641.000, 401.000)), Screen(x=2560.0, y=227.0, size=(1920.0, 1200.0), mm=(641.000, 401.000))]
>>>
>>> for s in ui.screens():
...   print(f'{s.name}: {str(s)}')
...
\\.\DISPLAY1: Screen(x=0.0, y=0.0, size=(2560.0, 1440.0), mm=(600.000, 340.000))
\\.\DISPLAY2: Screen(x=-2560.0, y=-166.0, size=(2560.0, 1600.0), mm=(641.000, 401.000))
\\.\DISPLAY3: Screen(x=2560.0, y=227.0, size=(1920.0, 1200.0), mm=(641.000, 401.000))
>>>
>>>
>>>
>>> time.sleep(3); ui.active_window().id; mimic('snap next')
6229688
>>>
>>>
>>>
>>> w
Window(app=App(pid=26068, "Notepad"), title="Untitled - Notepad")
>>> w.id
6229688
>>> w.rect
Rect(1604, 399, 455, 365)
>>> w.screen.name
'\\\\.\\DISPLAY1'
>>> w.screen
Screen(x=0.0, y=0.0, size=(2560.0, 1440.0), mm=(600.000, 340.000))
>>>
>>>

[codecat555]

Note: in this test, the notepad.exe window started on my main display, named 'DISPLAY1'.

The 'snap next' command moved the window to the display named 'DISPLAY3', which is correct.

After, the window reference 'w' shown above did not show the updated values.

Then, I re-acquired the 'w' reference and it did show the new values -

>>>
>>> time.sleep(3); w=ui.active_window()
>>> w.screen.name
'\\\\.\\DISPLAY3'
>>> w.id
6229688
>>>                                                                  

I expected the reference to show the updated values without needing to be reset. That's the way it used to work.

[jcaw]

I've also encountered this, in the same context

[lunixbochs]

@codecat555 when you say "That's the way it used to work" do you have a timeframe?

git says I haven't really touched this code since December, and that was only changes to Window focus handling. I don't think I've messed with anything besides the focus code since much longer than that (maybe last June or earlier)

Oh, I just realized what you meant by reacquiring ui.active_window(). Can you show me these for your test, before setting w = ui.active_window() again?

id(w)
id(ui.active_window())
w == ui.active_window()
id(w.app)
id(ui.active_app())
id(w.app._windows[w.id])
id(ui.active_app()._windows[w.id])

[codecat555]

Yes, I was working intensively on my window move/resize code for a month or two during November/December of last year. Even into January...

During that time, I frequently used the repl in this way to troubleshoot my code.

To be clear, I originally started looking into this because the snap next/last commands were behaving oddly on my windows 10 system. I wasn't interested in the behavior of references, but just trying to figure out why my windows weren't behaving.

For instance, at one point yesterday I said 'snap next' and the current window jumped to the next screen and then immediately popped back to its previous location.That's not happening anymore, but perhaps it's related.

[lunixbochs]

I just added some values to grab in my last comment, they may help diagnose

[codecat555]

In your example, should this be assignment instead?

w == ui.active_window()

[codecat555]

>>> time.sleep(3); ui.active_window().id; mimic('snap next')
6229688
>>>
>>>
>>> id(w)
76808096
>>> time.sleep(3); id(w); id(ui.active_window()); w == ui.active_window(); w = ui.active_window()
76808096
76808096
True
>>> time.sleep(3); id(w.app); id(ui.active_app()); id(w.app._windows[w.id]); id(ui.active_app()._windows[w.id])
85748416
85748416
76808096
76808096
>>>

[codecat555]

I see in the output above that it looks like the window id changed from 6229688 to 76808096.

That seems odd, so I repeated the test. Interestingly, this time 'snap next' moved my window from DISPLAY1 to DISPLAY2 rather than to DISPLAY3.

Here's the output -

>>> time.sleep(3); w=ui.active_window(); id(ui.active_window());
76808096
>>> time.sleep(3); id(w.app); id(ui.active_app()); id(w.app._windows[w.id]); id(ui.active_app()._windows[w.id])
85748416
85748416
76808096
76808096
>>> time.sleep(3); ui.active_window().id; mimic('snap next')
6229688
>>> time.sleep(3); id(w); id(ui.active_window()); w == ui.active_window(); w = ui.active_window()
76808096
76808096
True
>>> time.sleep(3); id(w.app); id(ui.active_app()); id(w.app._windows[w.id]); id(ui.active_app()._windows[w.id])
85748416
85748416
76808096
76808096
>>>

[lunixbochs]

id(w) is the address of the object, that's different from w.id which is the window handle
I don't see anything in this output that suggests ui.active_window() would have fixed w.rect

Can you switch to a big print in this style so it's easier to read? And try to show the w.rect != ui.active_window().rect issue at the same time you print all the IDs.

print(f"{w.id=} {id(w.app)=}")

[jcaw]

I remember talking about an issue similar to this on Slack a long while ago - I think it was when I was first working on the knausj window snapping implementation.

When windows are manipulated programmatically with talon's API (on Windows), it seems the ui.Window.hidden attribute will sometimes start returning True, even if the window is still visible. I have a note in my window snapping code talking about this dated 2020/10/05, so that may be when the slack conversations happened.

I remember there were issues interfacing with the Windows API and you eventually triaged them as something to tackle another time. I don't have any other notes there, so I'm not sure if other attributes came up as failing to update in the discussion.

[lunixbochs]

hidden comes from win32gui.IsWindowVisible, which might just be wrong sometimes. Maybe I could fetch that one on demand instead of caching it.

[lunixbochs]

@codecat555

Then, I re-acquired the 'w' reference and it did show the new values -

In this example, you sleep(3) and presumably change window focus before grabbing ui.active_window() again, so the output doesn't rule out the sleep or window focus as the reason for the attribute change. I think you should set up one big print() like the template I showed you and run the entire print for each test, printing the attributes and IDs of both ui.active_window() and your saved w object. Maybe version your w, so you have w1, w2, w3 based on when you fetched the ui.active_window(). And you can check w1 is w2 to see if they're the same object or not.

[jcaw]

hidden comes from win32gui.IsWindowVisible, which might just be wrong sometimes. Maybe I could fetch that one on demand instead of caching it.

Alternatively, a bool param for force refresh?

[codecat555]

@lunixbochs

The problem magically fixed itself while I was working on reproducing the problem with the alternate formatting you requested. I didn't make any changes to my system or talon config, I didn't restart talon or the repl or use a different target window. All I did was switch between various windows and edit the test code. Very frustrating.

Here's the output, just to document the new test procedure -

>>> time.sleep(3); w=ui.active_window();
>>> time.sleep(3); print(f"{w.id=}\n{id(w)=}\n{id(ui.active_window())=}\n{id(w.app)=}\n{w == ui.active_window()=}\n{w.rect == ui.active_window().rect=}\n{w.screen.name=}")
w.id=6229688
id(w)=76808096
id(ui.active_window())=76808096
id(w.app)=85748416
w == ui.active_window()=True
w.rect == ui.active_window().rect=True
w.screen.name='\\\\.\\DISPLAY1'
>>>
>>>
>>> time.sleep(3); mimic('snap next')
>>>
>>>
>>> time.sleep(3); print(f"{w.id=}\n{id(w)=}\n{id(ui.active_window())=}\n{id(w.app)=}\n{w == ui.active_window()=}\n{w.rect == ui.active_window().rect=}\n{w.screen.name=}")
w.id=6229688
id(w)=76808096
id(ui.active_window())=76808096
id(w.app)=85748416
w == ui.active_window()=True
w.rect == ui.active_window().rect=True
w.screen.name='\\\\.\\DISPLAY3'
>>>