Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows Terminal: Set up UIA tree and hook it up properly #634

Closed
ZoeyR opened this issue May 9, 2019 · 9 comments

Comments

@ZoeyR
Copy link
Contributor

commented May 9, 2019

  • Your Windows build number: (Type ver at a Windows Command Prompt)
    Microsoft Windows [Version 10.0.18362.53]

  • What you're doing and what's happening: (Copy & paste specific commands and their output, or include screen shots)

I am launching the Windows Terminal with the NVDA screen reader active. I then type the dir command.

  • What's wrong / what should be happening instead:

NVDA does not read out the contents of the dir command window. In traditional conhost, NVDA and other screen readers are able to read out the terminal content.

@zadjii-msft

This comment has been minimized.

Copy link
Member

commented May 9, 2019

This is correct - I don't think we have the UIA tree hooked up quite yet. I think @philnach is working on this though!

@miniksa

This comment has been minimized.

Copy link
Member

commented May 9, 2019

Phil was working on the TSF. This is UIA. We know we need to hook this up and get it going, but we haven't got to it yet.

We're pretty sure that the XAML Islands portions of the UI should provide some level of UIA pattern implementation already.

As for the TerminalControl itself, we imagine that we can crib the implementation from conhost.exe that is currently hanging out around interactivity/win32 in the code base starting from UiaTextRange.cpp and stitch that into the rest of the UIA tree with the XAML Islands patterns to get this satisfied.

@carlos-zamora

This comment has been minimized.

Copy link
Member

commented Jun 20, 2019

So, I'm changing this task to just be setting up the UIA tree on Windows Terminal and hooking it up properly. Check out the Area-Accessibility tag to see the other tasks since they all kinda feed into each other to get this exact scenario to work.

@carlos-zamora carlos-zamora changed the title New windows terminal is not accesible to screen readers Windows Terminal: Set up UIA tree and hook it up properly Jun 20, 2019

@carlos-zamora carlos-zamora self-assigned this Jun 20, 2019

@DHowett-MSFT DHowett-MSFT pinned this issue Jun 22, 2019

@josephsl

This comment has been minimized.

Copy link

commented Jun 22, 2019

Hi,

I'm one of the code contributors to NVDA screen reader project. This issue also affects Narrator as well.

If you need any assistance from NV Access and other NVDA community people, please ping us. Thanks.

CC @michaelDCurran, @jcsteh, @feerrenrut, @derekriemer, @codeofdusk

@carlos-zamora carlos-zamora referenced this issue Jun 28, 2019
2 of 9 tasks complete

@carlos-zamora carlos-zamora added the In-PR label Jul 12, 2019

@miniksa miniksa unpinned this issue Jul 31, 2019

@msftbot msftbot bot removed the Needs-Tag-Fix label Jul 31, 2019

@ZoeyR

This comment has been minimized.

Copy link
Contributor Author

commented Aug 2, 2019

@carlos-zamora I built the most recent branch of master and I can't seem to find any of the contents of the terminal control in the UIA tree. Should this bug be reopened?

@carlos-zamora

This comment has been minimized.

Copy link
Member

commented Aug 2, 2019

@carlos-zamora I built the most recent branch of master and I can't seem to find any of the contents of the terminal control in the UIA tree. Should this bug be reopened?

I'm just going to spew information out at you, so I hope I don't come off as rude.

Just rebuilt master. The TermControl is exposed as a TextProvider. So, using Inspect.exe, I've seen the data through the TextPattern Explorer.

I've been able to verify that you can access the text through Narrator too. The navigation is definitely wonky though.

Here's a copy of the UIA tree through Inspect:
image

Hopefully, what you're seeing is an issue with navigation. Let me know if that's not helping though.

@ZoeyR

This comment has been minimized.

Copy link
Contributor Author

commented Aug 2, 2019

Ah, if I navigate into line mode then I can get some of the text out of it, and this may be my fault, but when I execute the dir command and then try to line navigate through the output I can't seem to actually read the results.

Also, is there a plan to get live text reading working again? That was what threw me off originally.

@carlos-zamora

This comment has been minimized.

Copy link
Member

commented Aug 2, 2019

Ah, the live text reading is going to get hooked up after the navigation gets fixed up a bit. At least right now, the text from the buffer is accessible. But I need to hook up an event that handles new text appearing and directly passes that to the screen reader. This is definitely worth creating a new issue, for tracking. Mind creating one for me?

@msftbot

This comment has been minimized.

Copy link

commented Aug 3, 2019

🎉This issue was addressed in #1691, which has now been successfully released as Windows Terminal Preview v0.3.2142.0.🎉

Handy links:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.