Basic accessibility support for the terminal view

[pvagner]

Terminal view as implemented in the termux app is a view resembling terminal windows known from computers. It's designed to be efficient however for me and other people who are using accessibility services such as Google Talkback to access their android devices it's not very usable. By adding contentDescription property to the view dynamically as it repaints, we can get at least very basic accessibility support.
When using termux on Android with Talkback running it is possible to review currently visible text on the screen by characters, by words and by lines.
It is possible to use two finger scrolling (android built-in feature) to scroll the view and access other parts of the text.
I'm using this for a few months already and I have got some friends interested in this functionality so I'm submitting this as a pull request.
I know it's very minimalistic, but even in its current limited form it's a life changing feature for tech savvy blind people so hopefully you can accept it and / or point out how I might be able to enhance it so everyone can benefit.

[fornwall]

Thanks a lot!

Is it necessary to call setContentDescription(getText()) on each render call, or would it be enough to only call it when updating the screen in TerminalView.onScreenUpdated()?

[pvagner]

Hello,
I just wanted to make sure we will get all the updates.
Now when thinking more about it I assume
TerminalView.onScreenUpdated()
would be enough.
I'll change and test it today.

[pvagner]

Changed, tested and working fine.

[pvagner]

@fornwall Please can you take a look one more time?
I hope it's changed as you have suggested.

[stormdragon2976]

Did this ever get added to Termux? I installed it today and wasn't able to read the terminal's output. Is there something I need to do to get it working with Talkback?

[pvagner]

This PR is still open thus not merged to the code base.
It still merges cleanly so your only option in order to try this out is to rebuild termux on your own either using Android studio or gradle directly cherrypicking this patch.

[stormdragon2976]

I'm having lots of issues with the jre and android-studio playing nicely together. Is there an apk somewhere that I can just install until this is merged?

[Grimler91]

@stormdragon2976 you can try this one: https://grimler.se/apps/termux-app-access.apk
I have the other apps available from there as well, signed with the same key.

I haven't tested it at all so it's great if you are willing to!

[stormdragon2976]

Thanks so much for this. It is very helpful. The accessibility is very basic indeed, but it does work. Definitely a good start.

[fornwall]

@pvagner I'm really sorry for forgetting about this one, and I would really like to see it included!

Could you (or anyone else interested) have a look at enabling this only if accessibility services are enabled (or something similar), as this may slow down the app needlessly if not?

Perhaps with something like this in the TerminalView constructor

private boolean mAccessibilityEnabled;
[...]
public TerminalView(Context context, AttributeSet attributes) {
[...]
    AccessibilityManager am = (AccessibilityManager) context.getSystemService(ACCESSIBILITY_SERVICE);
    mAccessibilityEnabled = am.isEnabled();

and then guarding the call with if (mAccessibilityEnabled) setContentDescription(getText());. Would this work?

[pvagner]

I'll be able to use and integrate this code snipped in a week or so. I'm sure something like this is doable.

[fornwall]

Thanks a lot!

It's beyond the scope of this immediate change, but could you give some pointers about possible next steps to improve accessibility support?

[pvagner]

@fornwall I apologize for another delay. I have tested your suggestion on my devices and it's working fine over here. Hopefully I've got it right.

I can imagine a few more improvements to the TerminalView accessibility support however I am not sure I am able to implement them.

• It would be nice to sent out AccessibilityEvent.TYPE_VIEW_SCROLLED when scrolling the view so we know how much content there is displayed in the view.
• Currently we are exposing the whole view similar to a text view. We set the text retrieved from the visible content as an accessible description and we are using screen reader features to navigate over that. We might consider implementing our own AccessibilityNodeInfo representing each line seperately. That would enable screen reader users to use their fingers to explore individual lines on the screen. This might then be further enhanced by implementing custom accessibility actions for example allowing screen reader users to select and copy parts of the text from the TerminalView.
• There is one major disadvantage with the current approach I have taken in regard to TerminalView accessibility and I am unable to come up with a better plan. However we need to be able to somehow expose the caret control where it makes sense. We can type, but screen readers can't detect where the caret is pointing to. For simple commands it's all right but for more complex commands and when using text editors, ncurses apps and similar pseudo UI equivalent interfaces accessibility support with the caret handling is not very usefull.

Perhaps there is more however these are things I can imagine being either critical for the comfortable usage of the app or easy or moderate to implement.

[fornwall]

Thanks a lot, both for this change and the suggestions for further improvements!

[vortex1024]

any idea about what would be required to make new text be read automatically in the terminal? I guess using an accessibility live region for the whole terminal view won't work, as the whole terminal would be read out, not only the new text. A hidden accessibility region containing only the new text would work, but sounds like an ugly hack. Other ideas? Maybe something linked to the accessibility node info @pvagner talked about?