Investigate focus-based web UIs, e.g., for TVs and Game Consoles

[necolas]

Reading material:

• Implementing TV remote control navigation
• Pass the Remote: User Input on TV Devices
• React TV
• React Native: building for TV devices

[dcalhoun]

I (and my employer) am very much interested in shipping products to focus-based platforms using react-native/react-native-web. If you come across work/explorations/investigations that you feel like you can delegate, let me know. I'll see if I can find company time or free time (more likely) to help.

[necolas]

PlayStation have done some work in this space, and reached out about this functionality. Hopefully they will share their focus algorithm at some point. /cc @matthargett

[azogheb]

The DirectionalNavigation library from Microsoft may be a good place to start from. https://github.com/Microsoft/TVHelpers/blob/master/tvjs/src/DirectionalNavigation/directionalnavigation-1.0.0.0.js

[necolas]

Every focusable element in the DOM will include data-focusable="true". This should be enough for spatial navigation libraries, like the one Playstation wrote, to manage focus.

[necolas]

@matthargett did your team ever open source the focus library you mentioned?

[shirakaba]

Unless anyone's done any further work in this field, I intend to (try) to make some minor changes to the sublime react-tv-navigation project to support React Native Web. We'll see how it goes.

EDIT 1: After investigating this a bit more, I've realised that the implementation is heavily coupled to the DOM (i.e. it uses DOM Events and various DOM-only APIs). So this library could be ported to react-native-web, but what I'm really aiming for is a solution that will work on React Native (so that it may code-share with React Native Web). This makes the task much bigger than I'd anticipated – potentially more time than I can justify for an implementation – so I'd prefer to check again: has anyone open-sourced any work on this already?

Tagging @matthargett.

EDIT 2: There's another library: react-vidio-tv-navigation. I've only skim-read it so far, but it's clearly far less coupled to the DOM (in part because it seems less comprehensive than the above solution, e.g. not taking any special consideration of varying element sizes). It was made for use with react-dom.

[shirakaba]

I experimented with react-vidio-tv-navigation for a few hours, but as I converted it to TypeScript, I found many holes and worrying code paths that made me lose faith in it as a solid solution.

I then spent a few hours getting familiar with react-tv-navigation (again, by converting it to TypeScript) but couldn't figure out how to decouple it from the DOM in the end.

Finally, at the end of the work day, I found that there is in fact an existing library for Spatial Navigation in React Native, based on the latter (great!). It does matrix algebra and everything, and it's clearly being used in production:

• Medium blog-post: Smart TV Navigation with React
• GitHub repo: react-spatial-navigation

I'd like to thank Search Engine Optimisation for concealing this..!

I'll report back with how it works out for me with React Native Web.

[shirakaba]

Update: I'm now using react-spatial-navigation for a React Native Web + React Native app, and it works like a dream. Will be a while before I can try on tvOS or Android TV, but NoriginMedia are using it in production for those platforms, so it should fit the bill.