interaction with math content does not work when NVDA language is Japanese

[nvaccessAuto]

Reported by nishimotz on 2015-06-01 12:38
setup:

• Windows 8.1 32bit (Windows UI language is Japanese)
• installed MathPlayer 4.0 beta 1
• installed NVDA 2015.2rc2 (SAPI5 Japanese Haruka Desktop Voice)

tested contents:

(1)
IE11 with
http://www.dessci.com/en/products/mathtype/compare/mathplayer.htm#ZEqnNum571653

(2)
Firefox 38.0.1 with
http://www.w3.org/Math/XSL/csmall2.xml

steps:

• NVDA General Settings, Language is set to Windows Default (i.e. Japanese)
• open the content and navigate to the MathML object
• NVDA announces math content in Japanese language
• press NVDA+Alt+M to start interactive navigation
• NVDA again announces the math content in Japanese language
• press down arrow key
• exptected: zoom into a portion
• actual: NVDA says "No navigation files for this speech style in this language"

It works as expected when NVDA language is switched to English.

[nvaccessAuto]

Comment 1 by jteh on 2015-06-02 11:40
As the error suggests, mathPlayer unfortunately doesn't support navigation in Japanese. You'd need to work with Design Science to fix this. There's nothing we can do on our end.
Changes:
Added labels: cantfix
State: closed

[nvaccessAuto]

Comment 2 by nishimotz on 2015-06-02 13:29
I think MathPlayer does support navigation in Japanese, because it works with the following setup:

• Windows 8.1 UI language: Japanese
• NVDA language: English
• NVDA synthesizer and voice: SAPI5 Microsoft Haruka Desktop Japanese

With this setup, NVDA announces math contents in Japanese language.

NVDA+Alt+M can zoom into portions, and with the arrow keys it announces the portions in Japanese language as well, although messages such as 'zoom in' or 'zoomed in all of the way' are still in English.

[nvaccessAuto]

Comment 3 by jteh on 2015-06-03 00:15
In this case, MathPlayer will probably use English as its primary language. However, because your synthesiser is Japanese, numbers, letters, etc. will be pronounced in Japanese.

[nvaccessAuto]

Comment 4 by nishimotz on 2015-06-03 06:27
I should follow up regarding the case reported previously.

content:

http://www.w3.org/Math/XSL/csmall2.xml

such as the second one:

<math xmlns="http://www.w3.org/1998/Math/MathML">
<apply>
  <in/>
  <cn type="complex-cartesian">17<sep/>29</cn>
  <complexes/>
</apply>
</math>

setup 1:

• Windows 8.1 UI language: Japanese
• installed NVDA 2015.2rc2
• installed MathPlayer 4.0 beta 1
• Firefox 38.0.5
• NVDA language: English
• Synthesizer: SAPI5
• Voice: Microsoft Haruka Desktop Japanese
• Voice: Punctuation/symbol: none
• Voice settings "Trust voice's language when...": not checked

In this case, NVDA can announce the math as follows and NVDA+Alt+M works.

"17 plus 29 i is an element of set the complex numbers"

setup 2 (changed from setup 1):

• Voice settings "Trust voice's language when...": checked

In this case, NVDA can announce it in Japanese and the Speech Viewer shows:

17 
purasu (in Japanese Katakana)
29 
ai (in Japanese Katakana)
youso obu (in Japanese Kanji and Katakana)

"is an element of set" is translated into "youso obu" written in Japanese characters, which does not come from NVDA localization.

"the complex numbers" is not translated into Japanese.
It seems MathPlayer do not have the Japanese resource regarding this.

If NVDA+Alt+M is pressed:

math
17 
purasu (in Japanese Katakana)
29 
ai (in Japanese Katakana)
youso obu (in Japanese Kanji and Katakana)

Followed by the down arrow key:

zoom in  
17 
purasu (in Japanese Katakana)
29 
ai (in Japanese Katakana)

In this case, interactive navigation works in Japanese.

setup 3 (changed from setup 2):

• NVDA language: Default (Japanese)

NVDA can announce it in Japanese in the same way:

17 
purasu (in Japanese Katakana)
29 
ai (in Japanese Katakana)
youso obu (in Japanese Kanji and Katakana)

If NVDA+Alt+M is pressed:

suushiki (math equation in Japanese Kanji)
17 
purasu (in Japanese Katakana)
29 
ai (in Japanese Katakana)
youso obu (in Japanese Kanji and Katakana)

Followed by the down arrow key:

No navigation files for this speech style in this language

[nvaccessAuto]

Comment 5 by jteh on 2015-06-03 06:57
It seems MathPlayer uses the English navigation rules in setup 2 for some reason, even though it uses the Japanese math rules. I'd actually say this is a bug, even though it does allow partial navigation.

In setup 3, are you certain you didn't disable "Trust voice's language when processing characters and symbols"? Otherwise, I don't see why the language we report to MathPlayer would be different.

For setups 2 and 3, what does NVDA report for this command in the Python console?

speech.getCurrentLanguage()

[nvaccessAuto]

Comment 6 by nishimotz on 2015-06-03 07:24
In setup 3, "Trust voice's language" is still checked.

For both setups 2 and 3, speech.getCurrentLanguage() is 'ja_JP' when NVDA+Alt+M is activated.

[nvaccessAuto]

Comment 7 by jteh (in reply to comment 6) on 2015-06-03 07:35
Replying to nishimotz:

In setup 3, "Trust voice's language" is still checked.

In that case, I have no idea why you're seeing different results in setups 2 and 3. In either setup, MathPlayer should be told that the language is "ja_JP". If you are able, you might wish to double check what we're passing to MathPlayer in mathPres.mathPlayer.MathPlayer._setSpeechLanguage (around line 131).

[nvaccessAuto]

Comment 8 by nishimotz on 2015-06-03 09:34
I agree that MathPlayer does not work with Japanese locale for navigation.

My tentative work around it is as follows, which always uses English locale for navigation.

Is this issue only in Japanese language?

diff --git a/source/mathPres/mathPlayer.py b/source/mathPres/mathPlayer.py
index e21627a..a2aec5e 100644
--- a/source/mathPres/mathPlayer.py
+++ b/source/mathPres/mathPlayer.py
@@ -77,7 +77,7 @@ class MathPlayerInteraction(mathPres.MathInteractionNVDAObject):

        def __init__(self, provider=None, mathMl=None):
                super(MathPlayerInteraction, self).__init__(provider=provider, mathMl=mathMl)
-               provider._setSpeechLanguage(mathMl)
+               provider._setSpeechLanguage(mathMl, interaction=True)
                provider._mpSpeech.SetMathML(mathMl)

        def reportFocus(self):
@@ -124,10 +124,12 @@ class MathPlayer(mathPres.MathPresentationProvider):
                self._mpNavigation = mpSpeech.QueryInterface(IMathNavigation)
                self._mpBraille = mpSpeech.QueryInterface(IMathBraille)

-       def _setSpeechLanguage(self, mathMl):
+       def _setSpeechLanguage(self, mathMl, interaction=False):
                lang = mathPres.getLanguageFromMath(mathMl)
                if not lang:
                        lang = speech.getCurrentLanguage()
+               if interaction and lang[:2] == 'ja':
+                       lang = 'en'
                self._mpSpeechSettings.SetLanguage(lang.replace("_", "-"))
                self._language = lang

[nvaccessAuto]

Comment 9 by jteh on 2015-06-03 11:05
The problem with this workaround is that it may cause MathPlayer to use English for all math content spoken while navigating. The correct solution really is to work with Design Science to get this translated properly.