New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for applications running in an application player like java applications #4360

Closed
nvaccessAuto opened this Issue Aug 2, 2014 · 23 comments

Comments

Projects
None yet
2 participants
@nvaccessAuto

nvaccessAuto commented Aug 2, 2014

Reported by heikofolkerts on 2014-08-02 19:18
Applications written in interpreted languages such as java applications use a kind of player when being run. So java aplications are all identified by javaw.exe or java.exe from NVDA. Since the applications ,may need very different and conflicting tweaks from an app module one cannot put the code for different applications in one app module.

Here is a proposal how this could be supported:

  1. an app module for the application player like javaw.exe or python.exe is loaded every time NVDA sees an java or python application. thius is the current and standard behaviour of NVDA.
  2. the application player app module can then forward the application support to a specific app module. 'To do so it must identify the application for which it was loaded. It can try to get the command line for the process ID. This is at least possible with VBA and should hopefully also be possible in python. The identification of an application from its process ID should be done from the application player's app module since the parsing of the command lines will be different for the players.

In case of java one would have to extract the package and class name of the main class that was given on the command line. In case of a python application one would need to identify the name of the main python file.

The support for example java applications is crucial for situations where a user would have to work with more than one java application and an specific application module willb be needed for one or more of them. In case of java applications this can happen quite often since keyboard support in those applications is usually very poor.

Blocked by #4556
Blocking #4454, #4569

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Aug 3, 2014

Comment 1 by jteh on 2014-08-03 22:23
Just for reference, not all Java applications run like this. I've never seen this behaviour on my system; both Eclipse and AccProbe run as their respective executables rather than javaw.exe. This needs to be kept in mind when developing; whoever does this work will need to make sure they can find a version of an application which actually produces this.

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Aug 3, 2014

Comment 2 by jteh on 2014-08-03 22:36
For reference, Heiko posted this link regarding getting the command line for another process. This uses WMI, so we'd need to check whether WMI is always available on modern systems. (We switched away from using WMI a few years ago, but I don't recall why.) There is another way to do this, but it's fairly ugly and uses internal Windows APIs.

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Sep 11, 2014

Comment 3 by jteh on 2014-09-11 04:15
For reference, here's how to get the command line for a process using Python, comtypes and WMI:

def getCommandLineForProcess(processId):
    wmi = comtypes.client.CoGetObject(r"winmgmts:root\cimv2", dynamic=True)
    results = wmi.ExecQuery("select CommandLine from Win32_Process "
        "where ProcessId = %d" % processId)
    for result in results:
        return result.CommandLine
    raise LookupError("No such process")
@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Sep 11, 2014

Comment 4 by jteh on 2014-09-11 11:08
I've taken a first stab at implementing this for javaw.exe. Please try this try build and report whether this works. You can see the name NVDA is using in the NVDA developer info obtained by pressing NVDA+f1. For example:

appModule: <'appModuleHandler' (appName u'javaw_com_sun_deploy_panel_ControlPanel', process ID 7872) at address 4e42910>

I'm particularly interested in whether this reports "eclipse" for users where Eclipse normally reports as "javaw".

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Sep 11, 2014

Comment 5 by driemer.riemer@... on 2014-09-11 16:45
Yay! it does report eclipse.

appModule: <'eclipse' (appName u'eclipse', process ID 3112) at address 56773f0>
appModule.productName: u'Java(TM) Platform SE 7 U51'
appModule.productVersion: u'7.0.510.13'
TextInfo: <class 'displayModel.EditableTextDisplayModelTextInfo'>

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Sep 11, 2014

Comment 6 by driemer.riemer@... on 2014-09-11 16:46
The double treeview bug goes away when this happens as well.

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Sep 11, 2014

Comment 7 by jteh (in reply to comment 5) on 2014-09-11 23:20
Replying to driemer.riemer@…:

TextInfo: <class 'displayModel.EditableTextDisplayModelTextInfo'>

Err, I'm a bit worried that it's using EditableTextDisplayModelTextInfo. I thought Eclipse was all IA2 now. What control was that?

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Sep 12, 2014

Comment 8 by driemer.riemer@... on 2014-09-12 05:43
The main edit text control.

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Sep 12, 2014

Comment 9 by jteh on 2014-09-12 05:46
What version of Eclipse is that? Is it ancient? What does TextInfo report when you use this version of Eclipse with a release/master/next NVDA (not the try build)?

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Sep 13, 2014

Comment 10 by driemer.riemer@... on 2014-09-13 05:47
Well if you consider ancient to be a year old, then yeah. but next reports u'javaw'

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Sep 13, 2014

Comment 11 by jteh (in reply to comment 10) on 2014-09-13 11:49
Replying to driemer.riemer@…:

Well if you consider ancient to be a year old, then yeah. but next reports u'javaw'

What does TextInfo report when you use this version of Eclipse with a release/master/next NVDA (not the try build)?

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Sep 13, 2014

Comment 12 by driemer.riemer@... on 2014-09-13 21:23
TextInfo: <class 'NVDAObjects.IAccessible.IA2TextTextInfo'>

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Sep 13, 2014

Comment 13 by manish (in reply to comment 4) on 2014-09-13 22:31
in the spring tool suite version of eclipse, the try build reports the app module name as "sts" for the name of the executable, which is "sts.exe". The master build reports "javaw".
Can we have some kind of mapping where a user can select which appmodule to use for which application. As in, even if nvda correctly reports the name of the application instead of javaw, there are so many variants of eclipse out there that one is likely to need to manually rename or copy the eclipse.py file anyway. It will be nice if there was a dialog in nvda that will allow me to pick an existing appmodule for an app, which will also be driven by the executable name but the .py file will not need to have the same name then.

Replying to jteh:

I've taken a first stab at implementing this for javaw.exe. Please try this try build and report whether this works. You can see the name NVDA is using in the NVDA developer info obtained by pressing NVDA+f1. For example:

appModule: <'appModuleHandler' (appName u'javaw_com_sun_deploy_panel_ControlPanel', process ID 7872) at address 4e42910>

I'm particularly interested in whether this reports "eclipse" for users where Eclipse normally reports as "javaw".

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Sep 14, 2014

Comment 15 by jteh (in reply to comment 13) on 2014-09-14 07:43
Replying to manish:

Can we have some kind of mapping where a user can select which appmodule to use for which application.

This still requires technical knowledge. Furthermore, if one user needs this, chances others will as well, so it makes sense to get such changes into the core. Filed #4454 for STS.

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Sep 14, 2014

Comment 16 by jteh (in reply to comment 12) on 2014-09-14 10:17
Replying to driemer.riemer@…:

TextInfo: <class 'NVDAObjects.IAccessible.IA2TextTextInfo'>

Okay. Can you please double check the TextInfo for the main edit area with both the try build and a master/next build? I'm very concerned that the try build is somehow causing EditableTextDisplayModelTextInfo to be used for Eclipse (as suggested by comment:5) instead of IA2TextTextInfo. The TextInfo shouldn't be affected by this change. If it is, something is very wrong.

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Sep 14, 2014

Comment 17 by driemer.riemer@... on 2014-09-14 10:59
Nope in the try build iyt says
"
TextInfo: <class 'NVDAObjects.IAccessible.IA2TextTextInfo'>

"

I still am confused as to why that other day it was different.
I tried onh next and the try build. with the same textInfo.

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Oct 10, 2014

Comment 18 by James Teh <jamie@... on 2014-10-10 08:14
In [26aca87]:

Merge branch 't4360' into next

Incubates #4360.

Changes:
Added labels: incubating

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Oct 10, 2014

Comment 19 by jteh on 2014-10-10 09:41
I'm just going to cover javaw.exe in this ticket, though the core framework can be easily used to support other application hosts if need be.

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Oct 21, 2014

Comment 21 by James Teh <jamie@... on 2014-10-21 04:28
In [d4a83c0]:

Merge branch 't4360' into next

Incubates #4360. Fixes #4556.

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Nov 6, 2014

Comment 23 by James Teh <jamie@... on 2014-11-06 04:46
In [8a576f9]:

For executables which host many different apps (e.g. javaw.exe), code can now be provided to load specific app modules for each app instead of loading the same app module for all hosted apps.

Support for javaw.exe is implemented.
Fixes #4360.

Changes:
Removed labels: incubating
State: closed

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Nov 6, 2014

Comment 24 by jteh on 2014-11-06 04:47
Changes:
Milestone changed from None to 2014.4

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Nov 12, 2014

Comment 25 by jteh on 2014-11-12 03:33
Significant fixes to the javaw app module in 4f69696 (committed straight to master).

@nvaccessAuto

This comment has been minimized.

nvaccessAuto commented Jan 3, 2015

Comment 26 by nvdakor on 2015-01-03 03:43
Hi all,
Coming back to this, as I'm researching ways of applying javaw fixes to wwahost (#4569).
There appears to be a flaw: in some Java-based programs, the command line is the full path to the executable/jar archive. A very good example is Braille Blaster (this translator uses braille tables from LibLouis). There might be a way to get the actual name of the jar file (not the full path) - see #4770.
Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment