Running on older Windows versions (7, 8 & 8.1) (was: how to run this in win64)

[CyrilFeng]

No description provided.

[mmarquee]

Hi, Do you mean with 64-bit WIndows, or 64-bit Java?
I have it running with both these, do you get errors or failures?
Start with running the demo examples that would show you what is happening and how to get basic information, both for the example test-bed programs and Explorer and Notepad
Let me know how you get on

[CyrilFeng]

I try to run it with the follow code

UIAutomation  automation = UIAutomation.getInstance();
AutomationApplication application = automation.launchOrAttach("notepad.exe");
application.waitForInputIdle(5000);
AutomationWindow window = automation.getDesktopWindow("Untitled - Notepad");

But when I run this it gives a Exception

Exception in thread "main" java.lang.NullPointerException
	at mmarquee.automation.UIAutomation.get(UIAutomation.java:257)
	at mmarquee.automation.UIAutomation.getDesktopWindow(UIAutomation.java:288)
	at foolqq.Test.main(Test.java:18)

win8 64bit and jdk1.8 64bit

[mmarquee]

Do you see Notepad start?
I don't have access to an Windows 8 machine, but it might be an issue with the version of the COM control in Windows 8.

[CyrilFeng]

I saw the Notepad window

[CyrilFeng]

WinNT.HRESULT result0 = uRoot.QueryInterface(new Guid.REFIID(IUIAutomationElement3.IID), pRoot);
this function return -2147467262

[mmarquee]

Hi, thanks. This looks like it a Windows 8 issue - The IUIAutomationElement3 interface is only available in Windows 8.1 and above. I don't have access to a Windows 8 machine on which to test this. I will think about what needs to be done over this weekend, unless you feel like having a go at trying to fix it?
I'll stick some notes here, as I think about it.

[mmarquee]

These are the minimums for the parts that we are using internally,

• IUIAutomationElement3 is available from Windows 8.1 onwards.
• IUIAutomation3 is available from Windows 8.1 onwards.

[mmarquee]

Update: I have created a branch called "windows8-legacy-support", which has an example implementation for a possible version that supports Windows 8, using the IUIAutomation2 and IUIAutomationElement2. If you run the MainLegacy demo, and let me know whether it works (I don't have machine on which to test it). If it is OK, I have some refactoring to do on it, as it has lots of copied code in, but it SHOULD work.
I will also update documentation.

[CyrilFeng]

it return "Not found, retrying"

[mmarquee]

Well, it's a start. It is now loading the controls properly, and looking for the notepad application. Is you Windows installation non-English? It might not be able to find the name of the window if it is not 'Notepad'. It would be possible to add support for other languages here, but if the window title is non-English, then change the demo to try and find the window.

Meanwhile I will see if the Notepad example works with the Legacy version.

[parvuselephantus]

I have very same issue. My environment is Windows 7 64bit - English version. Java jdk1.8.0_05. Code:

UIAutomation automation = UIAutomation.getInstance();
automation.getDesktopWindow("Calculator");

Exception in thread "main"

 java.lang.NullPointerException
	at mmarquee.automation.UIAutomation.get(UIAutomation.java:257)
	at mmarquee.automation.UIAutomation.getDesktopWindow(UIAutomation.java:303)
	at ktm.MainPOC.main(MainPOC.java:13)

I do not have any Visual Studio installed (I try to take the biggest possible JNA advantage on clean windows + Java).

[mmarquee]

Hi, I think this is the same thing, but for Windows 7, it uses the even earlier version of the COM library. I will make the legacy version use the oldest available library and make sure it still works - it will go into the windows8-legacy-support branch if you can get the code out and build it yourself.

[mmarquee]

OK, I have merged all the elements to use the lowest version of the code by default, which as broken one thing, but that is only used in my demos, and I can probably work around it sooner or later.
Please check the windows8-legacy-support branch to test it.

[parvuselephantus]

I have downloaded both your newest version and this:
https://github.com/mmarquee/ui-automation/tree/windows8-legacy-support
version as well. Both crash on my Calculator test with same exception:
Exception in thread "main" java.lang.NullPointerException at com.sun.jna.platform.win32.COM.COMInvoker._invokeNativeObject(COMInvoker.java:42) at com.sun.jna.platform.win32.COM.Unknown.QueryInterface(Unknown.java:70) at mmarquee.automation.UIAutomation.<init>(UIAutomation.java:58) at mmarquee.automation.UIAutomation.getInstance(UIAutomation.java:72) at ktm.MainPOC.main(MainPOC.java:12)

[mmarquee]

Are you using Wndows 7 as above?

[parvuselephantus]

Yes, I am. It's installed as guest OS in VirtualBox.

[mmarquee]

Hmm. That should work, I will try on a Windows 7 machine tomorrow.

[igivon]

hi,
does the legacy version support both IUIAutomation and IUIAutomation3?
it will be grate, because i need it both for windows7 and windows10

[mmarquee]

IUIAutomation is still supported on all platforms. There is one call to IUIAutomationElement3 - to show a popup men, but that should fail gracefully if called from Window 7 or 8

[igivon]

i know IUIAutomation is supported on all platforms but when trying to open Calculator or get any element it fails on Windows 7 64 that support only IUIAutomation.
the only way i manage to get it work is to remove the usage of IUAutomation3 and change it to IUAutomation.
also i didnt see the fall back to IUIAutomation from IUIAutomation3 or any mechanism that will deal with the different platforms . i will be glad if you can point me to it if its presents

[mmarquee]

Are you looking at the https://github.com/mmarquee/ui-automation/tree/windows8-legacy-support branch, as that is where the changes are? Like you said this has involved replacing all the IUIAutomationElement3 and IUIAutomation3 with their previous versions.
This caused a problem with the showContextMenu that is only in IUIAutomationElement3, so there is a COM IUnknown query in there to check that the interface is supported, otherwise it will fail.
If you give that branch a try (I don;t have regular access to a machine with Windows 7 or 8 on it), and see whether it work as you expect. If not Pull Requests are always welcome ;-)

[igivon]

i have tried the legacy branch but it still has issues and not working good with windows7 64.
for example: getFocusedElement in the end, try to use the IUIAutomation3 when trying to getAutyomationElementFromRefrance and throws exception

public IUIAutomationElement getAutomationElementFromReference(final PointerByReference pbr)
throws AutomationException {
Unknown uElement = makeUnknown(pbr.getValue());

    WinNT.HRESULT result0 = uElement.QueryInterface(new Guid.REFIID(IUIAutomationElement3.IID), pbr);

    if (COMUtils.FAILED(result0)) {
        throw new AutomationException(result0.intValue());
    }

    return IUIAutomationElementConverter.PointerToInterface(pbr);
}

[mmarquee]

Thanks, - OK, I've fixed that and committed again, so can you test again.
UPDATE: I've just updated again and removed some more non-backward compatibility issues - most;y in the tests)

[parvuselephantus]

I just checked - still getting exactly the same NullPointerException (I took latest 'windows8-legacy-support' version). Any Idea what I could have done wrong? Is there anything I could check for you?

[mmarquee]

@parvuselephantus Is it the same stack trace as before? Code you send me the code that makes it happen? That way I can see whether there is anything obvious about that would make it not work, and it myself and debug it.

[mmarquee]

(also see update above)

[parvuselephantus]

Hi @mmarquee. Sorry for that, but I confirm it's all exactly same: same code, same win7 64bit, same stacktrace (even numbers of lines are same). I have just downloaded the newest version of ui-automation-windows8-legacy-support branch.

[mmarquee]

@parvuselephantus Oh no. I am not going to be near a Window 7 PC until tuesday (and even then might not be able to get to use it). I'll try and get access to it and see what happens.

[mmarquee]

@parvuselephantus I have just run this on a Windows 7 machine, and it fails (even when run from the legacy branch), so I have something to work through now. It should work, as the interface is implemented in Windows 7.

[mmarquee]

The issue is in getRootElement from the automation library call itself.

Not sure why that would fail on Windows 7.

[mmarquee]

I have merged the changes in the legacy branch(well applied the necessary changes). It has been published as a snapshot (0.5.0-SNAPSHOT), which has a lot of other changes as well. I don't have access to a Windows 7, or 8/8.1 PC, so it would be great if someone could test this on one of those and feedback to me here.

[pbi-qfs]

I have it running successfully on Windows 8.1, but it fails to run on Windows 7 saying:

 java.lang.NullPointerException
	at com.sun.jna.platform.win32.COM.COMInvoker._invokeNativeObject(COMInvoker.java:42)
	at com.sun.jna.platform.win32.COM.Unknown.QueryInterface(Unknown.java:70)
	at mmarquee.automation.UIAutomation.<init>(UIAutomation.java:101)
	at mmarquee.automation.UIAutomation.getInstance(UIAutomation.java:147)

[mmarquee]

Hmm. That was what I was seeing on my old WIndows 7 laptop (which is at work). I am really not sure that Windows 7 actually supports this library.

[FloMarks]

Hey,
is there a way now to run UI Automation on Windows 7 ?

[mmarquee]

It is still an ongoing issue I am afraid.

The Microsoft documentation implies that it should work (at least in it's most basic implementation) on Windows 7 (and in Vista), however from investigating on several Windows 7 machines, this doesn't seem to be true. (see comment above). So at the moment the library does not support Windows 7.

It should be possible of do much the same automation using the older (but still supported MSAA -
Microsoft Active Accessibility Support - library), but would need someone to investigate and have access to a Windows 7 machine - which I don't. I think JNA (The underlying library that is used by this application already has support for it).

[mmarquee]

See this document for details

https://github.com/mmarquee/ui-automation/blob/master/docs/windows-support.md

[FloMarks]

Okay thanks, this is good to know.
Are you still working on that issue or are you focusing on the newer versions of Windows ?
If not, I will try MSAA and investigate how the functions I need are implemented.

[mmarquee]

I was hoping to ignore this, but it is the single most active issue I've got, so once I have tidied the code up a bit - I have some additional code to merge in, then I might return to it - I have some MSAA support already, but that might require the UIAutomationCore library and so we would be stuck again.

Keeping it seamless would be the key, and the library would have to degrade nicely back to MSAA if nothing else is available. something I would have to simulate in Windows 10.

I will have play with this over the weekend and see whether I can work out how to make a start.

[FloMarks]

Ok, I've checked MSAA and I don't think it's that what im looking for, because there are too many improvements on the UI Automation side.
When do you think is UI Automation approximately available for Windows 7 ? Im currently working with Windows 10 and test some stuff.

[mmarquee]

I have bought a Windows 7 machine to look at this issue. I'll start to look at this asap.

[ghost]

@mmarquee Wow! You bought a Windows 7 machine especially for it, can we help you? Like donations or something?

[mmarquee]

It was the cheapest laptop I could find, so very cheap. Also very slow! I've managed to get the code on there and running. 'Good news' is that most of the tests fail - so it's something to work on, as I am getting the same behaviour as everyone else.

[ghost]

@mmarquee Is there a possibility to contact you by email. We have professional interest and might need some help in the world of Java + UI Automation.

[mmarquee]

I've added a contact email to my profile - it is just for github traffic, so I don't check it every day

[khanhtd36]

I have the same issue when I use
UIAutomation instance = UIAutomation.getInstance();

I got exception:
Exception in thread "JNativeHook Dispatch Thread" java.lang.NoSuchMethodError: com.sun.jna.platform.win32.Guid$GUID.createFieldsOrder([Ljava/lang/String;)Ljava/util/List; at com.sun.jna.platform.win32.Guid$GUID.<clinit>(Guid.java:108) at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Unknown Source) at com.sun.proxy.$Proxy6.<clinit>(Unknown Source) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source) at java.base/java.lang.reflect.Proxy.newProxyInstance(Unknown Source) at java.base/java.lang.reflect.Proxy.newProxyInstance(Unknown Source) at com.sun.jna.Native.loadLibrary(Native.java:415) at com.sun.jna.platform.win32.Ole32.<clinit>(Ole32.java:47) at mmarquee.automation.Ole32Wrapper.createWrapper(Ole32Wrapper.java:50) at mmarquee.automation.Ole32Wrapper.<init>(Ole32Wrapper.java:46) at mmarquee.automation.UIAutomation.<init>(UIAutomation.java:71) at mmarquee.automation.UIAutomation.getInstance(UIAutomation.java:132)

I use your 0.4.3 version
I tried JNA 4.2.2, 4.3.0 and 4.5.0 and all failed

Win 7 64-bit
JDK 9.0.4 64-bit

[mmarquee]

Not sure about the above error, it looks different from the original issue - it might be a Java 9 issue.

[mmarquee]

I have tried the library on Windows 7 and it doesn't work. However the uispy application does work, so I'm confused.

[violinner]

I have a Windows 7 laptop and Eclipse.
Starting with the .jar file for version 0.4.3 from Maven Central:
// https://mvnrepository.com/artifact/com.github.mmarquee/ui-automation
compile group: 'com.github.mmarquee', name: 'ui-automation', version: '0.4.3'

• I decompiled and altered the following classes to change all "IUIAutomationElement3" to "IUIAutomationElement:"
  IUIAutomationElementConverter.java
  AutomationElement.java
  BaseAutomation.java
  UIAutomation.java
  AutomationBase.java

• The library now appears to work on Windows 7

• This is a light at the end of the tunnel, but it does not help my project: I am trying to do automated testing on a very large Java/Swing application that is compiled into a native Windows application. It only presents one window to the ui-automation library, so I suppose this is not going to work for me.

[pbi-qfs]

@violinner That's great news for testing native windows applications on Windows 7. We can build upon that information to improve the ui-automation lib.

As you also recognized, Java Swing applications are not mapped to the UI-Automation framework by Java, so you need test-software which especially integrates with Java UIs, as for example QF-Test (https://www.qfs.de/en.html) or others.

(Disclaimer: I'm an active contributor of the ui-automation-library as well as a developer of QF-Test)

[mmarquee]

@violinner Thanks.
I will see whether I have missed one out of my experiments. Could you do look in the registry and see whether the IUIAutomation interface is registered - just search for it. In my standard Windows 7 installations, it isn't and this seems to be the issue. If it is there for you, then we will have to work out how (and why) it is.

As @pbi-qfs noted the Swing controls are opaque to IUIAutomation, all it sees is the overall window.

[violinner]

@mmarquee
Searching for "IUIAuto", I found (in triplicate):
IUIAutomationEventHandler, with IID matching IUIAutomationEventHandler.java:
"146C3C17-F12E-4E22-8C27-F894B9B79C69"

also:
IUIAutomationPropertyChangedEventHandler, IUIAutomationCrossBitnessHook, IUIAutomationFocusChangedEventHandler, IUIAutomationStructureChangedEventHandler

Does this help?

[mmarquee]

@violinner Thanks - that is what I see on my Windows 7 machine, which doesn't work. I will have a look when I get back tonight.

[mmarquee]

Looking at the code I have in the current branch, it doesn't mention IUIAutomationElement3 in those files (unless checking first). Can you try the same approach with the 0.5.0-SNAPSHOT?

[autoasi]

Hi @mmarquee
Any updates on UI Automation with Windows 7 ? is there any other way to run UI Automation on Windows 7?

[vali92]

On Windows 10 is working fine, but on Windows 7 64 bit i just tried and it is not working.

JAR ver: ui-automation-0.4.3.jar

It is failing when trying to get the Form, e.g.:

 UIAutomation automation = UIAutomation.getInstance();
 AutomationApplication application = automation.launchOrAttach("...\\Project1.exe");
 **AutomationWindow window = automation.getDesktopWindow("Form1");**
 window.focus();

java.lang.NullPointerException
at mmarquee.automation.UIAutomation.get(UIAutomation.java:257)
at mmarquee.automation.UIAutomation.getDesktopWindow(UIAutomation.java:303)
at frameworkSrc.autoInterface.desktopTemplate.beforeMethod(desktopTemplate.java:43)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
at org.testng.internal.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:59)
at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:458)
at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:222)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:523)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:719)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:989)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
at org.testng.TestRunner.privateRun(TestRunner.java:648)
at org.testng.TestRunner.run(TestRunner.java:505)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415)
at org.testng.SuiteRunner.run(SuiteRunner.java:364)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1137)
at org.testng.TestNG.runSuites(TestNG.java:1049)
at org.testng.TestNG.run(TestNG.java:1017)
at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:114)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:251)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:77)

[mmarquee]

OK, this is the same issues as everyone has. I don't think it is going to be possible to make this library work on Windows 7. I have a Windows 7 PC to experiment with, but I cannot get this libraries to load properly.

[denlye]

CANALIZED_OLE32_INSTANCE = Canalizer.canalize(com.sun.jna.platform.win32.Ole32.INSTANCE);
when Windows 7
CANALIZED_OLE32_INSTANCE = Ole32.INSTANCE;