Improve support & documentation for UI testing on a mac

[mrlacey]

I'm submitting a...

• Documentation issue or request

Current behavior

UI Testing on a Mac is difficult and poorly documented or supported.
I have previously gotten it to work but updates (that I'm not entirely sure about) mean it no longer does 😢.

It seems that there are many factors that can influence how/if this works and having the right combination of library versions to match the versions of OS X, XCode, VS4Mac, & more(?) are required.

There is also a lack of resources to help debug related issues.

Expected behavior

Documentation of how to run UI tests on a mac would be really helpful.
Ideally, these will include:

• A guide to UI testing on a mac (how to get started, what settings are needed, etc.)
• Known good configurations of libraries, tooling, and OS versions.
• Debugging guide for when things don't work as desired/expected.
• Documentation of the scripts used for UI testing in the main Uno project.

Minimal reproduction of the problem with instructions

n/a

Environment

Package Version(s): 

Visual Studio
- [ ] 2019 (version: )
- [ ] 2019 Preview (version: )
- [x] for Mac (version: 8.9.7)

[jeromelaban]

Thanks for the report. Can you indicate what's not working ? Which error messages ? Sometimes it's about updating Xamarin.UITest.

[mrlacey]

I was getting a Calabash.XDB.Exceptions.DeviceAgentException and then an NSInternalConsistencyException

I assumed that it was related to Xamarin.UITest but because of dependency compatibility issues, I couldn't update that on its own.

In the end, I gave up and updated everything: OSX, XCode, VS4Mac, Xamarin workloads, and all the NuGet packages and managed to get it working again.

I thought I had turned off every automatic update setting on my machines to avoid the situation where something changes unexpectedly and then something I rely on stops working. I guess this just needs to be yet another area where I have to assume I must keep updating to the latest version of everything to keep using it.

[jeromelaban]

That's correct. Unfortunately, xCode updates tend to break stuff very regularly. My assumption is that the Xamarin.UITest team will make an update that will fix this.