Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

A0 UIAutomation instruments problems

Karl Krukow edited this page · 5 revisions

Calabash has two ways of launching apps: SimLauncher and Apple's instruments tool. Each has its own advantages and problems (see below).

For iOS7 only the instruments launch method is supported. If instruments is not used to automatically launch the app touch events wont work.

NOTE There is a bug in Apple's instruments tool when running the iPhone non-retina sim! If you're seeing the following

ScriptAgent[58044:2207] CLTilesManagerClient: XPC_ERROR_CONNECTION_INVALID!

when running with DEBUG=1, please try switching to the iPhone Retina simulator.

Since version 0.9.154, Calabash is using instruments and UIAutomation to launch apps by default. The primary reason for this is two-fold: (a) in iOS7, Apple removed access to APIs previously used to synthesize touch events and we needed a different approach, (b) there are things possible with UIA that are not possible from "inside the app", i.e., in the Calabash server (like accepting the Access to current location dialog or sending the app to the background).

Launching

The launch method can be controlled using environment variables. With Calabash 0.9.157+, the use of environment variables was greatly simplified. If you're updating from a previous version, it is recommended to try removing all the environment variables, and then adding only the ones you're sure you need.

  • DEVICE_TARGET controls which device to launch on. When set Calabash uses instruments to launch. The default is 'simulator' unless a device is plugged in via USB, in which case it uses that device (see Testing on iDevices). You can also set DEVICE_TARGET=UDID_OF_CONNECTED_DEVICE to pick a specific connected device, or DEVICE_TARGET=simulator to force use of instruments and simulator. (You can see the UDID of your device in the Devices tab in Xcode's Organizer window).

  • SDK_VERSION is only used when running on the simulator. It controls which iOS version of the simulator is launched. Note, SDK_VERSION forces the launch method to be SimLauncher (not instruments). The reason for this is that the instruments tool does not support launching a simulator that is not the latest version.

  • DEVICE=ipad if you want to ensure we launch the iPad simulator.

  • OS is deprecated and now automatically detected (it was previously used to decide which "recordings" to use for playback).

  • NO_LAUNCH is not supported when running with instruments. It tells Calabash to not automatically launch the app (it is mostly used when you want to start the app yourself - should rarely be used).

  • NO_STOP means don't stop the app after the test completes. (It is mostly used to leave the app open in case of a test failure and should be used for debugging purposes only).

Stability of instruments

With Xcode5 things seem to have improved, however, the commandline interface to Instruments (the instruments tool) is somewhat unstable and not very good at providing good error messages. These problems with instruments propagate up to Calabash when trying to launch apps.

If you're experiencing problems alá "Time out waiting for UIAutomation run-loop to Start" try the following and post the output to the Google group if you can't make it work.

  1. Have you built the -cal target from XCode?

  2. Are you using Xcode5? Have you run sudo xcode-select -switch PATH_TO_XCODE/Content/Developer. When using the final version of Xcode5 you should have

xcode-select -print-path /Applications/Xcode.app/Contents/Developer

➜ ~ instruments instruments, version 5.0 (51166)

Xcode5 must be installed in the default /Applications/Xcode.app location

  1. Please stop the simulator. Do not start it from Xcode, instead let Calabash launch the simulator.

  2. Make sure you use the iPhone Retina or the iPad simulator. There is a well-known bug with instruments and its ability to launch the iPhone (non-retina simulator).

  3. With release 0.9.157 the use of environment variables was simplified. Calabash tries to do the right thing by default. There should generally be no need to set ANY environment variables for default project setups. APP_BUNDLE_PATH should only be set if Calabash is not able to detect it correctly. SDK_VERSION should only be set if you don't want to run in the latest iOS simulator version. OS should never need to need to be set.

  4. Is Calabash detecting the correct APP_BUNDLE_PATH?

Try the following - it is the easiest way to test if it is working:

In your project folder (typically the folder just above your 'features' folder), run

DEBUG=1 calabash-ios console
Running irb...
irb(main):001:0> start_test_server_in_background(:timeout => 30)

{:launch_method=>:instruments, :reset=>false, :bundle_id=>"com.littlejoysoftware.Briar-cal", :device=>"iphone", :no_stop=>false, :no_launch=>false, :sdk_version=>nil, :device_target=>"simulator", :app=>"/Users/krukow/Library/Developer/Xcode/DerivedData/Briar-ghprbavkjaxwjvdoetpjtgjfijef/Build/Products/Debug-iphonesimulator/Briar-cal.app", :script=>"/Users/krukow/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/run_loop-0.0.20/scripts/run_loop.js"}

device_target=simulator
udid=
bundle_dir_or_bundle_id=/Users/krukow/Library/Developer/Xcode/DerivedData/Briar-ghprbavkjaxwjvdoetpjtgjfijef/Build/Products/Debug-iphonesimulator/Briar-cal.app
script=/var/folders/j8/g6wblrl552zfjqrsl5p6fd9m0000gn/T/run_loop20130919-24192-i2dymk/_run_loop.js

log_file=/var/folders/j8/g6wblrl552zfjqrsl5p6fd9m0000gn/T/run_loop20130919-24192-i2dymk/run_loop.out
Preparation took 0.047794 seconds

### Starting on simulator App: /Users/krukow/Library/Developer/Xcode/DerivedData/Briar-ghprbavkjaxwjvdoetpjtgjfijef/Build/Products/Debug-iphonesimulator/Briar-cal.app ###
2013-09-19 18:57:45 +0200 instruments -D "/var/folders/j8/g6wblrl552zfjqrsl5p6fd9m0000gn/T/run_loop20130919-24192-i2dymk/trace" -t /Applications/Xcode.app/Contents/Applications/Instruments.app/Contents/PlugIns/AutomationInstrument.bundle/Contents/Resources/Automation.tracetemplate "/Users/krukow/Library/Developer/Xcode/DerivedData/Briar-ghprbavkjaxwjvdoetpjtgjfijef/Build/Products/Debug-iphonesimulator/Briar-cal.app" -e UIARESULTSPATH /var/folders/j8/g6wblrl552zfjqrsl5p6fd9m0000gn/T/run_loop20130919-24192-i2dymk -e UIASCRIPT /var/folders/j8/g6wblrl552zfjqrsl5p6fd9m0000gn/T/run_loop20130919-24192-i2dymk/_run_loop.js &> /var/folders/j8/g6wblrl552zfjqrsl5p6fd9m0000gn/T/run_loop20130919-24192-i2dymk/run_loop.out
Launching took 8.055606 seconds

#<Calabash::Cucumber::Launcher:0x007fe581b1f800 @run_loop={:pid=>24239, :udid=>nil, :app=>"/Users/krukow/Library/Developer/Xcode/DerivedData/Briar-ghprbavkjaxwjvdoetpjtgjfijef/Build/Products/Debug-iphonesimulator/Briar-cal.app", :repl_path=>"/var/folders/j8/g6wblrl552zfjqrsl5p6fd9m0000gn/T/run_loop20130919-24192-i2dymk/repl-cmd.txt", :log_file=>"/var/folders/j8/g6wblrl552zfjqrsl5p6fd9m0000gn/T/run_loop20130919-24192-i2dymk/run_loop.out", :results_dir=>"/var/folders/j8/g6wblrl552zfjqrsl5p6fd9m0000gn/T/run_loop20130919-24192-i2dymk", :initial_offset=>422}, @device=#<Calabash::Cucumber::Device:0x007fe5810825f0 @endpoint=#<URI::HTTP:0x007fe5810705f8 URL:http://localhost:37265/>, @system="x86_64", @device_family="iPhone", @simulator_details="iPhone Simulator 463.9.4, iPhone OS 7.0 (iPhone Retina (3.5-inch)/11A465)", @ios_version="7.0", @framework_version="0.9.155">>
irb(main):002:0>

Note the line: log_file = /var/folders/j8/g6wblrl552zfjqrsl5p6fd9m0000gn/T/run_loop20130919-24192-i2dymk/run_loop.out

What is the contents of this file? run

cat /var/folders/j8/g6wblrl552zfjqrsl5p6fd9m0000gn/T/run_loop20130919-24192-i2dymk/run_loop.out

Something went wrong with that request. Please try again.