Skip to content

Latest commit

 

History

History
778 lines (569 loc) · 46.2 KB

TESTING.rst

File metadata and controls

778 lines (569 loc) · 46.2 KB

Testing AngularJS Library

These are instructions for pulling in all the parts and testing the AngularJS Library for Robot Framework

Setup Environment

We will have both a base set of pythons packages as well as the source for the AngularJSLibrary and the Selenium2Library all of which will will want to keep isolated from your system python and its packages. As such we will use Python's virtual environment. Let's start by creating a a root folder for testing.

mkdir test-ng
cd test-ng

Within this root folder we will create the virtualenv and clone source repositories

virtualenv -p /usr/bin/python2.7 --no-site-packages clean-python27-env
source clean-python27-env/bin/activate
pip install decorator docutils robotframework selenium

git clone git@github.com:Selenium2Library/robotframework-angularjs.git rf-ng
git clone git@github.com:robotframework/Selenium2Library.git rf-s2l

We will also clone the protractor repository. From Protractor we will use their test site, testapp, but not their test server. For the test server we will use the Selenium2Library test server with some modifications.

git clone git@github.com:angular/protractor.git ptor
cp -R ptor/testapp rf-s2l/test/resources/.

I modified the async testapp page so that the implicit wait for angular functionality can be tested. The modified version of async.html and async.js can be moved over to the testapp directory under rf-s2l directory.

cp rf-ng/AngularJSLibrary/async.html rf-s2l/test/resources/testapp/ng1/async/.
cp rf-ng/AngularJSLibrary/async.js rf-s2l/test/resources/testapp/ng1/async/.

Modifying the test server of Selenium2Library, rf-s2l\test\resources\testserver\testserver.py, add the following method, do_GET, to the StoppableHttpRequestHandler class.

def do_GET(self):
    """Response pages for Angular tests.

    Added by Edward Manlove - June 5, 2014
    """
    if self.path.endswith('/fastcall'):
        self.send_response(200)
        self.send_header('Content-type', 'text/html')
        self.end_headers()
        self.wfile.write('done')
    elif self.path.endswith('/slowcall'):
        sleep(2)
        self.send_response(200)
        self.send_header('Content-type', 'text/html')
        self.end_headers()
        self.wfile.write('finally done')
    elif self.path.endswith('/fastTemplateUrl'):
        self.send_response(200)
        self.send_header('Content-type', 'text/html')
        self.end_headers()
        self.wfile.write('fast template contents')
    elif self.path.endswith('/slowTemplateUrl'):
        sleep(2)
        self.send_response(200)
        self.send_header('Content-type', 'text/html')
        self.end_headers()
        self.wfile.write('slow template contents')
    else:
        SimpleHTTPServer.SimpleHTTPRequestHandler.do_GET(self)

Don't forget with the added sleep statements you need to include the time package

from time import sleep

otherwise several tests will fail.

Finally, let's move the test files over to the Selenium2Library test directory. Although this may not be necessary I do it to keep all the test files together. Ultimately I would like to see the Selenium2Library test directory moved into the src directory so the tests get distributed and then allow the test scripts for AngularJSLibrary be abe to be run from its own test directory. But for now we will combine them.

cp rf-ng/AngularJSLibrary/angular.robot rf-s2l/test/acceptance/locators/.
cp rf-ng/AngularJSLibrary/angular_wait.robot rf-s2l/test/acceptance/keywords/.

Directory Structure

So taking a step back and looking at the whole structure we should see the following directories

rf-s2l/
The source code for Robot Framework Selenium2Library.
rf-ng/
The source code for Robot Framework AngularJSLibrary.
ptor/
The source code for Robot Framework Seleniu2Library.

Within those directories we should see some modifications

rf-s2l/test/resources/testserver/testserver.py
A modified version of the test server containing the additional do_GET() method.
rf-s2l/test/acceptance/locators/angular.robot
AngularJSLibrary acceptance tests testing locators.
rf-s2l/test/acceptance/keywords/angular_wait.robot
AngularJSLibrary acceptance tests testing wait for angular functionality.

rf-s2l/test/resources/testapp/ng/async/async.html rf-s2l/test/resources/testapp/ng/async/async.js

A modified version of the async testapp page containing buttons which appear after the angular $timeouts and $http requests are completed.

And if we activate our virtual Python instance we should see

# pip list
decorator (4.0.10)
docutils (0.12)
pip (8.1.2)
robotframework (3.0)
selenium (2.53.6)
setuptools (8.2.1)

Note your versions may be different then mine listed here but key is you have installed robotframework and selenium packages and have not installed selenium2library as we will use the source code instead.

Starting the modified testserver

Open a new bash terminal from which we will run the test sever

cd test-ng

source clean-python27-env/bin/activate

cd rf-s2l

python test/resources/testserver/testserver.py start

You can test the server by navigating in a browser to

http://localhost:7000/testapp

Running the test scripts

In another terminal we will run the test scripts

cd ng

source clean-python27-env/bin/activate

cd rf-s2l

python test/run_tests.py python FF --suite acceptance.locators.angular --pythonpath ../rf-ng

python test/run_tests.py python FF --suite acceptance.keywords.angular_wait --pythonpath ../rf-ng

Note there is currently an issue with the Selenium2Library test runner script where if you specify a specific suite the output log and report files will not be created automatically. To get those files you can type

rebot -d test/results/ test/results/output.xml

Understanding how AngularJSLibrary works

It is important for you, the end user, to understand what is going on in the underlying library and there are many reasons for that. For one as I continue to develop this library I realize some initial assumptions and thus original implementations were simply wrong. I also have very narrow focus as my daily work focuses on a single (and usually older) version of AngularJS. So there could be issues I am not seeing and thus not addressing. These and many more reasons support the argument that as a library user we should all be well informed as to how the library works and what is Protractor / AngularJS doing in the functions we are mimicing.

Let's start off by examining the waitForAngular functionality in Protractor. At the core is this function (with some code removed) in ptor/lib/clientsidescripts.js

/**
 * Wait until Angular has finished rendering and has
 * no outstanding $http calls before continuing. The specific Angular app
 * is determined by the rootSelector.
 *
 * Asynchronous.
 *
 * @param {string} rootSelector The selector housing an ng-app
 * @param {function(string)} callback callback. If a failure occurs, it will
 *     be passed as a parameter.
 */
functions.waitForAngular = function(rootSelector, callback) {
  var el = document.querySelector(rootSelector);

  try {
    /* [SNIP] Newer vesions (which ones? not sure) there is a function for waiting. This
    one is off the window object. For now we will ignore this method and look at the original
    method for waiting...
    */
    /* [SNIP] Check to make sure we're on an angular page. */
    if (angular.getTestability) {
      /* [SNIP] Another function for waiting that comes from angular's testability api. */
    } else {
      /* Another check to verify we are within the ng-app. */

      angular.element(el).injector().get('$browser').
          notifyWhenNoOutstandingRequests(callback);

    }
  } catch (err) {
    callback(err.message);
  }
};

So striping out a lot of the code (see [SNIP]s above), the core is simply this

angular.element(el).injector().get('$browser').
    notifyWhenNoOutstandingRequests(callback);

a method which sounds like will give notification when there are no more outstanding requests or angular "actions". But what does callback do? What exactly does this method look like and how does one thus use it information? To answer what this looks like in practice we can use the testapp above. Start up the test server

cd ng

source clean-python27-env/bin/activate

cd rf-s2l

python test/resources/testserver/testserver.py start

In a browser navigate to

http://localhost:7000/testapp/ng1/alt_root_index.html#/async

[You'll see here I am using the angular1 portion of testapp. Also I am using the alt_root_index so I can hardcode which version of Angular1.x I'll want.] With the site running open the developers tools (F12) and in the console editor paste the following code, but before you run it let's tear it apart.

var callback = function () {console.log('*')}
var el = document.querySelector('#nested-ng-app');
var h = setInterval(function w4ng() {
    console.log('.');
    try {
        angular.element(el).injector().get('$browser').
            notifyWhenNoOutstandingRequests(callback);
    } catch (err) {
      console.log(err.message);
      callback(err.message);
    }
  }, 10);

You should see it is basically a call to setInterval which will continually call the function with a 10 ms delay each time till the interval is cleared. The function it is calling basically outputs a dot, '.', and calls the notifyWhenNoOutstandingRequests function from the waitForAngular passing along the callback. That callback will print out a star, '*', to the console. Want to take a guess as to what will happen when you run this code?

You will see a continual series of dots then stars printed to the console. Now on the async test page click the button label $timeout. Only dots are printed to the console for some time. Then only stars. What is happening at this time? When only the dots are outputed we are waiting for angular. More so, the callback that would print stars has not returned. And when just the stars are print, its all those callbacks returning while we were waiting for angular to complete. Go ahead and click on some of the other asyncrouous actions on the async page and see what the output is.

Note when you want to stop the output type the following line into the console to stop the continious interval call.

clearInterval(h);

So we can visualize the waiting for angular within javascript and from within the browser. We want, though, to not be in javascript (otherise we would just use Protrator and WebDriverJS) but in python. So let's do something similar with a simple python unittest.

import unittest
from selenium import webdriver

js_waiting_var="""
    var waiting = true;
    var callback = function () {waiting = false;}
    var el = document.querySelector('#nested-ng-app');
    angular.element(el).injector().get('$browser').
                notifyWhenNoOutstandingRequests(callback);
    return waiting;
"""


class ExecuteWaitForAngularTestCase(unittest.TestCase):

    def setUp(self):
        self.driver = webdriver.Firefox()

    def test_exe_javascript(self):
        driver = self.driver
        driver.get("http://localhost:7000/testapp/ng1/alt_root_index.html#/async")
        try:
            while (True):
                waiting = driver.execute_script(js_waiting_var)
                print('%s' % waiting)
        except KeyboardInterrupt:
            pass

    def tearDown(self):
        self.driver.close()

if __name__ == "__main__":
    unittest.main()

I went through a couple interations before settling on the above. Let me go through the syncronous javascript script. First, I like the simplicity of it. One iteration had a couple of calls to notifyWhenNoOutstandingRequests() with the (incorrect) thinking that I needed to ask twice to force the javascript execution stack to push through, if you will, the callback function. Remember, having the callback function return (with false) is the indication we are not waiting. But it turns out this not necessary as the function notifyWhenNoOutstandingRequests immediately calls the callback function if the outstanding request count is zero and thus sets the waiting flag to false. Summarizing, the javascript code sets the waiting flag to true stating we are waiting, calls notifyWhenNoOutstandingRequests and if not waiting sets the flag to false then returns the flag. So with a syncronous call we get back an immediate answers of the state of angular.

The use of a syncronous call by the AngularJSLibrary differs from other non-WebDriverJS ports of protractor. Almost all other ports use asyncronous javascript call. For this I don't understand [1]_. I understand why I choose a syncronious call but I don't see why asynchronous. So just as above I broke it down I tried to make an asycronous call to do the same. No luck. Then I did the second option, Google. [Note, this is the correct order. I tried something first and then tried Google. This is the best approach because it helps you to really think about the problem and not be trapped by the first answer that comes up.] So I tired Google and ... no luck. Some good resources but nothing worked as expected. Then I had the ah ha moment (which was really a duh moment) - Selenium test code!

The javascript tests can be found under py/test/selenium/webdriver/common/executing_async_javascript_tests.py. These async tests make more sense (to me at least) but don't give much depth to asyncronous javascript calls.

...[I think I need to finish this thought]...

Implicit Wait for Angular

As advertised on Protractor's homepage, Protractor "can automatically execute the next step in your test the moment the webpage finishes pending tasks, so you don’t have to worry about waiting for your test and webpage to sync." This implicit wait for angular functionality is implemented at couple points. First, as found in the ElementArrayFinder, "the first time [Protractor is] looking for an element". Second, as noted in protractor/lib/plugins.ts, "[b]etween every webdriver action, Protractor calls browser.waitForAngular() to make sure that Angular has no outstanding $http or $timeout calls." So whenever Protractor looks for an element [2]_ or whenever it makes a Selenium WebDriverJS library call it waits for angular thus fufilling the claim that you no longer need explicit waits. For the AngularJSLibrary then we will also want to wait when looking for an element or when calling a selenium method.

Interestingly enough, for the Selenium2Library when one makes a selenium call one is also looking for an element. This leads to a really slick (IMHO) solution for the Angular2Library. `Here it is<https://github.com/Selenium2Library/robotframework-angularjs/blob/master/AngularJSLibrary/__init__.py#L69>`_...

class ngElementFinder(ElementFinder):
    def __init__(self, ignore_implicit_angular_wait=False):
        super(ngElementFinder, self).__init__()
        self.ignore_implicit_angular_wait = ignore_implicit_angular_wait

    def find(self, browser, locator, tag=None):
        timeout = self._s2l.get_selenium_timeout()
        timeout = timestr_to_secs(timeout)

        if not self.ignore_implicit_angular_wait:
            try:
                WebDriverWait(self._s2l._current_browser(), timeout, 0.2)\
                    .until_not(lambda x: self._s2l._current_browser().execute_script(js_waiting_var))
            except TimeoutException:
                pass
        strategy = ElementFinder.find(self, browser, locator, tag=None)
        return strategy

Essentially we override the find method of Selenium2Library. So whenever you pass a locator to one of the Selenium2Library keywords you are calling, implicitly, wait for angular. One can see this in the Robot Framework log file when you have set loglevel to DEBUG. Here is the log file output when we click an element

KEYWORD Selenium2Library . Click Element model=show
Documentation:

Click element identified by `locator`.
Start / End / Elapsed:      20161112 11:45:37.794 / 20161112 11:45:37.917 / 00:00:00.123
11:45:37.794        INFO    Clicking element 'model=show'.
11:45:37.795        DEBUG   POST http://127.0.0.1:54972/hub/session/2d75d46c-de31-4a23-85d5-665234b73eb9/execute {"sessionId": "2d75d46c-de31-4a23-85d5-665234b73eb9", "args": [], "script": "\n var waiting = true;\n var callback = function () {waiting = false;}\n var el = document.querySelector('[ng-app]');\n if (typeof angular.element(el).injector() == \"undefined\") {\n throw new Error('root element ([ng-app]) has no injector.' +\n ' this may mean it is not inside ng-app.');\n }\n angular.element(el).injector().get('$browser').\n notifyWhenNoOutstandingRequests(callback);\n return waiting;\n"}
11:45:37.804        DEBUG   Finished Request
11:45:37.805        DEBUG   POST http://127.0.0.1:54972/hub/session/2d75d46c-de31-4a23-85d5-665234b73eb9/execute {"sessionId": "2d75d46c-de31-4a23-85d5-665234b73eb9", "args": [], "script": "return document.querySelectorAll('[ng-model=\"show\"]');"}
11:45:37.813        DEBUG   Finished Request
11:45:37.814        DEBUG   POST http://127.0.0.1:54972/hub/session/2d75d46c-de31-4a23-85d5-665234b73eb9/element/{087ef768-948b-4a41-ad41-422b49d3a143}/click {"sessionId": "2d75d46c-de31-4a23-85d5-665234b73eb9", "id": "{087ef768-948b-4a41-ad41-422b49d3a143}"}
11:45:37.916        DEBUG   Finished Request

The first POST is an execute javascript call where the javascript function is the internal wait for angular script. In this case Angular was not waiting and thus the next POST was a call to the find element; in this case a ng-model and another javascript call. One would see a similar call to the implicit wait for angular even if the locator strategy was an id, css, xpath or any other standard locator strategy. As compared to the above example here is the (truncated) output when there is a stack of unfufilled promises

KEYWORD Selenium2Library . Click Button css=[ng-click="slowAngularTimeoutHideButton()"]
Documentation:

Clicks a button identified by `locator`.
Start / End / Elapsed:      20161112 11:53:41.863 / 20161112 11:53:47.127 / 00:00:05.264
11:53:41.864        INFO    Clicking button 'css=[ng-click="slowAngularTimeoutHideButton()"]'.
11:53:41.865        DEBUG   POST http://127.0.0.1:59197/hub/session/2b715259-07c2-41d4-90a8-0fa97e271447/execute {"sessionId": "2b715259-07c2-41d4-90a8-0fa97e271447", "args": [], "script": "\n var waiting = true;\n var callback = function () {waiting = false;}\n var el = document.querySelector('[ng-app]');\n if (typeof angular.element(el).injector() == \"undefined\") {\n throw new Error('root element ([ng-app]) has no injector.' +\n ' this may mean it is not inside ng-app.');\n }\n angular.element(el).injector().get('$browser').\n notifyWhenNoOutstandingRequests(callback);\n return waiting;\n"}
11:53:41.879        DEBUG   Finished Request
11:53:42.080        DEBUG   POST http://127.0.0.1:59197/hub/session/2b715259-07c2-41d4-90a8-0fa97e271447/execute {"sessionId": "2b715259-07c2-41d4-90a8-0fa97e271447", "args": [], "script": "\n var waiting = true;\n var callback = function () {waiting = false;}\n var el = document.querySelector('[ng-app]');\n if (typeof angular.element(el).injector() == \"undefined\") {\n throw new Error('root element ([ng-app]) has no injector.' +\n ' this may mean it is not inside ng-app.');\n }\n angular.element(el).injector().get('$browser').\n notifyWhenNoOutstandingRequests(callback);\n return waiting;\n"}
11:53:42.096        DEBUG   Finished Request

...                 ...     ...

11:53:47.037        DEBUG   POST http://127.0.0.1:59197/hub/session/2b715259-07c2-41d4-90a8-0fa97e271447/execute {"sessionId": "2b715259-07c2-41d4-90a8-0fa97e271447", "args": [], "script": "\n var waiting = true;\n var callback = function () {waiting = false;}\n var el = document.querySelector('[ng-app]');\n if (typeof angular.element(el).injector() == \"undefined\") {\n throw new Error('root element ([ng-app]) has no injector.' +\n ' this may mean it is not inside ng-app.');\n }\n angular.element(el).injector().get('$browser').\n notifyWhenNoOutstandingRequests(callback);\n return waiting;\n"}
11:53:47.052        DEBUG   Finished Request
11:53:47.053        DEBUG   POST http://127.0.0.1:59197/hub/session/2b715259-07c2-41d4-90a8-0fa97e271447/elements {"using": "css selector", "sessionId": "2b715259-07c2-41d4-90a8-0fa97e271447", "value": "[ng-click=\"slowAngularTimeoutHideButton()\"]"}
11:53:47.058        DEBUG   Finished Request
11:53:47.059        DEBUG   POST http://127.0.0.1:59197/hub/session/2b715259-07c2-41d4-90a8-0fa97e271447/element/{e9c1e40c-74c7-44a8-801e-45151329fadc}/click {"sessionId": "2b715259-07c2-41d4-90a8-0fa97e271447", "id": "{e9c1e40c-74c7-44a8-801e-45151329fadc}"}
11:53:47.127        DEBUG   Finished Request

Note the time before and after the (...); about five seconds has passed. Here I truncated, so this printout is not so long, all the javascript calls asking angular if it has any outstanding promises. Eventually the promise have been fufilled and the script looks for an element and clicks it.

This DEBUG output comes from the internal AngularJSLibrary acceptance tests

Implicit Wait For Angular On Timeout
    Wait For Angular

    Click Button  css=[ng-click="slowAngularTimeout()"]

    Click Button  css=[ng-click="slowAngularTimeoutHideButton()"]

Implicit Wait For Angular On Timeout With Promise
    Wait For Angular

    Click Button  css=[ng-click="slowAngularTimeoutPromise()"]

    Click Button  css=[ng-click="slowAngularTimeoutPromiseHideButton()"]

To the Protractor testapp, I added some buttons

<li>
  <button ng-click="slowAngularTimeout()">$timeout</button>
  <span ng-bind="slowAngularTimeoutStatus"></span>
  <button ng-show="slowAngularTimeoutCompleted" ng-click="slowAngularTimeoutHideButton()">Hide</button>
</li>
<li>
  <button ng-click="slowAngularTimeoutPromise()">$timeout promise</button>
  <span ng-bind="slowAngularTimeoutPromiseStatus"></span>
  <button ng-show="slowAngularTimeoutPromiseCompleted" ng-click="slowAngularTimeoutPromiseHideButton()">Hide</button>
</li>
<li>
  <button ng-click="slowHttpPromise()">http promise</button>
  <span ng-bind="slowHttpPromiseStatus"></span>
  <button ng-show="slowHttpPromiseCompleted" ng-click="slowHttpPromiseHideButton()">Hide</button>
</li>

that will become visible when the "timeouts" are completed. As shown in the test above, the script clicks both buttons in succession without any explicit delay in the script. This provides us a good test suite to validate the implicit wait for angular. I also added a function to re-hide the button so the tests can be reset. One more test allows us the ability to validate this click the two buttons without delay will fail if we ignore the implicit wait for angular

Toggle Implicit Wait For Angular Flag
    Element Should Not Be Visible  css=[ng-click="slowAngularTimeoutHideButton()"]

    Set Ignore Implicit Angular Wait  ${true}

    Click Button  css=[ng-click="slowAngularTimeout()"]

    Run Keyword And Expect Error  *  Click Button  css=[ng-click="slowAngularTimeoutHideButton()"]

    Wait For Angular
    Element Should Be Visible  css=[ng-click="slowAngularTimeoutHideButton()"]
    Click Element  css=[ng-click="slowAngularTimeoutHideButton()"]
    Element Should Not Be Visible  css=[ng-click="slowAngularTimeoutHideButton()"]

    Set Ignore Implicit Angular Wait  ${false}

    Click Button  css=[ng-click="slowAngularTimeout()"]

    Click Button  css=[ng-click="slowAngularTimeoutHideButton()"]

    Element Should Not Be Visible  css=[ng-click="slowAngularTimeoutHideButton()"]

Angular 2

Looking at filling in the gap of Angular 2 support. Taking a look at the the current state of Protractor the `waitForAngular function<https://github.com/angular/protractor/blob/33393cad633e6cb5ce64b3fc8fa5e8a9cae64edd/lib/clientsidescripts.js#L135>`_ has some code to handle both Angular 1 and Angular 2+ code. Taking this Protractor code and combining it with test javascript code above (where we tested tthe core check printing out only '.' while Angular is busy) we have some asemblance of the Angular 1 and Angular 2+ support.

var callback = function () {console.log('*')};
var el = document.querySelector('[ng-app]');
var h = setInterval(function w4ng() {
    console.log('.');
    try {
        if (window.angular && !(window.angular.version &&
              window.angular.version.major > 1)) {
          /* ng1 */
          angular.element(el).injector().get('$browser').
              notifyWhenNoOutstandingRequests(callback);
        } else if (window.angular.getTestability) {
          window.angular.getTestability(el).whenStable(callback);
        }
    } catch (err) {
      console.log(err.message);
      callback(err.message);
    }
  }, 10);

Some important notes on running this script. Since I wrote the above portions of this write-up Firebug has ceased development and it has been combined with Firefox's developer tools. Under Firefox 53 (my current version) console.log when used within the console prompt no longer outputs to the console. [Yes it returns 'undefined' which is well explained out there and is perfectly valid but not very user friendly]. Chrome on the other hand does. So for now you will need to run the above code in the console within Chrome's dev tools. The other issue is a matter of the getTestibility function and its parent object. It appears that with the Angular development this method has been moved in the object tree and renamed. Under Protractor this function is now window.getAngularTestability. While investigating I was using several test sites. The testapp within Protractor does has a Angular 2 version although greatly simplified over the Angular 1 version. Due to some complications of Chrome, running on a VM, limited RAM, building the ng2 testapp with node, etc. I simplified my investigation by using other test sites. I tried angular.io's `tutorial example<https://angular.io/resources/live-examples/toh-6/ts/eplnkr.html >`_ but was slightly problimatic. It also is Angular ver 1.6.3 ?!? which isn't very helpful. I settled upon simply the angular.io site - although ... I am realizing many of these Angular site are still Angular 1.

Ok, this may explain the difference between window.angular.getTestability and window.getAngularTestability. The prior was introduced and available a while back and back in the Angular 1. I was simply lazy in that for my work the notifyWhenNoOutstandingRequests was sufficient. It could be that window.getAngularTestability is purely Angular 2+. ... [Researching] ... Ok form a very brief look, ok one site, it looks like this is the case. Let me put forth what I am thinking

var callback = function () {console.log('*')};
var el = document.querySelector('[ng-app]');
var h = setInterval(function w4ng() {
    console.log('.');
    try {
        if (window.angular && !(window.angular.version &&
              window.angular.version.major > 1)) {
          /* ng1 */
          angular.element(el).injector().get('$browser').
              notifyWhenNoOutstandingRequests(callback);
        } else if (window.getAngularTestability) {
          window.getAngularTestability(el).whenStable(callback);
        } else if (window.getAllAngularTestabilities) {
          var testabilities = window.getAllAngularTestabilities();
          var count = testabilities.length;
          var decrement = function() {
            count--;
            if (count === 0) {
              callback();
            }
          };
          testabilities.forEach(function(testability) {
            testability.whenStable(decrement);
          });
        } else if (!window.angular) {
          throw new Error('window.angular is undefined.  This could be either ' +
              'because this is a non-angular page or because your test involves ' +
              'client-side navigation. Currently the AngularJS Library is not ' +
              'designed to wait in such situations. Instead you should explicitly ' +
              'call the \'Wait For Angular\' keyword.');
        } else if (window.angular.version >= 2) {
          throw new Error('You appear to be using angular, but window.' +
              'getAngularTestability was never set.  This may be due to bad ' +
              'obfuscation.');
        } else {
          throw new Error('Cannot get testability API for unknown angular ' +
              'version "' + window.angular.version + '"');
        }
    } catch (err) {
      console.log(err.message);
      callback(err.message);
    }
  }, 10);

which one could compared with the full Protractor code

if (window.angular && !(window.angular.version &&
      window.angular.version.major > 1)) {
  /* ng1 */
  var hooks = getNg1Hooks(rootSelector);
  if (hooks.$$testability) {
    hooks.$$testability.whenStable(callback);
  } else if (hooks.$injector) {
    hooks.$injector.get('$browser').
        notifyWhenNoOutstandingRequests(callback);
  } else if (!!rootSelector) {
    throw new Error('Could not automatically find injector on page: "' +
        window.location.toString() + '".  Consider using config.rootEl');
  } else {
    throw new Error('root element (' + rootSelector + ') has no injector.' +
       ' this may mean it is not inside ng-app.');
  }
} else if (rootSelector && window.getAngularTestability) {
  var el = document.querySelector(rootSelector);
  window.getAngularTestability(el).whenStable(callback);
} else if (window.getAllAngularTestabilities) {
  var testabilities = window.getAllAngularTestabilities();
  var count = testabilities.length;
  var decrement = function() {
    count--;
    if (count === 0) {
      callback();
    }
  };
  testabilities.forEach(function(testability) {
    testability.whenStable(decrement);
  });
} else if (!window.angular) {
  throw new Error('window.angular is undefined.  This could be either ' +
      'because this is a non-angular page or because your test involves ' +
      'client-side navigation, which can interfere with Protractor\'s ' +
      'bootstrapping.  See http://git.io/v4gXM for details');
} else if (window.angular.version >= 2) {
  throw new Error('You appear to be using angular, but window.' +
      'getAngularTestability was never set.  This may be due to bad ' +
      'obfuscation.');
} else {
  throw new Error('Cannot get testability API for unknown angular ' +
      'version "' + window.angular.version + '"');
}

The biggest difference is the simplification of the Angular 1 code. I could simply add the window.angular.getTestability check. For now I am going to move forward with this and then we can revisit this code.

Execute Async vs Sync Script

So I am coming around to revisiting my decision to use the syncronous javascript call. Reviewing my reasons again was 1) it worked - that is, syncronous worked where my initial attempts with asycronous failed and then 2) I wanted to control loop (i.e. the eventual timeout) to be within Python and 3) I understood what it was doing and, well, it worked. As previously noted I grasped the basic concept of asycronous javascript but I couldn't envision it from the Python call side nor was I fully understanding the implementation of the callback. Since then several events are pushing me forward with, at least, trying out an asyncronous version. Those events include a good conversation with the very wise Marvin Ojwang at the spring 2017 Selenium Conference and some issues I have had with an in-house Angluar sites. Also I have been thinking about the python async call and think I have a better understanding as well as a way of actually showing the async call work with python.

For the async demo I need as the core a Javascript "sleep" command. I am using the solution provide by Dan Dascalescu `in this stackoverflow posting<https://stackoverflow.com/questions/951021/what-is-the-javascript-version-of-sleep>`_. Note this will work for latest version of Edge, Firefox, and Chrome but not IE11. Also note if you run this within the developer tools (F12) console you will not get the console.log output. I must admit this is both frustrating and, although technical correct, absolutly wrong and useless. If someone wants to log let them log whether from within the console itself or from an internal script or even a selenium execute script call. That sai here is the javascript

function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

async function demo() {
  console.log('Taking a break...');
  await sleep(2000);
  console.log('Two second later');
}

demo();

If you really want to run this in the console window you can change console.log to alert. Just remember that you need to dismiss the alert dialog to complete the initial 'Taking a call...' statement and have the script procede. As we really want to work within Python let's move this there...

>>> from selenium import webdriver
>>> driver = webdriver.Firefox()
>>> js="""function sleep(ms){return new Promise(resolve=>setTimeout(resolve,ms))}
... async function demo(){console.log('Taking a break...');await sleep(2000);console.log('Two second later')}
... demo()"""
>>> driver.execute_script(js)
>>>

Now I know this is not the form I want to create for an asyncronous call but let's try this out anyways and see what happens...

>>> driver.execute_async_script(js)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/emanlove/angular/clean-python27-env/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 553, in execute_async_script
    'args': converted_args})['value']
  File "/home/emanlove/angular/clean-python27-env/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 297, in execute
    self.error_handler.check_response(response)
  File "/home/emanlove/angular/clean-python27-env/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Timed out

>>>

Well it works and then ... we seemingly don't return from the method .. till we get the Timed out message. Ok this is good for a couple reasons. One it should be expected as, first, there is no return (and no callback) and second I need to eventually work out and explain how an internal library timeout interacts with the selenium timeouts. So this is confirmation and a reminder. For kicks and curiosity I am going to try a return value. I'll note I really think the issue with the async script is not the lack of a return value but the missing callback. Changing the script to

>>> js_w_return="""function sleep(ms){return new Promise(resolve=>setTimeout(resolve,ms))}
... async function demo(){console.log('Taking a break...');await sleep(2000);console.log('Two second later')}
... demo();
... return undefined;"""
>>> val=driver.execute_script(js_w_return)
>>> val is None
True
>>> driver.execute_async_script(js_w_return)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/emanlove/angular/clean-python27-env/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 553, in execute_async_script
    'args': converted_args})['value']
  File "/home/emanlove/angular/clean-python27-env/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 297, in execute
    self.error_handler.check_response(response)
  File "/home/emanlove/angular/clean-python27-env/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Timed out

>>>

A few observations. The return value had no effect on the async call. Also, not noticed before nor called out but the sync script returns immediately while the script is still running within the test browser. Yes expected, but still interesting none the less. The last observation was that undefined value in Javascript translates to None within Python. I hadn't really thought about this till I didn't see a return value which I'll admit was a little unexpected here. I wanted to confirm I got a return value and the python interpretor on return None does "nothing". I modified the script to return a Javascript false instead,

js_w_bool="""function sleep(ms){return new Promise(resolve=>setTimeout(resolve,ms))}
async function demo(){console.log('Taking a break...');await sleep(2000);console.log('Two second later')}
demo();
return false;"""

saw False return value, and then went back and tested the undefined val against None. Minor observation but still important in confirming my understanding and observations.

Thinking I recalled that the last arguement is supposed to be the callback function which will be called when the async javascript script is completed, I tried something like

>>> js="""function sleep(ms){return new Promise(resolve=>setTimeout(resolve,ms))}
... async function demo(){console.log('Taking a break...');await sleep(2000);console.log('Two second later')}
... demo();"""
>>> val=driver.execute_script(js,"console.log(); return undefined;")

But providing what I thought was the callback function didn't work so I went back to the Selenium unit tests. Here we see, as an example, this async call

>>> driver.execute_async_script("arguments[arguments.length - 1](123);")
123

noting that this function returns without the time exception which will be noteworthy here shortly. Trying to resolve where my thinking about execute_async_script came from I re-disovered `this stackoverflow post<https://stackoverflow.com/a/29520344>`_ which i think was the source for my statement "the last arguement is supposed to be the callback function which will be called when the async javascript script is completed". Honestly, re-reading this post just confuses me again. So I have a basic understanding as arguments and we actually use them in the library because certain characters cause issues within a script and pone solution is to pass them as arguments. Wanting to understand this beeter and thinking about the selenium unit test above I tried the following

>>> driver.execute_async_script("console.log(arguments.length)")
1
>>> driver.execute_async_script("console.log(arguments[0])")
function ()
>>> driver.execute_async_script("console.log(arguments[0].toSource())")
rv => __webDriverCallback(rv)
>>> driver.execute_async_script("console.log(arguments[0].toString())")
rv => __webDriverCallback(rv)
>>>

All of these timed out, by the way which was surprising and interesting. Also the response for the toSouce/toDString function is interesting. What gets me confused about the stackflow article is that I have been thinking of the execute_async_script as the javascript function which may be incorrect. The arguments, the javascript scripts that are been passed into the python method, are not going into a execute_async_script function but may be run within some other function. Trying

>>> driver.execute_async_script("var funcName=arguments.callee.toString(); console.log(funcName)")

results in the following console output

This just re-enforces my confusion. "Without an additional argument passed to the python execute_async_script method, Argument[0] is the javascript function itself one wishes to execute. Thus statements like driver.execute_async_script("arguments[arguments.length - 1](123);") are just some sort of bad circlular function call."

Funny. As I explore this more I tried the following

>>> driver.execute_async_script("var funcName=arguments[0].callee.toString(); console.log(funcName)")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/emanlove/angular/clean-python27-env/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 553, in execute_async_script
    'args': converted_args})['value']
  File "/home/emanlove/angular/clean-python27-env/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 297, in execute
    self.error_handler.check_response(response)
  File "/home/emanlove/angular/clean-python27-env/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: TypeError: arguments[0].callee is undefined

>>> driver.execute_async_script("var funcName=arguments[0]().callee.toString(); console.log(funcName)")
>>>

The first call above throws an error, as expected, because callee is not is not a function of the first agrument but of arguments. Arguments[0](), on the other hand, is a function and thus should have the callee child object. But when we make this second call the function returns without a timeout and the console log has no output. See a bad recursive function call!

I had a chance to walk away an forget about this question for a short time. Coming back I am asking whether I can essentially fit the async example into the javascript sleep example. Here is code that says yes we can,

>>> from selenium import webdriver
>>> driver=webdriver.Firefox()
>>> js="""var done = arguments[0];
... function sleep(ms){return new Promise(resolve=>setTimeout(resolve,ms))}
... async function demo(){console.log('Taking a break...');await sleep(2000);done('Two second later')}
... demo()"""
>>> driver.execute_async_script(js)
u'Two second later'
>>>

Jumping over to our specific goal of asking Angular whether or not it is "busy", trying ...

>>> from selenium import webdriver
>>> driver=webdriver.Firefox()
>>> driver.get("http://angular.github.io/angular-phonecat/step-14/app/#!/phones")
>>> js_wait_for_angularjs = """
...     var callback = arguments[0];
...     var el = document.querySelector('[ng-app]');
...     if (typeof angular.element(el).injector() == "undefined") {
...         throw new Error('root element ([ng-app]) has no injector.' +
...                ' this may mean it is not inside ng-app.');
...     }
...     angular.element(el).injector().get('$browser').
...                 notifyWhenNoOutstandingRequests(callback);
... """
>>> driver.execute_async_script(js_wait_for_angularjs)
>>> el=driver.find_elements_by_xpath("//select")[0]
>>> sel=webdriver.support.select.Select(el)
>>> sel.select_by_value('age');driver.execute_async_script(js_wait_for_angularjs);
>>> sel.select_by_value('name');driver.execute_async_script(js_wait_for_angularjs);
>>> sel.select_by_value('age');driver.execute_async_script(js_wait_for_angularjs);
>>>

These last few lines give us a very unscientific but visual sanity check making sure that we are indeed waiting for angular to complete.

[... more to come... Need to implement an all javascript version wait for angular up to some give up timeout. Also want to talk the merits of both async and sync solutions and demostrate a very busy angular/javascript test case and show what happens when a blocking call is made.]

One question I have is with the model above can we send back some value in the sense that we are not wanting to call the arguments[0](some_return_var) function but instead pass it as the callback and when the notifyWhenNoOutstandingRequests function completes then call the callback and return the value.

Footnotes

[1] Ok, not entirely true. I understand WebDriverJS and Protractor is asycronious javascript and thus when one makes a call to a function you may get the response back in some unknown amount of time or asycronously. Fine. But that is not what I want here. I want an answer back now telling me whether or not the current (now) state of Angular has any outstanding promises or whether or not it is waiting, right now. Don't delay. Thus I am making, conscientiously, a syncronous javascript call and then "waiting" or polling within the AngularJSlibrary.

[2] The statement that "whenever Protractor looks for an element" may not be entitrely true. If you read the code the waitForAngular call is when you are looking for "all" elements, as in element.all(by.id('notPresentElementID')). [See `protractor/lib/element.ts<https://github.com/angular/protractor/blob/6b7b6fb751f574056cb80b7238f62c77ef78497e/lib/element.ts#L168>`_]. It is unclear, to me without spending a lot more time tracing through the code atleast, that a call to say element(by.binding('username')), for example, would go through element.all and thus be invoking the implicit wait for angular.