Add SELENIUM_RUN_ON_FAILURE-variable to support custom keyword to be …

…executed at failing step; Move Debug keyword from keywords.robot to selenium.robot and refactor it to stop the test with first working approach; Add new script bin/robot-debug to run robot with SELENIUM_RUN_ON_FAILURE=Debug to automatically stop tests on failure [fixes #21]

docs/source/debugging.rst

@@ -5,20 +5,36 @@ It's not always so easy to get the used Selenium keywords right. There are
 a few ways to pause the test runner in middle of a test to ease figuring out
 what to do next:
 
-1. Use interactive `robotframework-debuglibrary`_ with *Debug*-keyword'
-   (requires that **plone.app.robotframework** is required with **[debug]**
-   extras and the used python is compiled with readline-support):
+1. Set the variable ``SELENIUM_RUN_ON_FAILURE`` to use the Debug-keyword
+   provided in ``selenium.robot`` resource file (which is included in all
+   test suites relying on **plone.app.robotframework**, e.g. with:
 
-      *** Settings ***
+   .. code-block:: bash
 
-      Resource  plone/app/robotframework/keywords.robot
+      $ ROBOT_SELENIUM_RUN_ON_FAILURE=Debug bin/test -t robot
+
+   Or when testing against robot-server, just run your test suite with provided
+   script:
+
+   .. code-block:: bash
+
+      $ bin/robot-debug src/path/to/my/test.robot
+
+   This will stop the test automatically at the first failing step with the
+   first working approach listed also below.
+
+2. Use interactive `robotframework-debuglibrary`_ with *Debug*-keyword'
+   (requires that the used python is compiled with readline-support):
+
+   .. code-block:: robotframework
 
       *** Test Cases ***
 
-      Start interactive debugger with included Debug-keyword
+      Start interactive debugger with Debug-keyword from DebugLibrary
+          Import library  DebugLibrary
           Debug
 
-2. Pause Selenium (WebDriver) completely to inspect your step with
+3. Pause Selenium (WebDriver) completely to inspect your step with
    *Pause execution* keywords from *Dialogs*-library shipped with
    Robot Framework:
 
@@ -44,7 +60,7 @@ what to do next:
       Pause tests with included Pause-keyword
          Pause
 
-3. Let Selenium (WebDriver) sleep for long time:
+4. Let Selenium (WebDriver) sleep for long time:
 
    .. code-block:: robotframework
 
@@ -53,15 +69,15 @@ what to do next:
       Pause test with non-interactive (and auto-continuing) sleep
          Sleep  10 min
 
-4. Slow down Selenium (WebDriver) to make the tests easier to follow:
+5. Slow down Selenium (WebDriver) to make the tests easier to follow:
 
    .. code-block:: robotframework
 
       *** Settings ***
 
       Suite setup  Set Selenium speed  0.5s
 
-5. Use provided Python keyword to drop Zope server (or Robot Framework
+6. Use provided Python keyword to drop Zope server (or Robot Framework
    test runner) into debugger:
 
    .. code-block:: robotframework
@@ -72,7 +88,7 @@ what to do next:
            Import library  plone.app.robotframework.Debugging
            Stop
 
-6. Write a custom python keyword into your custom Python keyword library
+7. Write a custom python keyword into your custom Python keyword library
    to drop Zope server (or Robot Framework test runner) into debugger.
 
    But there's one catch in debugging your code while running Robot Framework

setup.py

@@ -18,6 +18,7 @@
 console_scripts = [
     "robot-server = plone.app.robotframework.server:server",
     "robot = plone.app.robotframework.robotentrypoints:robot",
+    "robot-debug = plone.app.robotframework.robotentrypoints:robot_debug",
     "pybot = plone.app.robotframework.robotentrypoints:pybot",
     "ride = plone.app.robotframework.robotentrypoints:ride",
     "libdoc = plone.app.robotframework.robotentrypoints:libdoc",

src/plone/app/robotframework/keywords.robot

@@ -14,17 +14,6 @@ Pause
     Import library  Dialogs
     Pause execution
 
-
-Debug
-    [Documentation]  Pause test execution with interactive debugger (REPL)
-    ...              in the current shell.
-    ...
-    ...              This keyword is based on ``robotframework-debuglibrary``
-    ...              and requires that the used Python is compiled with
-    ...              ``readline``-support.
-    Import library  DebugLibrary  WITH NAME  DebugLibrary
-    DebugLibrary.Debug
-
 # ----------------------------------------------------------------------------
 # Access Resources
 # ----------------------------------------------------------------------------

src/plone/app/robotframework/robotentrypoints.py

@@ -35,6 +35,12 @@ def robot():
             + sys.argv[1:])
 
 
+def robot_debug():
+    run_cli(['--listener', 'plone.app.robotframework.RobotListener',
+             '-v', 'SELENIUM_RUN_ON_FAILURE:Debug']
+            + sys.argv[1:])
+
+
 def ride():
     if HAS_RIDE:
         from robotide import main

src/plone/app/robotframework/selenium.robot

@@ -2,6 +2,7 @@
 
 Library  Selenium2Library  timeout=${SELENIUM_TIMEOUT}
 ...                        implicit_wait=${SELENIUM_IMPLICIT_WAIT}
+...                        run_on_failure=${SELENIUM_RUN_ON_FAILURE}
 
 Resource  variables.robot
 Resource  ${CMFPLONE_SELECTORS}
@@ -10,6 +11,7 @@ Resource  ${CMFPLONE_SELECTORS}
 
 ${SELENIUM_IMPLICIT_WAIT}  0.5
 ${SELENIUM_TIMEOUT}  30
+${SELENIUM_RUN_ON_FAILURE}  Capture Page Screenshot
 
 ${BROWSER}  Firefox
 ${REMOTE_URL}
@@ -18,6 +20,37 @@ ${DESIRED_CAPABILITIES}
 
 *** Keywords ***
 
+Debug
+    [Documentation]  Pause test execution for test debugging purposes.
+    ...
+    ...              When DebugLibrary is found, pauses test execution
+    ...              with interactive robotframework debugger (REPL)
+    ...              in the current shell. This s based on
+    ...              ``robotframework-debuglibrary`` and requires that the
+    ...              used Python is compiled with ``readline``-support.
+    ...
+    ...              When DebugLibrary is NOT found, pauses test execution
+    ...              with Dialogs-library's Pause Execution library, which
+    ...              requires that the used Python is compiled with TkInter
+    ...              support.
+    ...
+    ...              When Dialogs-library cannot be imported, pauses test
+    ...              execution with interactive Python debugger (REPL)
+    ...              in the current shell.
+    ${debug} =  Run keyword and ignore error
+    ...          Import library  DebugLibrary  WITH NAME  DebugLibrary
+    ${dialogs} =  Run keyword and ignore error
+    ...           Import library  Dialogs  WITH NAME  DialogsLibrary
+    ${fallback} =  Run keyword and ignore error
+    ...            Import library  plone.app.robotframework.keywords.Debugging
+    ...            WITH NAME  DebuggingLibrary
+    Run keyword if  ${debug}[0] == 'PASS'
+    ...             DebugLibrary.Debug
+    Run keyword if  ${debug}[0] == 'FAIL' and ${dialogs}[0] == 'PASS'
+    ...             DialogsLibrary.Pause Execution
+    Run keyword if  ${debug}[0] == 'FAIL' and ${dialogs}[0] == 'FAIL'
+    ...             DebuggingLibrary.Stop
+
 # ----------------------------------------------------------------------------
 # Browser
 # ----------------------------------------------------------------------------