Improvements to documentation and CL interface

- Improved README - Made main options configurable using Behave userdata options - Using Chrome as default driver - Added AM-version-specific tag
artefactual-labs · Apr 4, 2017 · 3777b1c · 3777b1c
1 parent eab5dfc
commit 3777b1c
Show file tree

Hide file tree

Showing 7 changed files with 222 additions and 85 deletions.
diff --git a/README.rst b/README.rst
@@ -2,43 +2,149 @@
   Archivematica Acceptance Tests
 ================================================================================
 
-This respository contains acceptance tests for Archivematica written using
-Python behave and the Gherkin language.
+Acceptance tests for Archivematica_ (AM) written using Python behave_ and the
+Gherkin_ language. Deploying an Archivematica system to test against is a
+necessary separate step. The tests use Selenium_ to launch a browser in order to
+interact with Archivematica's web GUI. (They also make vanilla requests to AM's
+API using Python's Requests_ library). They have been run successfully with
+Firefox and Chrome, and in CI scenarios using Xvfb_ (X virtual framebuffer).
+
+Using Gherkin to express tests makes them quite readable to non-programmers.
+Consider the following snippet from the premis-events.feature file::
+
+    Feature: PREMIS events are recorded correctly
+      Users of Archivematica want to be sure that the steps taken by
+      Archivematica are recorded correctly in the resulting METS file, according
+      to the PREMIS specification.
+
+      Scenario: Isla wants to confirm that standard PREMIS events are created
+      Given that the user has ensured that the default processing config is in its default state
+      When a transfer is initiated on directory ~/archivematica-sampledata/SampleTransfers/BagTransfer
+      Then in the METS file there are/is 7 PREMIS event(s) of type ingestion
+
+The ``Given``, ``When`` and ``Then`` statements in the .feature files are
+implemented by "step" functions in the features/steps/steps.py file, which, in
+turn, may interact with Archivematica GUI(s) by calling methods of an
+``archivematicaselenium.py::ArchivematicaSelenium`` instance.
 
 
 Installation
 ================================================================================
 
 Create a virualenv using Python 3 and activate it::
 
-    $ sudo virtualenv -p python3 env
+    $ virtualenv -p python3 env
     $ source env/bin/activate
 
-Install the dependencies::
+Clone the source (either to the same machine where Archivematica is installed,
+or to another)::
+
+    $ git clone https://github.com/artefactual-labs/archivematica-acceptance-tests.git
+
+Since lxml is a dependency, you may need to install python3-dev. On Ubuntu
+14.04 with Python 3::
+
+    $ sudo apt-get install python3-dev
+
+Install the Python dependencies::
 
     $ pip install -r requirements.txt
 
+One way to run the tests headless, i.e., without a visible browser, is with
+Xvfb. To install Xvfb on Ubuntu 14.04::
+
+    sudo apt-get update
+    sudo apt-get install -y xorg xvfb dbus-x11 xfonts-100dpi xfonts-75dpi xfonts-cyrillic
+
+See also:
+
+- http://stackoverflow.com/questions/34548472/trying-to-configure-xvfb-to-run-firefox-headlessly
+- http://elementalselenium.com/tips/38-headless
+
+A browser (Chrome or Firefox) must be installed on the system where the tests
+are being run; see below. On a dev or CI server, this may require installation.
+
+
+Install Chrome on Ubuntu 14.04
+--------------------------------------------------------------------------------
+
+Following the instructions from
+http://askubuntu.com/questions/510056/how-to-install-google-chrome::
+
+    $ wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - 
+    $ sudo sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list'
+    $ sudo apt-get update
+    $ sudo apt-get install google-chrome-stable
+    $ google-chrome --version
+    Google Chrome 57.0.2987.133
+
+Install chromedriver following the instructions at
+https://christopher.su/2015/selenium-chromedriver-ubuntu/::
+
+    wget -N http://chromedriver.storage.googleapis.com/2.26/chromedriver_linux64.zip
+    unzip chromedriver_linux64.zip
+    chmod +x chromedriver
+    sudo mv -f chromedriver /usr/local/share/chromedriver
+    sudo ln -s /usr/local/share/chromedriver /usr/local/bin/chromedriver
+    sudo ln -s /usr/local/share/chromedriver /usr/bin/chromedriver
+
+
+Install a specific Firefox version on Ubuntu 14.04
+--------------------------------------------------------------------------------
+
+We have had variable success running these tests on various versions of
+Firefox. It may be necessary to purge an existing Firefox and install an older
+version. We have had some success with Firefox v. 47 and provide instructions
+for installing that on Ubuntu 14.04 here::
+
+    $ firefox -v
+    Mozilla Firefox 48.0
+    $ sudo apt-get purge firefox
+    $ wget sourceforge.net/projects/ubuntuzilla/files/mozilla/apt/pool/main/f/firefox-mozilla-build/firefox-mozilla-build_47.0.1-0ubuntu1_amd64.deb
+    $ sudo dpkg -i firefox-mozilla-build_47.0.1-0ubuntu1_amd64.deb 
+    $ firefox -v
+    Mozilla Firefox 47.0.1
+
+
+Troubleshooting
+================================================================================
+
+If the tests generate ``cannot allocate memory`` errors, there may be unclosed
+browsers. Run the following command to look for persistent firefox or chrome
+browsers and kill them::
+
+    $ ps --sort -rss -eo rss,pid,command | head
+
 
 Usage
 ================================================================================
 
-To run the tests::
+Basic usage::
 
     $ behave
 
-If a feature file contains tagged scenarios or scenario outlines (i.e., those
-with something like ``@tag-name`` above them), then you can run just those
-scenarious by passing the ``--tags`` argument to behave. For example, to just
-run the metadata-only AIP re-ingest tests, run::
+The above will launch many annoying browser windows. Use Xvfb to hide all that
+rubbish. Start Xvfb on display port 42 and background the process::
+
+    $ Xvfb :42 &
+
+Tell the terminal session to use the display port::
+
+    $ export DISPLAY=:42
 
-    $ behave --tags=mo-aip-reingest
+Run the tests, this time just those targetting the correct creation of PREMIS
+events::
 
-There are also two convenience scripts for closing all transfers and closing
-all ingests via the GUI (i.e., using Selenium)::
+    $ behave --tags=premis-events --tags=standard --no-skipped
 
-    $ python close_all_transfers.py
-    $ python close_all_ingests.py
+The scenarios in the .feature files may be tagged with zero or more tags. The
+above command runs all scenarios tagged ``@premis-events`` and ``@standard``.
 
+There are two convenience scripts for closing all transfers and closing all
+ingests via the GUI (i.e., using Selenium)::
+
+    $ ./close_all_transfers.py
+    $ ./close_all_ingests.py
 
 
 Configuration
@@ -47,30 +153,54 @@ Configuration
 Install a Compatible Archivematica System
 --------------------------------------------------------------------------------
 
-These tests are a work in progress and some of them require specific
-Archivematica configurations. In particular, the tests encoded in the feature
-files that mention "conformance" and "policy" require the latest
-MediaConch-integration related branch of Archivematica
-(dev/issue-10133-ingest-policy-check-good at the time of writing).
-
+The tests require access to a live Archivematica installation. The tests tagged
+``am16`` should pass against Archivematica version 1.6. Those tagged ``dev``
+require specific development branches to be installed, e.g., ``dev`` tests also
+tagged with ``preforma`` require AM at branch dev/issue-9478-preforma. Such
+dependencies should be indicated in the comments of the relevant .feature files.
 
-The deploy-pub ansible playbook set at
-https://github.com/jrwdunham/deploy-pub/tree/dev/issue-9478-acceptance-tests-preforma
-should allow you to install such a system. Assuming you have VirtualBox,
-Vagrant and Ansible installed, use the following instructions::
+Archivematica is most easily installed using the deploy-pub ansible playbook
+set at
+https://github.com/artefactual/deploy-pub.git
+Assuming you have VirtualBox, Vagrant and Ansible installed, here is the
+quickstart::
 
-    $ git clone https://github.com/jrwdunham/deploy-pub.git
-    $ cd deploy-pub
-    $ git checkout dev/issue-9478-acceptance-tests-preforma
-    $ cd playbooks/archivematica
+    $ git clone https://github.com/artefactual/deploy-pub.git
+    $ cd deploy-pub/playbooks/archivematica
     $ ansible-galaxy install -f -p roles/ -r requirements.yml
     $ vagrant up
 
-The tests also assume that you have configured your Archivematica installation
-so that it is being served at the following URL and has an administrator-level
-user with the following username and password. These values can be altered in
-features/environment.py.
 
-- URL:      http://192.168.168.192/
-- username: test
-- password: testtest
+Configuration via features/environment.py or Behave userdata options
+--------------------------------------------------------------------------------
+
+The tests assume by default that you have configured your Archivematica
+installation to be served at a specific URL, viz. http://192.168.168.192/.
+The tests should be able to detect a fresh AM install, in which case they will
+create an administrator-level user with username ``test`` and
+password ``testtest``. These and other configuration options can be overridden
+by altering the following constants in features/environment.py...::
+
+- ``AM_URL``
+- ``AM_USERNAME``
+- ``AM_PASSWORD``
+- ``SS_URL``
+- ``SS_USERNAME``
+- ``SS_PASSWORD``
+- ``TRANSFER_SOURCE_PATH``
+- ``HOME``
+- ``DRIVER_NAME``
+
+... or by passing the equivalent lowercased parameters as Behave "userdata"
+options. For example, the following would run the tests against an
+Archivematica instance at 123.456.123.456 using the Firefox driver::
+
+    $ behave -D am_url=123.456.123.456 -D driver_name=Firefox
+
+
+.. _Archivematica: https://github.com/artefactual/archivematica
+.. _behave: http://pythonhosted.org/behave/
+.. _Gherkin: https://github.com/cucumber/cucumber/wiki/Gherkin
+.. _Selenium: http://www.seleniumhq.org/
+.. _Requests: http://docs.python-requests.org/en/master/
+.. _Xvfb: https://www.x.org/archive/X11R7.6/doc/man/man1/Xvfb.1.xhtml
diff --git a/archivematicaselenium.py b/archivematicaselenium.py
@@ -56,6 +56,11 @@ def test_feature(self):
        Selenium 2.53.6
        Python 3.4.2
 
+    4. Chrome 56.0.2924.87 (64-bit)
+       Ubuntu 16.04
+       Selenium 2.53.6
+       Python 3.5.2
+
 WARNING: this will *not* currently work with a headless PhantomJS() webdriver.
 With PhantomJS, it can login, but when it attempts to use the interface for
 selecting a transfer folder it times out when waiting for the 'home' folder to
@@ -110,6 +115,9 @@ def test_feature(self):
 DEFAULT_SS_USERNAME = 'test',
 DEFAULT_SS_PASSWORD = 'test',
 DEFAULT_SS_URL = 'http://192.168.168.192:8000/',
+DEFAULT_AM_API_KEY = None
+DEFAULT_SS_API_KEY = None
+DEFAULT_DRIVER_NAME = 'Chrome'  # 'Firefox' should also work.
 
 DUMMY_VAL = 'Archivematica Acceptance Test'
 METADATA_ATTRS = ('title', 'creator')
@@ -150,13 +158,9 @@ class ArchivematicaSeleniumError(Exception):
 
 
 class ArchivematicaSelenium:
-    """Selenium tests for MediaConch-related functionality in Archivematica.
-
-    TODOs:
+    """Convenience class for using Selenium to interact with a live
+    Archivematica instance.
 
-    1. Test in multiple different browser and platform combinations.
-    2. Run headless.
-    3. Fix issues: search for "TODO/WARNING"
     """
 
     # =========================================================================
@@ -170,17 +174,20 @@ def __init__(self,
              am_username=DEFAULT_AM_USERNAME,
              am_password=DEFAULT_AM_PASSWORD,
              am_url=DEFAULT_AM_URL,
-             am_api_key=None,
+             am_api_key=DEFAULT_AM_API_KEY,
              ss_username=DEFAULT_SS_USERNAME,
              ss_password=DEFAULT_SS_PASSWORD,
              ss_url=DEFAULT_SS_URL,
-             ss_api_key=None):
+             ss_api_key=DEFAULT_SS_API_KEY,
+             driver_name=DEFAULT_DRIVER_NAME
+             ):
         self.am_username = am_username
         self.am_password = am_password
         self.am_url = am_url
         self.am_api_key = am_api_key
         self.ss_username = ss_username
         self.ss_password = ss_password
+        self.driver_name = driver_name
         self.ss_url = ss_url
         self._ss_api_key = ss_api_key
         self._tmp_path = None
@@ -191,12 +198,6 @@ def __init__(self,
     # Test Infrastructure.
     # =========================================================================
 
-    # Valuate this to 'Firefox' or 'Chrome'. 'PhantomJS' will fail.
-    # Note/TODO: Chrome is currently failing on my machine because the
-    # transfers are not displaying their jobs/microservices.
-    driver_name = 'Firefox'
-    # driver_name = 'PhantomJS'
-
     all_drivers = []
 
     def get_driver(self):
@@ -210,7 +211,12 @@ def get_driver(self):
                  ' AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116'
                  ' Safari/537.36')
             return webdriver.PhantomJS(desired_capabilities=cap)
-        driver = getattr(webdriver, self.driver_name)()
+        elif self.driver_name == 'Chrome':
+            driver = webdriver.Chrome()
+            driver.set_window_size(1700, 900)
+        else:
+            driver = getattr(webdriver, self.driver_name)()
+        driver.set_script_timeout(10)
         self.all_drivers.append(driver)
         return driver
 
@@ -1780,7 +1786,7 @@ def search_for_fpr_rule(self, purpose, format, command_description):
                                         command_description)
         self.search_rules(search_term)
 
-    def ensure_fpr_rule_enabled(purpose, format, command_description):
+    def ensure_fpr_rule_enabled(self, purpose, format, command_description):
         self.navigate(self.get_rules_url())
         self.search_for_fpr_rule(purpose, format, command_description)
         info_el = self.driver.find_element_by_id('DataTables_Table_0_info')