CiviCRM Testapalooza

This project demonstrates of how to write tests for CiviCRM. The techniques here are pretty generic -- they ought to work with different package types (CiviCRM core, CiviCRM extensions, Drupal modules, WordPress plugins) and different test-runners (e.g. phpunit, codeception, protractor, behat).

Testing CiviCRM is trickier than testing a basic library -- tests may involve system services (from Civi or the CMS), and CiviCRM administrators may use different CMS's, file structures, and URLs. One way to mitigate this would be creating configuration files (e.g. each extension could have its own copy of codeception.yml or karma.conf.js which must be fine-tuned with file-paths and URLs for a local CiviCRM). Unfortunately, if you have a dozen extensions with tests, creating those custom configuration files becomes unwieldy.

To resolve this, we use the helper command, cv. This command automatically searches for CiviCRM and bootstraps it. It can be integrated into your extension tests -- enabling them to lookup CiviCRM folders and URLs without custom configuration files.

In the remainder of this document, we'll discuss some general testing rules and end with links to specific examples.

Requirements

• Working understanding of LAMP and CiviCRM installation.
• General familiarity with some test tool (eg phpunit or codeception).
• CiviCRM v4.7.1+ (via git)
  • Note: You may be able to engineer tests to work with other versions, but this is best because:
    • The civicrm.settings.php template was updated in v4.7.1 to facilitate testing.
    • The civicrm-core test classes in v4.7.1 were refactored to be more re-usable.
    • The civicrm-core test classes do not ship with the tarballs.
• Recommended: Vanilla file structure.
  • For example, in Drupal use sites/all/modules; in WP, use wp-content. Avoid symlinks. Use single-site. More sophisticated schemes may work, but they haven't been tested. See also: Stackexchange: "Failed to locate civicrm.settings.php"

Setup: Option A: Buildkit

• Install buildkit.
• Create a test site using civibuild.

Setup: Option B: Manual

• Install Drupal/WordPress. (Backdrop/Joomla may also work - but not tested as much.)
• Install CiviCRM v4.7+
• Install cv. Ensure it is located somewhere in the PATH.
• cd into your Drupal/WordPress site and run cv vars:fill. This will create a file ~/.cv.json.
  • Tip: If you need to share this installation with other local users, you may specify export CV_CONFIG=/path/to/other/file.json
• Edit the file ~/.cv.json. You may need to fill some or all of these details:
  • Credentials for an administrative CMS user (ADMIN_USER, ADMIN_PASS, ADMIN_EMAIL)
  • Credentials for a non-administrative CMS user (DEMO_USER, DEMO_PASS, DEMO_EMAIL)
  • Credentials for an empty test database (TEST_DB_DSN)
• Install your favorite test-runner (e.g. phpunit, codeception, behat).

Mind your data

Civi is DB-oriented application. When writing tests, you'll often do crazy things to the DB. I recommend keeping a recent DB snapshot so that you can quickly restore.

(If you use civibuild, it stores a DB snapshot by default. You can manage it with civibuild snapshot <mybuild> and civibuild restore <mybuild>.)

Techniques

• Bootstrap: Fast, in-process testing ("Headless") - The fastest test-suites will load Civi one time, but they provide a less thorough simulation.
• Bootstrap: End-to-end, multi-process testing ("E2E") - The most realistic test-suites execute tests in a fully configured Civi+CMS installation, but they incur greater performance penalties.
• Data management: Civi\Test and Transactions - Declare the configuration in which tests should run.

Examples

This README focuses on generic guidance, but the git repo also has examples for particular tools in alternative branches:

• phpunit (in-memory or end-to-end tests)
• codeception-2.x (in-memory or end-to-end tests; experimental)
• protractor (end-to-end tests only; TODO)