Skip to content
Implementation of NeSI.org.nz in Drupal - NB this is deprecated, we now use the Pantheon Git repository
PHP JavaScript CSS Shell ASP Perl
Find file
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
includes
misc
modules
private/tests
profiles
scripts
sites
themes
.gitignore
.htaccess
CHANGELOG.txt
COPYRIGHT.txt
INSTALL.mysql.txt
INSTALL.pgsql.txt
INSTALL.sqlite.txt
INSTALL.txt
LICENSE.txt
MAINTAINERS.txt
README.md
README.txt
UPGRADE.txt
authorize.php
cron.php
favicon.ico
index.php
install.php
robots.txt
update.php
web.config
xmlrpc.php

README.md

NeSI Website Code Repository

This repository version-controls custom code for the website of NeSI, New Zealand eScience Infrastructure: www.nesi.org.nz

This article documents how use this repository, test, deploy, maintain and manage a website hosting environment—an instance of the website used for production, staging, development or sandbox.

Third Party Code

Third party code is excluded from this repository:

  • Drupal core
  • Anything from Drupal.org such as modules or themes
  • Javascript and jQuery libraries, unless they are custom libraries for NeSI
  • Patches for any of the above

Third party code (along with version numbers and patches) is referenced in nesi.drush.make instead of being added to the repository. Drush make can then include Drupal core, contrib and any other dependent third party code as well as apply patches to it.

Drush make adds contrib modules to sites/all/modules. Custom modules go in sites/default/modules.

Requirements

Installing and Managing Environments

Setup New Environment

Use this process to create a new environment of the website.

This process is automated in a puppet script. The puppet script is used for staging and production environments: https://github.com/nesi/puppet-drupal

  1. git clone git@github.com:nesi/nesi-webportal.git nesi-website
  2. Configure nesi-website/drupal/ as the apache web document root.
  3. cd nesi-website/drupal/
  4. drush make nesi.drush.make
  5. cd sites/default/
  6. cp default.settings.php settings.php
  7. Configure $databases in settings.php with mysql database and credentials
  8. Run echo "require 'settings.inc';" | cat >> settings.php to add additional system settings
  9. Import database and files. See Import Content.

Update Existing Environment

Use this process to update an existing environment with latest code.

  1. cd to the root of the git repository.
  2. git pull or if switching git branches; git fetch and git checkout BRANCH
  3. cd drupal
  4. rm -rf sites/all
  5. drush make nesi.drush.make
  6. Update the database. See Update Database.

Additional Assumptions

These additional requirements and assumptions apply to the Import Content, Update Database and Import & Update Content processes below.

  • Linux user account on web.dev.nesi.org.nz with SSH key-pair access configured. (Required)
  • The only directories in sites/ are default and all.
  • This patch for Drush sql-sync so that it uses $options['dump-dir'] as documented. How to apply Drupal.org patches.
  • The following changes to ~/.drush/drushrc.php;
    • Add or uncomment $options['structure-tables']['common'] = array('cache', 'cache_filter', 'cache_menu', 'cache_page', 'cache_views', 'history', 'sessions', 'watchdog');
    • Configure $options['result-file'] so that drush sql-dump dumps to file instead of STDOUT. E.g.
      Linux: $options['result-file'] = '/home/brud046/sql-dumps/@DATABASE_@DATE.sql';
      Mac OS X: $options['result-file'] = '/Users/brud046/sql-dumps/@DATABASE_@DATE.sql';
    • Configure $options['dump-dir'] to optimise drush sql-sync through it's use of rsync. E.g.
      Linux: $options['dump-dir'] = '/home/brud046/sql-dumps';
      Mac OS X: $options['dump-dir'] = '/Users/brud046/sql-dumps';
    • The directories must exist. Drush will not create them.

Import Content

Use this process to pull the website's content (non version-controlled dynamic data: database & files directory) into a staging or development environment. Do not run it in the production environment.

@prod or @test can be used instead of @dev to pull content from the corresponding environments.

These aliases are version-controlled in aliases.drushrc.php. They are available when Drush executes in the Drupal root directory or any sub-directory. To use these aliases from any directory on the filesystem in the form @nesi.dev instead of shorthand @dev, create a symlink to the file from ~/.drush/nesi.aliases.drushrc.php. Note however that this breaks sql-sync from inside the doc root. See this issue in the drush issue queue.

  1. cd to the web document root.
  2. drush sql-dump
  3. drush sql-drop
  4. drush sql-sync @dev
  5. drush rsync @dev:%files %files
  6. Update the database if it was running on an older code base. See Update Database.

Update Database

This process is necessary when:

  • An environment's code base is updated. E.g. A newer code base is deployed to production.
  • A database is copied from an environment with an older code base. E.g. Content is pulled from production to a staging or sandbox environment.

This process updates the database schema, rebuilds registries, clears caches and makes any other changes that are required for the database to be consistent with the code base. It is can not be undone.

  1. If the database was not recently backed up as part of a preceding process: drush sql-dump
  2. drush registry-rebuild
  3. drush cache-clear all
  4. drush updb
  5. drush features-revert-all
  6. Using a web browser, login to the website at user/login
  7. Check admin/reports/status and admin/reports/dblog
  8. If not production: drush sql-query 'UPDATE users SET mail = "user@example.com" WHERE uid > 1'
    This prevents email notifications being sent from staging and sandbox environments. It may be useful to use a real email address instead of user@example.com.

Some email providers support address tags. E.g. bevan.rudge+foobar@nesi.org.nz. Gmail, Google Apps and @nesi.org.nz support this feature. @auckland.ac.nz does not. To take advantage of this feature, consider SET mail = CONCAT("bevan.rudge+", name, "@nesi.org.nz").

Import & Update Content

The Drush shell alias drush pull-data combines the Import Content process and most of the Update Database process into one simple step.

Add the following to ~/.drush/drushrc.php:

$options['shell-aliases']['pull-data'] = '!drush sql-dump && drush sql-drop && drush sql-sync @dev && drush registry-rebuild && drush cache-clear all && drush updb && drush features-revert-all && drush rsync @dev:%files %files';

This alias is especially useful when testing database update code (implementations of hook_update_N() in a development sandbox, but can also be used on staging environments with some tweaks; at least @dev will need to be changed.

This alias excludes the drush sql-query command because of a bug in drush.

Do not use this alias on production.

Automated Testing

Automated tests are implemented using the Behat framework.

Dependencies

  • php 5.3.2+
  • php5-curl
  • Composer
  • cd tests/
  • Composer manages further dependencies: composer install (This one takes a while)
  • Java runtime
  • Selenium standalone server into tests/bin/

Configure

From the repository root;

  • cd tests/
  • cp behat.local.example.yml behat.local.yml
  • Set base URL and user credentials in behat.local.yml.

Run Tests

From the repository root;

  • cd tests/
  • java -jar bin/selenium-server-standalone-2.33.0.jar
  • And in a new bash session/tab:
  • cd tests/
  • bin/behat

Write Tests

The quickest way to get familiar with BDD and Behat is to purchase the knpuniversity.com behat screencast an excellent and informative tutorial.

Useful links

Theme Information

The NeSI website theme is a subtheme of the Bootstrap theme for Drupal which leverages the Bootstrap framework. Use drush make (see above) to include both dependencies.

Theme files are in drupal/sites/default/themes/nesi_bootstrap/.

Modifying CSS

It is recommended to update nesi_bootstrap/css/style.css file with any style overrides. This process will change when all CSS is managed with LESS and the lessc compiler.

Modifying LESS

The nesi_bootstrap theme also has .less files in nesi_bootstrap/assets/css/less/. bootstrap.less can be compiled to recreate the css files that Drupal includes via nesi_bootstrap/nesi_bootstrap.info. You will need a LESS compiler. See the Bootstrap framework's guide.

@todo Include the command to compile and watch for changes.

Something went wrong with that request. Please try again.