Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Source for theguardian.com
HTML JavaScript Scala CSS Java Shell
Branch: master
Failed to load latest commit information.
admin Test to see the impact of longer caching with Soft Purging.
applications Give healthchecks a bit of leeway to run in tests.
archive Revert "Merge pull request #9815 from guardian/revert-play-upgrade"
article fix linked data on articles
commercial Merge pull request #10336 from guardian/tm-soulmates-amends
common Merge branch 'remove-cardstyle'
data adding width and height to pinterest button
dev-build Merge branch 'master' into show-deduped
dev Make private to remove npm warnings
diagnostics Give healthchecks a bit of leeway to run in tests.
discussion Give healthchecks a bit of leeway to run in tests.
docs updating inline svg docs
facia-end-to-end rename capi preview endpoint
facia-press Remove jsonModels
facia-tool Merge branch 'master' into remove-cardstyle
facia Give healthchecks a bit of leeway to run in tests.
git-hooks Fix pre commit hook to use bash instead of sh
grunt-configs disable facia tool tests
identity Give healthchecks a bit of leeway to run in tests.
image Revert "Merge pull request #9815 from guardian/revert-play-upgrade"
integrated-tests Improve Ads Integration Tests
nginx Fix broken relative link
onward Merge branch 'master' into remove-cardstyle
preview Remove ParseCollection and friends
project Merge branch 'master' into remove-cardstyle
router Revert "Merge pull request #9815 from guardian/revert-play-upgrade"
rss Merge branch 'master' into kc-empty-mpus
sanity-tests/test Changed trait to sealed trait
sport Show the score events table if one of the teams has points.
standalone Rugby API urls need /sport at the start to work with nginx
static Merge pull request #10360 from guardian/white-video-icon
tools Split dist-root-tc into three audit-friendly scripts
training-preview Ensure super healthcheck is always called
.editorconfig just trying this out
.eslintrc ESLint: use no-unused-vars rule
.gitignore Remove redundant line in gitignore
.jscsrc Use dotfile for JSCS config, as per convention
.scss-lint.yml Sass lint
CONTRIBUTING.md Added email verification to ID API
Gemfile Merge branch 'master' of github.com:guardian/frontend into styleguide
Gemfile.lock First attempt at google webtools.
Gruntfile.js lint
IMAGES.md We do not know if there is a video at the top of the body.
LICENSE L'an de grâce deux mil quinze
README.md updating how to run the app, and changing broken example link
Vagrantfile Sync the .ivy2 directory
browserslist restore US to browserslist
bundle.js Fix a11y preferences: do not duplicate AMD modules
cla-corporate.txt Add Contributor License Agreements to the project
cla-individual.txt Add Contributor License Agreements to the project
dist-assets-tc Tidy up team city messages
dist-npm-tc use grunt-tc during install process
dist-play-tc No Dist for png-resize
grunt-tc Set config into jspm registry
hologram_config.yml adding initial button styles and hologram setup
install-dependencies.sh Add script for installing deps
makefile install dev packages from root
npm-shrinkwrap.json Upgrade jspm to 0.16.2 (out of beta!)
package.json Upgrade jspm to 0.16.2 (out of beta!)
provision.sh Got rid of 3rd party image service for now, tests are over.
sbt Remove ivy-sbt folder
sbt-tc Bump the memory pool size for sbt on team city
setup.sh make sure all packages are installed before running compile in setup.sh
test-root-tc Make sure npm is installed for pull request builds

README.md

We're hiring!

Ever thought about joining us?
http://developers.theguardian.com/join-the-team.html

Frontend

The Guardian website frontend.

Frontend is a set of Play Framework 2 Scala applications.

Frontend is built in two parts, using Grunt for the client side asset build and SBT for the Play Framework backend.

Core Development Principles (lines in the sand)

These principles apply to all requests on www.theguardian.com and api.nextgen.guardianapps.co.uk (our Ajax URL)

On the server

  • Every request can be cached and has an appropriate Cache-Control header set.
  • Each request may only perform one I/O operation on the backend. (you cannot make two calls to the content API or any other 3rd party)
  • The average response time of any endpoint is less than 500ms.
  • Requests that take longer than two seconds will be terminated.

New developers quick-start

Contents:

  1. Local Test Server setup
  2. IDE Setup
  3. Troubleshooting
  4. Optional steps
  5. Useful information and hints
  6. Additional Documentation

Local Test Server setup

You need A Mac or Linux PC (ubuntu).

Automatic

  1. Check out the repository:

    git clone git@github.com:guardian/frontend.git
    cd frontend
    
  2. Run ./setup.sh to install dependencies and compile assets

  3. All being well, you should be able to run the app

Before checking out the repository you may need to add an SSH key to your GitHub account, information on how to do so is here - https://help.github.com/articles/generating-ssh-keys/

 Manual

Install each of the things listed:

Configuration files

You need 3 files on your machine.

  1. /etc/gu/install_vars
STAGE=DEV
  • ~/.gu/frontend.properties

    frontend.properties contains the content.

    Ask your team mates to share it with you if you don't get any results.

  • ~/.aws/credentials

    Ask your team mate to create an account for you and securely send you the access key. For security, you must enable MFA - ask if you're not sure what this means.

[nextgen]
aws_access_key_id=[YOUR_AWS_ACCESS_KEY]
aws_secret_access_key=[YOUR_AWS_SECRET_ACCESS_KEY]
region=eu-west-1

Homebrew

This is needed on Mac only:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

A JDK

Ubuntu:

sudo apt-get install openjdk-7-jdk

Mac: Install from Oracle web site

Node.js

Ubuntu:

curl -sL https://deb.nodesource.com/setup | sudo bash -
sudo apt-get install -y nodejs

Mac:

brew install node

Grunt (build tool)

Ubuntu/Mac:

sudo npm -g install grunt-cli

JSPM (package management)

Ubuntu/Mac:

sudo npm -g install jspm
jspm registry config github

It'll ask for a GitHub access token. Go to GitHub Settings -> Applications and generate new token. Ensure only the public_repo scope is checked. Now create a registry instance.

npm -g install jspm-bower-endpoint # jspm >= 0.15.0
jspm registry create bower jspm-bower-endpoint

Ruby >= v1.9.x (use ruby -v to check if you have it installed)

Ubuntu:

sudo apt-get install ruby ruby-dev

bundler

Ubuntu/Mac:

sudo gem install bundler

Xcode (if on a Mac, one of the Node modules requires it)

This is needed on Mac only: https://itunes.apple.com/gb/app/xcode/id497799835

Xcode installs an old version of git 1.9.3. If you need a newer version, you can run

brew install git
echo 'export PATH="/usr/local/bin:$PATH"' >> ~/.bash_profile

Quit Terminal, relaunch it and check that git --version outputs 2.1.3 or newer.

libpng

Ubuntu:

sudo apt-get install libpng-dev

Mac:

brew install libpng

The frontend code

Note: Remember to see Troubleshooting below if you have any issues.

git clone git@github.com:guardian/frontend.git

cd frontend

Install node dependencies:

npm install

Install additional dependencies:

bundle
grunt install

npm, bundle, and jspm are also run by install-dependencies.sh.

After this, you can compile the assets:

grunt compile

 Run the app

In another console, run the supplied bash script [sbt]. The dot and slash are important in this command.

./sbt

Wait for SBT to be up and running. This may take 15 mins or so to start the first time - you'll know it's done when you get a prompt. If it is your first time, compile the project.

compile

Switch project by typing

project dev-build

Then run the project locally by typing

run

This also can take a while the first time.

Now check that you are up and running by hitting the following URLs:

Congratulations, you have a local instance running! Now continue on to set up your IDE.

IDE setup

You need a copy of the source code from above. If not, run this command:

git clone git@github.com:guardian/frontend.git

EditorConfig plugin

Install to your IDE from http://editorconfig.org/#download

intelliJ metadata

To create project files for use in IntelliJ, you need to make sure you install the Scala plugin from Preferences->Plugins. It supports SBT and Play. Then load IntelliJ, then click Import project and import the directory as an SBT project. Default settings are fine, except you need to make sure you choose JDK 1.8 otherwise it won't import correctly.

Congratulations, you are now set up to edit frontend code! See the Optional steps below for other things to do.

Troubleshooting

NPM "EACCES"

If you get errors like this on npm install

npm WARN locking Error: EACCES, open '/Users/jduffell/.npm/_locks/karma-requirejs-4becac899d6c8f35.lock'

Sometimes when you install npm, it ends up owned by root (but in your home directory).

Check that you own your own .npm directory ls -ld ~/.npm

If it is owned by root, then take ownership of it sudo chown -R username:username ~/.npm

Global install permissions errors

The script installs global npm packages without sudo. If you get npm permission errors, follow the guide to using npm without sudo here.

phantomjs permissions errors (OSX)

If you get an error about not having permissions to execute phantomjs during grunt compile, your machine is probably set up as managed and you'll need to ask IT to make it unmanaged.

File handles - "Too many files open"

You may run into a "too many files open" error during compilation or reloading. You can find out how many file handles you are allowed per process by running ulimit -n. This can be quite low, e.g. 1024 on linux or 256 on Mac

Linux

To increase the limit do the following (instructions from Ubuntu 12.10)...

In the file /etc/security/limits.conf add the following two lines

*  soft  nofile 20000
*  hard  nofile 65000

And in the file /etc/pam.d/common-session add the following line.

session required pam_limits.so

Restart the machine.

For more info see http://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/

Mac

  • Edit your ~/.bash-profile file
  • add the following line: ulimit -n 1024
  • save and close the file
  • back at the command prompt enter: source .bash_profile and hit return.

Now you should be able to compile and run. Yay.

"No route to host"

If you get no route to host, it means you are not using the 1.8 jre. Type java -version to check. You may need to close and reopen your terminal if you installed 1.8 recently.

Optional steps

NVM

Some packages (imagemin) are not working with newest Node.js. So if you want to run multiple Node.js versions on your system you may want to use nvm

Memcached

Memcached sudo apt-get install memcached - (most of the time you do not want to use it as caching makes local development harder)

Nginx

If you are working on Identity or Discussion, Nginx must be installed and configured to correctly serve the application, please refer to /nginx/README.md in this project.

Vagrant

You can run the project with the supplied Vagrantfile - make sure you understand what vagrant is http://www.vagrantup.com/

  • Make sure you have Virtualbox and Vagrant installed (on Ubuntu sudo apt-get install virtualbox vagrant)
  • change directory into the folder where this README is located
  • vagrant up - this will take a while, make some coffee
  • You can now get onto the box by vagrant ssh
  • the project is located in /vagrant so cd /vagrant
  • ./sbt

Client-side development mode

There is a grunt watch task available to build and watch for development changes, but grunt-watch is pretty inefficient to compile our Sass into CSS so @mattosborn created a script called grunt-csdevmode.

grunt csdevmode also pushes stylesheets to all connected browsers: no need to reload a page to preview your changes, just like with Livereload.

grunt compile --dev
grunt csdevmode

Useful information and hints

Play console

Play Framework will recompile code changes on refresh.

Further information on using the Play console is available here.

Endpoints

The available endpoints are listed in conf/routes of each application and typically include:

  • /management: Operations support as per standard webapp guidelines. See guardian-management.
  • /<path>: Serve the Guardian URL at <path> if supported by this application.
  • /assets/<file>: A convenience for DEV machines. Assets are CDNed in PROD and would not be available on DEV.

Deploying

Deployment uses the Magenta library.

Debugging

You can debug your local Frontend application, by attaching a debugger.

  • Start Simple Build Tool in debug mode by typing ./sbt --debug
  • Build and run your application. See "Running" for steps.
  • Use a debugger to attach to the remote Java process, on localhost:1044.

Any IDE debugger should be compatible. In IntelliJ, add a new Debug Configuration, based on the Remote default. Ensure the Transport is Socket, the Debugger mode is Attach, and the port is set to 1044. Start a new Debug session, and your breakpoints should be active.

Additional Documentation

If you're new, you'll want to see what libraries we use in frontend.

Further documentation notes and useful items can be found in docs.

Something went wrong with that request. Please try again.