express-experiment

Learning express.js for people coming from Django or Rails background.

See also express-boilerplate

Summary

I am a Django dev. I highly value Django for many of its features. But there are some things that really feel old. One of my recent despairs was with wsgi. So I thought to take a look once more on node.js fresh. Aside from the fact that another tech which I really love, angular, must really benefit from node. Some early conclusions at the end of the document.

Desired technologies

I have decided to use the following technologies in order to cover existing Django functionality. The remaining document is structured on this, where it is covered why the particular technology has been selected.

• Nodejs
• Express
• Bookshelf (ORM)
• Jade (templates)
• Bower (asset management)
• Angular
• Less (css preprocessor)
• i18n (internationalization)
• Passport (user auth)

Other topics:

• Promises
• Reusable apps
• Deployment
• IDE

Node

Install on Ubuntu:

sudo apt-get install nodejs npm

Or use nvm as described in deployment section.

Useful links

• Node.js
• Loading modules
• Style guide

Express

Nodejs favorite unopinionated framework

Useful links

• Express.js
• Tutorial
• Promises
• What does ./bin/www do

Bookshelf

Bookshelf is the preferred ORM with many features that resemble the functionality of Django or Rails. Migrations is a very important feature for well-maintained apps and Bookshelf is thriving in that.

Other ORM possibilities include Sequelize, ORM2 and Waterline. Sequelize is the most common. Unfortunately, there are major shortages in documentation and many features including migrations are just not natural enough, at least for someone coming from Django or Rails. ORM2 does not offer migrations at all. Waterline offers some migration functionality but it seems like it is trying to do too much magic with many different db possibilities and it ends up using the lowest common denominator. Some critique here.

Guide

npm install -S bookshelf knex sqlite3 checkit moment
node_modules/.bin/knex init

The last one creates a knexfile.js, which needs to be updated with connection settings. Then add a directory services\ and files index.js and bookshelf.js. Initialisation happens in bookshelf.

Then create a subdirectory migrations and run

node_modules/.bin/knex migrate:make user

Edit the newly created file in migrations and then:

node_modules/.bin/knex migrate:latest

Then create the routes and views in routes/index.js and views/.

More on bookshelf

Useful links

• Bookshelf
• Tutorial
• CRUD examples

Jade

Express uses Jade. Other possibilities:

• Nunjucks (Jinja2 in node)
• Handlebars
• Underscore
• More...

Bower

Great utility that manages assets.

npm install bower
node_modules/.bin/bower init
node_modules/.bin/bower install

Scripts can be used as:

<script src="bower_components/jquery/dist/jquery.min.js"></script>

The first main advantage of bower is asset version management. This is worth against possible cdn speed gains (see this thread, this post by Steve Souders and this post).

Besides, grunt-cdnify could be used to change all bower references to cdn, but it has got issues as this one.

Furthermore, utilities such as grunt-bower-requirejs can be used to combine bower with RequireJS, which is great for async loading of scripts. Additionally, RequireJS has an optimisation feature available.

Useful links

• http://bower.io/
• http://stackoverflow.com/questions/21821773/configure-node-express-to-serve-static-bower-components

Angular

Replaced views/index.jade records and form with angular directives and added javascripts/controller.js. Modified routes/user.js to return json.

Useful links

• Angular tutorial
• Tutorial repo
• Angular in express

Less

Using the Less css preprocessor is very easy with node, and helps deal with css maintenance a lot.

npm install -S less-middleware

Then in app.js add:

app.use(require('less-middleware')(path.join(__dirname, 'public')));

i18n

Url-based i18n routing can easily be handled by express itself. Use i18n-node for l10n support:

npm install -S i18n

Add a small middleware in services/i18n_urls.js and use in app.js with app.use. The middleware assigns the language from url to the request object. Then configure module with i18n.configure(). The i18n routes can be handled with a helper function. The assigned language can then be accessed from eg controllers as req.getLocale().

Useful links

• https://github.com/mashpie/i18n-node
• http://stackoverflow.com/questions/12186644/multi-language-routes-in-express-js
• http://stackoverflow.com/questions/24446819/express-js-multilanguage-with-i18n-node
• http://rbeere.tumblr.com/post/41212250036/internationalization-with-express-jade-and
• https://www.npmjs.com/package/loc
• http://stackoverflow.com/questions/14125997/difference-between-app-all-and-app-use

Other important modules (not yet implemented)

Csurf

⚠️ CSRF protection module.

Useful links

• https://www.npmjs.com/package/csurf
• http://expressjs.com/en/advanced/best-practice-security.html

Passport

Authentication middleware. Express does not have anything like Django or Rails have, but Passport adds great capabilities such as easy OpenID and OAuth (FB, Twitter, Google etc).

Passport can be used along with any ORM or db. It provides callbacks to obtain the user object. It requires express-session for session management.

npm install -S passport passport-local

Useful links

• Documentation

• Tutorial

• [Using sequelize with passport] (http://www.hamiltonchapman.com/blog/2014/3/25/user-accounts-using-sequelize-and-passport-in-nodejs)

• [Other tutorial] (https://orchestrate.io/blog/2014/06/26/build-user-authentication-with-node-js-express-passport-and-orchestrate/)

• Permissions middleware

Express-session

This module is the standard for express session management.

By default it uses a memory store for cookies, but this is supposed to work only for dev environments (gives mem leaks too). Therefore a different store is required. Django uses [db storage by default] (https://docs.djangoproject.com/en/1.9/topics/http/sessions/#configuring-the-session-engine), but since a choice is necessary, the best option seems to be redis (no memcached option).

Other configuration

• name: cookie name, if on shared host

• secret: a secret string to sign the cookie, use something like this, required

Caching

Combining with connect-redis for express-session, redis can be used for route caching too with express-redis-cache.

Useful links

• Configuring redis
• Redis INFO command
• Sending commands to redis
• Using memcached

Other interesting modules

• multer
• express-debug
• image resize

Promises

Promises is a central point in node. Sooner or later a pyramid situation will come at play, where nesting callbacks make a horrible code. The use of promises is inevitable. Bluebird is an excellent library for helping with that. There is an excellent tutorial on the subject. Nevertheless, the following important points must be acknowledged.

Always return a promise from a chained then(). Otherwise the next then will be executed but will be missing some variables.

A promisified function can return anything. The following example is with using Jade:

jade.renderFileAsync('views/index.jade', {title: 'yoo'}).then(function (html) {
  // do sth
  return html;
}));

Last, Promise.all() can be used straightforward. Can supply any number of functions that return promises, even with any number of chains:

for (var i = 0; i < 10; i++) {
  renders.push(jade.renderFileAsync('views/index.jade', {title: 'yoo'}).then(function (html) {
    // do sth
    return html;
  }));
}
return Promise.all(renders).then(function (renders) {
  page.content = renders;
  return page;
});

Useful links

• http://bluebirdjs.com/docs/getting-started.html
• http://alexperry.io/node/2015/03/25/promises-in-node.html
• https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all
• http://stackoverflow.com/questions/21298190/bluebird-promises-and-then

Reusable apps

Useful links

• Singletons
• NPM tags
• Publishing
• How to write
• Copyfiles

Deployment

Create user

As a security consideration, it is better to run node as a user with limited permissions. Assign the user a home directory (for nvm below) and shell access. This can happen one-time for all node apps, or for more tightened security create a user for each app.

Where to deploy

Obviously create a location on the server on which to deploy the project. The location can be anywhere, as long as the above user has access to it.

In case of shared hosting, optionally create an appropriate domain or subdomain for the app.

Upload/rsync files.

Install node

Do not rely on OS repo node, chances are that is massively outdated. Express requires node >v0.8 and eg Ubuntu 12.04 has got node v0.6 (see also [this] (http://stackoverflow.com/questions/25874666/express-app-throws-500-typeerror-object-eventemitter-has-no-method-hrtime)). One-time installation of node is easy, but to allow multiple versions (as python virtualenv), nvm can be used easily. Also, v4.0 and above (after the merge with io.js) has compilation issues with older LTS releases such as Ubuntu 12.04 Therefore v0.12.x is recommended for general availability, which is maintained, unless particular features are required.

Prerequisites:

sudo apt-get update
sudo apt-get install build-essential libssl-dev

Install nvm:

curl https://raw.githubusercontent.com/creationix/nvm/v0.16.1/install.sh | sh
source ~/.profile

This creates an ~/.nvm directory and also executes the appended lines in bash profile.

Install a node version:

nvm ls-remote
nvm install 0.12.9

And use it with:

nvm ls
nvm use 0.12.9

Alias:

nvm alias default 0.12.9
nvm use default

Npm starts from the particular node version, eg in ~/.nvm/v0.12.0/lib/node_modules/npm.

Further reading

To [auto start nvm upon login] (http://stackoverflow.com/questions/14948179/how-to-make-nvm-automatically-sourced-upon-login), create a default alias and then append on ~/.bash_profile:

[[ -s $HOME/.nvm/nvm.sh ]] && . $HOME/.nvm/nvm.sh

Check with which node.

Serving the application

This covers Apache.

Make sure that mod_proxy is enabled or enable (Ubuntu):

sudo apache2ctl -M
sudo a2enmod proxy_http

Then add in site configuration file deploy/vhost.conf.

If using Plesk 11, add the above in /var/www/vhosts/<domain or subdomain>/conf/vhost.conf (please note that is the path, even if the subdomain is under eg /var/www/vhosts/<domain>/<subdomain>/) and then execute sudo /usr/local/psa/admin/sbin/httpdmng --reconfigure-domain <subdomain>.

If using Plesk 12, there is a link 'Web Server Settings' under 'Websites and Domains' that allows to place additional directives in there.

If socket.io is required, then probably Apache v2.4 is required. It might be possible to proxy sockets as recommended here. Then configure mod_proxy_wstunnel as recommended here. To upgrade Apache to 2.4 see here. This should be ok with Plesk as well.

For load balancing, consider PM2. Also excellent article here.

Running as a service

To make certain that node is spawned if for some reason crashes, add deploy/service.conf as /etc/init/yourapp.conf for each app. Make sure to edit the user and paths in the script.

Then start with (Ubuntu):

sudo service yourapp start

Further reading

Regarding the respawn limits, in the above script the script will be respawned every 5sec for 99 retries. Notice that, if the user hits Apache when node is down, a 503 error will be issueed and then Apache will retry after 60 seconds by default or whatever specified in [retry setting] (http://serverfault.com/questions/58707/how-to-avoid-restarting-apache-proxy-when-you-restart-couchdb) (reference).

IDE

I chose to use Jetbrains WebStorm. It offers several neat features that help development.

• Specify a .editorconfig file for the code styling.
• Specify the appropriate nvm node version in Settings: Language and frameworks: Node and NPM.
• Add express code completion in Settings: Language and...: Javascript: Libraries: Download: express.

See also this post.

Conclusions

Node.js has always been fascinating. There are many posts regarding the benefits of it, but I am going to describe what I found to be important from the aspect of development time cost. So in essence I am comparing my express-based experiment to Django (as much as this is possible).

• Deployment: Node thrives. It is a modern solution that allows far easier scaling and management.

• Async: Node's async nature is incredible. Nothing compared to other similar technologies in Python. Maybe it is not the "silver bullet" as many claim, but it definitely brings new possibilities design-wise. The use of bluebird promises greatly reduces the complication of code maintenance. Disclaimer: haven't debugged a large app yet in node.

• Modules: many available modules but with less quality in documentation.

• ORM: many mediocre solutions (see above about modules). I liked Bookshelf, but it reminds what Django has been 3 or 4 years earlier: basic functionality, even more basic migrations (south).

• Forms, formsets, serializers: important stuff in Django, which are almost totally irrelevant to node.

• Views, routes: pretty much the same functionality. I really do not understand why all router urls are hard-coded in express.

• Templates: In first glance Jade looks alien. But after spending some time with it, I found it to be the "Python" of templates. Lean small code, simply fantastic. Here I will also add the easiness to include angular in it.

• Contrib: auth requires some model work in node. Apart from that, passport takes auth possibilities to another level. Regarding admin, this is something totally missing. But it shouldn't be that hard to implement CRUD in node.