Skip to content

Commit

Permalink
Added a Dockerfile to make it easy to run tests. Fixes #125
Browse files Browse the repository at this point in the history
  • Loading branch information
sylvinus committed Dec 3, 2014
1 parent c1909ce commit 00b907e
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 4 deletions.
10 changes: 10 additions & 0 deletions Dockerfile
@@ -0,0 +1,10 @@
# Use the official Docker images
# https://registry.hub.docker.com/_/node/
#
FROM node:0.10-onbuild

RUN apt-get update

RUN apt-get install -y python python-pip

RUN pip install httpbin gunicorn
20 changes: 16 additions & 4 deletions README.md
Expand Up @@ -112,7 +112,7 @@ Retry options:

Server-side DOM options:

* `jQuery`: true, false or ConfObject (Default true)
* `jQuery`: true, false or ConfObject (Default true)
see below [Working with Cheerio or JSDOM](https://github.com/paulvalla/node-crawler/blob/master/README.md#working-with-cheerio-or-jsdom)

Charset encoding:
Expand All @@ -135,7 +135,7 @@ Other:
Working with Cheerio or JSDOM
-----------------------------

Crawler by default use [Cheerio](https://github.com/cheeriojs/cheerio) instead of [Jsdom](https://github.com/tmpvar/jsdom). Jsdom is more robust but can be hard to install (espacially on windows) because of [contextify](https://github.com/tmpvar/jsdom#contextify).
Crawler by default use [Cheerio](https://github.com/cheeriojs/cheerio) instead of [Jsdom](https://github.com/tmpvar/jsdom). Jsdom is more robust but can be hard to install (espacially on windows) because of [contextify](https://github.com/tmpvar/jsdom#contextify).
Which is why, if you want to use jsdom you will have to build it, and `require('jsdom')` in your own script before passing it to crawler. This is to avoid cheerio crawler user to build jsdom when installing crawler.

###Working with Cheerio
Expand All @@ -148,7 +148,7 @@ jQuery: {
name: 'cheerio',
options: {
normalizeWhitespace: true,
xmlMode: true
xmlMode: true
}
}
```
Expand All @@ -163,7 +163,7 @@ These parsing options are taken directly from [htmlparser2](https://github.com/f
```

For a full list of options and their effects, see [this](https://github.com/fb55/DomHandler) and
[htmlparser2's options](https://github.com/fb55/htmlparser2/wiki/Parser-options).
[htmlparser2's options](https://github.com/fb55/htmlparser2/wiki/Parser-options).
[source](https://github.com/cheeriojs/cheerio#loading)

###Working with JSDOM
Expand Down Expand Up @@ -192,6 +192,18 @@ node-crawler use a local httpbin for testing purpose. You can install httpbin as
// Finally
$ npm install && npm test

### Alternative: Docker

After [installing Docker](http://docs.docker.com/), you can run:

// Builds the local test environment
$ docker build -t node-crawler .

// Runs tests
$ docker run node-crawler sh -c "gunicorn httpbin:app -b 127.0.0.1:8000 -w 6 --daemon && npm install && npm test"

// You can also ssh into the container for easier debugging
$ docker run -i -t node-crawler bash

[![build status](https://secure.travis-ci.org/sylvinus/node-crawler.png)](http://travis-ci.org/sylvinus/node-crawler)

Expand Down

0 comments on commit 00b907e

Please sign in to comment.