Switch branches/tags
Nothing to show
Commits on Sep 6, 2014
  1. README update

    committed Sep 6, 2014
Commits on Jul 25, 2014
Commits on Jul 19, 2014
  1. Removing step dependency

    committed Jul 19, 2014
  2. Fix for Travis tests

    committed Jul 19, 2014
Commits on Jul 12, 2014
  1. Updated package.json formatting

    committed Jul 12, 2014
  2. Moved to Cheerio

    committed Jul 12, 2014
  3. Updated dependencies

    committed Jul 12, 2014
  4. Don’t specify npm version

    committed Jul 12, 2014
  5. Revert "Added Nodetime"

    This reverts commit bba741e.
    committed Jul 12, 2014
  6. Fix for webhooks

    committed Jul 12, 2014
  7. Compat updates

    To work with Neo4j 2.1 and latest neo4j lib
    committed Jul 12, 2014
Commits on Jul 11, 2014
  1. Adding Graphene URL

    committed Jul 11, 2014
Commits on Mar 25, 2013
  1. Wrapping request() in a try-catch

    Otherwise got potentially caught in auth errors being thrown when finding bad URL:s out in the wild
    committed Mar 25, 2013
Commits on Mar 24, 2013
  1. Updated packages

    committed Mar 17, 2013
  2. Removed Statsmix

    committed Mar 24, 2013
Commits on Mar 20, 2013
  1. Checking if body is empty to avoid JSDom error

    Repeatedly got error "jsdom.env requires a 'html' argument" - probably caused by this
    committed Mar 20, 2013
Commits on Mar 17, 2013
  1. Added Nodetime

    committed Mar 17, 2013
  2. Revert "Compatibility fix with Neo4j 1.7.2"

    This reverts commit 0c81927.
    committed Mar 17, 2013
  3. Revert "Some load average, CPU utilization and mem checks"

    This reverts commit ee610cd.
    committed Mar 17, 2013
Commits on Dec 1, 2012
  1. Linux top != OS X top

    committed Dec 1, 2012
Commits on Nov 1, 2012
Commits on Oct 24, 2012
  1. Cancel delayed fetches as well on shutdown

    Otherwise the script will wait for it to execute and only then realize that nothing should be fetched and since the delay can be rather long that will mean that we will go beyond the 10 seconds that Heroku gives us to complete a graceful shutdown.
    committed Oct 24, 2012
  2. Now shutdown gracefully on eg. Heroku

    Now initiates a shutdown when noticing a SIGTERM - it shuts down the HTTP server of the web workers and stops any new URL:s from being crawled and once the last URL has been crawled connections to database and cache systems are closed. In most cases this will hopefully happen within in 10 seconds and thus result in a graceful shutdown on Heroku servers so that restarts of scripts there works as smoothly as possible.
    committed Oct 24, 2012
  3. Defined module versions more narrowly

    Previously only the tested version was defined - basically to just keep track of what works and what doesn't - but since eg. Heroku installs packages itself on eg. every single little deploy I found it necessary to define the versions a bit more narrowly so that only updates resulting in non-breaking changes can be expected. This makes it a bit harder for me to discover new versions of modules(?) but should ensure better stability.
    committed Oct 15, 2012
  4. More logging to debug a Neo4j/Heroku error

    The traversing of the graph in Neo4j seems to currently be failing at my RelSpider Heroku instance - same code as before, but for some reason the Neo4j database is negliecting it. Added these additional logging to better understand what the issue is all about and I have contacted the Neo4j team with this information and hopefully a solution will be found.
    committed Oct 15, 2012