fuck joe ricketts. pulls down gothamist articles by author and stores as markdown files
this runs in docker, so you shouldn't need anything besides that. make sure that the
AUTHORS array is up to date (it is in
/src/constants.js) and then just run
docker-compose up --build.
you can see the queue activity at http://127.0.0.1:3000.
make sure you have a
data directory in this project folder (a sibling to
src). this is where the markdown files will be written.
you can use the
--scale flag with
docker-compose to restart the app without re-loading the initial data and setting the number of
worker containers with
docker-compose up --scale load=0 --scale worker=2.
markdown files should be written into the
data directory you created. i've been pushing updates occasionally from this app to jeremiak/gothamist-scrape-data.