Aquascraper

Aquascraper scrapes the results of the closed auctions from aquabid.com. It then sends that data to Firebase to be rendered. I wrote about the process of making this on medium

Installation

Requires:

• Phantom.js
• Casperjs
• Include Phantom and Casper in your PATH
• To send the data to Firebase, you'll need to replace the deets in deets.js with your own secret code and place that in your environmental variables. Otherwise you can write the data to disk, with the commented out code in scrape.js. If you're on Heroku, you'll also need to set up an environmental variable

Instructions

To Scrape:

In a terminal run casperjs scrape.js

Data is currently written to Firebase, code to write to disk is commented out.

To Grab the Data from Firebase

Make a JSONP request:

$.ajax({
  url: "https://aquascraper-data.firebaseio.com/"+monthAndYear+"/"+day+".json?callback=processJson&print=pretty",
  dataType: "jsonp",
  jsonpCallback: "processJson"
});

monthAndYear must be formatted like "Nov2017"
day must be formatted like "01-Wed"
The earliest available date is Oct2017, 15-Sun.