Skip to content
Switch branches/tags

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time


Aquascraper scrapes the results of the closed auctions from It then sends that data to Firebase to be rendered. The visualized data can be viewed at The visualized data repo is nodes777/aquadisplay. I wrote about the process of making this on medium.




To Scrape:

In a terminal run casperjs scrape.js

Data is currently written to Firebase, code to write to disk is commented out.

To Grab the Data from Firebase

By a Single Day's raw data

Make a JSONP request:

  url: ""+monthAndYear+"/"+day+".json?callback=processJson&print=pretty",
  dataType: "jsonp",
  jsonpCallback: "processJson"

monthAndYear must be formatted like "Nov2017" day must be formatted like "01-Wed" The earliest available date is Oct2017, 15-Sun.

Returns all closed auctions (sold and unsold) for that day as a JSON object.

By Stats Only

Follows Firebase API rules for filtering. Make a JSONP request:

  url: '"timestamp"&limitToLast=30',
  dataType: "jsonp",
  jsonpCallback: "processJSON"

Returns a JSON object of all fish type's market stats; average, standard deviation, and sales volume of each day within the url parameters. This example URL grabs the last 30 days.


  • Currently the data being displayed for the closed auctions aquabid, after selecting "view 1 previous day", includes sales that have been sold before their close date. This allows a single sale to remain as a valid data point for multiple days. If serious data analysis is to be done, this must be sorted to only display the data on the day that the sale is made. This was fixed on February 22nd 2018. All previous scrapes have sales from dates around that specific day so they have higher sales volumes. The current set up only scrapes sales if that sale was made on the day of the scrape. This is more accurate to the "sales that were made today".

To Dos

  • Add links to fish closed auction pages


As of January 2nd 2018, aquabid's robots.txt only prevents automated browsing on the "/cgi-bin/auction/vfb.cgi" pages. Presumably, this is to avoid automated bid making.


Scrapes sales data from



No releases published


No packages published