We've been using it for data vis, BI, and some other things.
Forked from escherize/crajure with no particular intent to contribute back.
- Section keywords are now namespaced, to deal with the fact that Craigslist has several different subcategories such as "housing - rooms to let".
:category
key now missing from responses, since due to UI changes in Craigslist it is no longer trivial to infer this. It should be able to recover this information from the listing URLs in the future.:item-url
has been renamed to:url
.:preview
being the URL for the item's first picture has been added.:address
being the street address for the item if available has been added.
To search for fixie bikes in san francisco, simply supply the space delimited query string, area
code (i.e. :sfbay
), and site section (i.e. :for-sale/all
); like so:
(query-cl "fixie bike" :sfbay :for-sale/all)
;=>
({:price 2500,
:title "2013 Giant Glory 2 Medium *price drop*",
:date "2014-10-22 21:20",
:region "morgan hill",
:item-url "http://sfbay.craigslist.org/sby/bik/4713638395.html"}
{:price 2500,
:title "FS: Cinelli Mash Original Bolt Complete 53cm",
:date "2014-10-20 11:57",
:region "san jose south",
:item-url "http://sfbay.craigslist.org/sby/bik/4681252594.html"}
...)
Also, one may call query-cl with a map like so:
(query-cl {:query "fixie bike"
:area "sfbay"
:section :for-sale/all})
;=> ({:price 2500,
:title "2013 Giant Glory 2 Medium *price drop*",
:date "2014-10-22 21:20",
:region "morgan hill",
:item-url "http://sfbay.craigslist.org/sby/bik/4713638395.html"}
{:price 2500,
:title "FS: Cinelli Mash Original Bolt Complete 53cm",
:date "2014-10-20 11:57",
:region "san jose south",
:item-url "http://sfbay.craigslist.org/sby/bik/4681252594.html"}
...)
One interesting benefit to using this library is we can query craigslist for the entire earth using
:all
as the area-code. We can also look at every section of the site with one query just the same
way, by using :all
in the site-section field. These queries can hit thousands of webpages so of
course it's not exactly reccomended when online speed is nessicary.
(query-cl "bicycle" :all :all)
;=> lots of maps.