Permalink
Browse files

Update README to reflect current status. Extract GitGred into its own…

… repo.
  • Loading branch information...
1 parent 1ddda6f commit 23873fd0ab70f845a89a60f839ff4c30fc299efd @mmcgrana committed Dec 12, 2009
View
@@ -2,16 +2,16 @@ h2. clj-garden
Various Clojure libraries and apps I'm working on:
- * <code>clj-bin</code>: Helper files that I use when working with Clojure at the terminal.
+ * <code>cling:</code>: A Clojure wiki, built with Weld
+ * <code>clj-bin</code>: Helper files that I use when working with Clojure at the terminal
+ * <code>clj-cache</code>: A caching abstraction for clojure
* <code>clj-file-utils</code>: Unix-like wrapper for Commons FileUtils
* <code>clj-html-helpers-ext</code>: Proving ground for <code>clj-html-helpers</code>
- * <code>clj-http-client</code>: HTTP client wrapping Commons HttpClient
- * <code>clj-jdbc</code>: High-level interface to JDBC databases
+ * <code>clj-jdbc</code>: High-level interface to JDBC databases'
+ * <code>clj-log</code>: Logging helpers for Clojure
* <code>clj-rome</code>: Universal RSS/Atom Feed parser wrapping Rome
* <code>clj-scrape</code>: Bundling of various clojure libraries to facilitate HTML scraping
* <code>clj-time</code>: Wrapper around Joda Time library
- * <code>cljurl</code>: Url shortening website, built with Weld and Ring.
- * <code>gitcred</code>: PageRanking GitHub users
* <code>stash</code>: Functional ORM
* <code>stash-migrations</code>: Migration for Stash models
* <code>stash-pagination</code>: Paginate Stash find results
@@ -21,12 +21,12 @@ Various Clojure libraries and apps I'm working on:
I've "harvested" several projects from @clj-garden@, moving them to their own top-level projects:
- * "<code>clj-backtrace</code>":http://github.com/mmcgrana/clj-backtrace: Tools for readable backtraces in Clojure programs.
+ * "<code>clj-stacktrace</code>":http://github.com/mmcgrana/clj-stacktrace: Tools for readable backtraces in Clojure programs
* "<code>clj-routing</code>":http://github.com/mmcgrana/clj-routing: Core of URL router for use in web frameworks
- * "<code>clj-unit</code>":http://github.com/mmcgrana/clj-unit: Unit testing library: non-magical, developer-friendly.
- * "<code>ring</code>":http://github.com/mmcgrana/ring: Web application library: abstracts HTTP to allow modular and concise webapps.
- * "<code>weld</code>":http://github.com/mmcgrana/weld: Web framework built on Ring: expressive and fast in a functional style.
- * "<code>cljre.com</code>":http://github.com/mmcgrana/cljre.com: Clojure online regular expression editor, built with Ring and Weld.*
+ * "<code>clj-unit</code>":http://github.com/mmcgrana/clj-unit: Unit testing library: non-magical, developer-friendly
+ * "<code>gitcred</code>":http://github.com/mmcgrana/gitcred: PageRanking GitHub users
+ * "<code>ring</code>":http://github.com/mmcgrana/ring: Web application library: abstracts HTTP to allow modular and concise webapps
+ * "<code>weld</code>":http://github.com/mmcgrana/weld: Web framework built on Ring: expressive and fast in a functional style
---
View
@@ -35,6 +35,10 @@ diffs
could show diff only and link to show-versions
not that we can use the << foo >> bar form for e.g. diff feeds.
+http://www.xml.com/lpt/a/1633
+http://www.atomenabled.org/developers/syndication/#sampleFeed
+http://validator.w3.org/feed/check.cgi
+
search interface
lucene search
@@ -1,54 +0,0 @@
-h1. clj-http-client
-
-@clj-http-client@ is a Clojure library providing basic HTTP client functionality. The library can execute @GET@, @HEAD@, @PUT@, @POST@, and @DELETE@ methods and send arbitrary headers and request bodies, as well as receive response bodies either as fully buffered values or as efficient streams.
-
-h2. Installation
-
-First, clone the @clj-garden@ project from GitHub and build the @clj-http-client@ subproject with @ant@:
-
- $ git clone git://github.com/mmcgrana/clj-garden.git
- $ cd clj-http-client
- $ ant
-
-This last step will produce a @clj-http-client.jar@ file, which you should include in your path. You will additionally need to include the following 4 dependencies in your path, which can be downloaded from the listed sites:
-
- * "commons-httpclient-3.1.jar":http://hc.apache.org/downloads.cgi (be sure to select version 3.1)
- * "commons-codec-1.3.jar":http://commons.apache.org/downloads/download_codec.cgi
- * "commons-io-1.4.jar":http://commons.apache.org/downloads/download_io.cgi
- * "commons-logging-1.1.1.jar":http://commons.apache.org/downloads/download_logging.cgi
-
-h2. Usage
-
-Once you have installed the 5 needed jars, you can experiment with the library at the REPL:
-
- user=> (use 'clj-http-client.core)
-
-To @GET@ the Clojure home page:
-
- user=> (http-get "http://clojure.org")
-
-The @http-get@ function fetches the specified URL and returns a 3-tuple of the response status code, headers, and body as a String. You can use Clojure destructuring to conveniently access the individual elements:
-
- user=> (let [[status headers body] (http-get "http://clojure.org")] status)
-
-@http-get@ is useful when you want the response body as a @String@, i.e. when you are interested in the text of the page. You may however be fetching non-textual data, in which case you should use a different function. If the response body will be reasonably small, you can use:
-
- user=> (http-get-bytes "http://clojure.corg/space/showimage/favicon.png")
-
-This will return a buffered byte array for the response body tuple element.
-If you are fetching large files, you may want to stream them instead of buffering them into memory. Here I use Apache Commons IO Utils to pipe a binary response to a file as it is being downloaded:
-
- user=>
-
-Note the the syntax for @http-get-stream@ is slightly different that for the other function. This is necessary because all operations on the streaming HTTP response need to occur within the scope of the @http-get-stream@ function so that it can assure IO resources are appropriately freed upon completion.
-
- user=> (import 'org.apache.commons.io.IOUtils 'java.io.FileInputStream)
- user=> (http-get-stream "http://clojure.org/space/showimage/clojure-icon.gif"
- (fn [status headers stream]
- (IOUtils/copy stream (FileOutputStream. "clojure.gif"))))
-
-@TODO: document other HTTP methods, request body handling@.
-
----
-
-Copyright (c) 2009 Mark McGranaghan and released under an MIT license.
View
@@ -1,3 +0,0 @@
-Finish documentation
-Upgrade to HttpComponents 4
-Test against a Ring echo server
View
@@ -1,19 +0,0 @@
-<project name="clj-http-client" default="jar">
- <description>
- Pack all sources into a JAR.
- </description>
-
- <property name="jarfile" location="clj-http-client.jar"/>
-
- <target name="clean"
- description="Remove generated files and directories.">
- <delete file="${jarfile}"/>
- </target>
-
- <target name="jar"
- description="Create jar file.">
- <jar jarfile="${jarfile}">
- <fileset dir="./src" includes="clj_http_client/**/*.clj"/>
- </jar>
- </target>
-</project>
@@ -1,138 +0,0 @@
-(ns clj-http-client.core
- (:use (clojure.contrib fcase except def))
- (:import (org.apache.commons.httpclient
- HttpClient DefaultHttpMethodRetryHandler Header
- HttpMethod HttpMethodBase)
- (org.apache.commons.httpclient.methods
- HeadMethod GetMethod PutMethod PostMethod DeleteMethod
- RequestEntity ByteArrayRequestEntity FileRequestEntity
- InputStreamRequestEntity StringRequestEntity EntityEnclosingMethod)
- (org.apache.commons.httpclient.methods.multipart
- MultipartRequestEntity)
- (org.apache.commons.httpclient.params HttpMethodParams)
- (org.apache.commons.httpclient.cookie CookiePolicy)
- (org.apache.commons.io IOUtils)
- (java.io File InputStream)))
-
-(defn- headers-map
- "Returns a header Map with string keys and string values, for the given array
- of Headers from a response."
- [headers-arr]
- (reduce
- (fn [memo #^Header header]
- (assoc memo (.getName header) (.getValue header)))
- {}
- headers-arr))
-
-(defn- apply-headers
- "Add the headers in the given Map to the given http method."
- [#^HttpMethod method headers]
- (doseq [[name value] headers]
- (.setRequestHeader method name value)))
-
-(defvar- ByteArray (class (make-array (Byte/TYPE) 0))
- "Class for primitive byte arrays.")
-
-(defn- #^RequestEntity request-entity
- "Returns an instance implementing RequestEntity for use in put/post requests."
- [[body & [content-type & [encoding]]]]
- (instance-case body
- String (StringRequestEntity. body content-type encoding)
- File (FileRequestEntity. body content-type)
- ByteArray (ByteArrayRequestEntity. body content-type)
- InputStream (InputStreamRequestEntity. #^InputStream body
- #^String content-type)
- (throwf "Unrecognized body: %s" body)))
-
-(defn- method-body
- "Returns the response body for the given method as a String"
- [#^HttpMethodBase method]
- (when-let [rbody (.getResponseBodyAsStream method)]
- (IOUtils/toString rbody (.getResponseCharSet method))))
-
-(defn- http-execute-method
- "Generalized http request."
- [#^HttpMethod method headers body-args handler]
- (let [client (HttpClient.)
- method-params (.getParams method)]
- (.setParameter method-params HttpMethodParams/RETRY_HANDLER
- (DefaultHttpMethodRetryHandler.))
- (.setCookiePolicy method-params CookiePolicy/IGNORE_COOKIES)
- (when headers (apply-headers method headers))
- (when body-args
- (.setRequestEntity #^EntityEnclosingMethod method
- (request-entity body-args)))
- (try
- (let [status (.executeMethod client method)
- headers (headers-map (.getResponseHeaders method))]
- (handler status headers method))
- (finally
- (.releaseConnection method)))))
-
-(defn http-head
- "Returns a [status headers] tuple corresponding to the response from a
- HEAD request to the given url, optionally qualified by the given headers."
- [url & [headers]]
- (http-execute-method (HeadMethod. url) headers nil
- (fn [h s method] [h s])))
-
-(defn http-get
- "Returns a [status headers body-string] tuple corresponding to the response
- from the given url."
- [url & [headers]]
- (http-execute-method (GetMethod. url) headers nil
- (fn [s h method] [s h (method-body method)])))
-
-(defn http-get-bytes
- "Returns a [status headers body-byte-array] tuple corresponding to the
- response from the given url."
- [url & [headers]]
- (http-execute-method (GetMethod. url) headers nil
- (fn [s h #^HttpMethod method]
- [s h (IOUtils/toByteArray (.getResponseBodyAsStream method))])))
-
-(defn http-get-stream
- "Experimental. Invokes the given handler with the status, headers, and
- response body input stream for a request to the given url with the given
- headers."
- ([url headers handler]
- (http-execute-method (GetMethod. url) headers nil
- (fn [s h #^HttpMethod get-method]
- (with-open [b-stream (.getResponseBodyAsStream get-method)]
- (handler s h b-stream)))))
- ([url handler]
- (http-get-stream url {} handler)))
-
-(defn- headers-and-body-args
- "Returns a tuple of [headers body-args] corresponding to the given
- request-args"
- [request-args]
- (if (map? (first request-args))
- [(first request-args) (rest request-args)]
- [nil request-args]))
-
-(defn http-put
- "Returns a [status headers body] tuple corresponding to the response from a
- PUT request to the given url. request-args are of the form
- [headers? put-entity content-type? encoding?]"
- [url & request-args]
- (let [[headers body-args] (headers-and-body-args request-args)]
- (http-execute-method (PutMethod. url) headers body-args
- (fn [s h put-method] (method-body put-method)))))
-
-(defn http-post
- "Returns a [status headers body] tuple corresponding to the response from a
- POSt request to the given url. request-args are of the form
- [headers? post-entity content-type? encoding?]"
- [url & request-args]
- (let [[headers body-args] (headers-and-body-args request-args)]
- (http-execute-method (PostMethod. url) headers body-args
- (fn [s h post-method] [s h (method-body post-method)]))))
-
-(defn http-delete
- "Returns a [status headers body] tuple corresponding to the response
- from a DELETE request to the given url, optionally qualified by the given
- headers."
- [url & [headers]]
- (http-execute-method (DeleteMethod. url) headers nil
- (fn [s h delete-method] [s h (method-body delete-method)])))
View
@@ -1,30 +0,0 @@
-GitCred attempts to measure the 'cred' of GitHub users. It works by fetching followers data from GitHub pages and then applying the classic PageRank algorithm to the resulting users/followers graph.
-
-You can find the results for a recent GitCred computation in `RESULTS.txt`. These values are logarithmically scaled and normalized so that they correspond to the familiar Google PageRank measure.
-
-Implementation Notes
---------------------
-
-GitCred itself is implemented in Clojure. It leverages the Jung graph library to perform the actual PageRank computations and the FleetDB database to manage and persist the graph state.
-
-Reproducing the Results
------------------------
-
-To run the GitCred calculations yourself, you will need the following dependencies in your classpath:
-
-* `clojure.jar` & `clojure-contrib.jar`: recent versions from the masters at `http://github.com/richhickey`
-* `clj-stacktrace.jar`: , `http://github.com/mmcgrana/clj-stacktrace`
-* `fleetdb.jar`: `http://cloud.github.com/downloads/mmcgrana/fleetdb/fleetdb.jar`
-* `jung-*.jar` and their associated dependencies: `http://sourceforge.net/projects/jung/files/`
-* `src/` in this distribution
-
-Then to start populating the user/followers graph database, run:
-
- $ clj bin/fetch.clj /path/to/data.fdb
-
-If the data file does not exist it will be created for you. If it does exist the database will be loaded from it and fetching will continue from there.
-
-Once you have some data in the database, you can generate results with:
-
- $ clj bin/write.clj /path/to/data.fdb /path/to/results.txt
-
Oops, something went wrong.

0 comments on commit 23873fd

Please sign in to comment.