Data

Toby Crawley edited this page Apr 25, 2017 · 30 revisions

Clojars provides a couple of data feeds which you might find useful.

API

Restful API provides few endpoints which can be used to retrieve data for users, groups and artifacts. The API supports multiple output formats which can be selected using request’s Accept-header.

Example:
curl -H "Accept: application/edn" https://clojars.org/api/groups/cljsjs

Formats

Format Content-type
JSON (default) application/json
EDN application/edn
YAML application/x-yaml
Transit application/transit+json

Endpoints

User: https://clojars.org/api/users/{{user}}

https://clojars.org/api/users/deraen

Group’s artifacts: https://clojars.org/api/groups/{{group_name}}

https://clojars.org/api/groups/cljsjs

Artifact: https://clojars.org/api/artifacts/{{artifact_name}}

https://clojars.org/api/artifacts/leiningen

Artifact: https://clojars.org/api/artifacts/{{group_name}}/{{artifact_name}}

https://clojars.org/api/artifacts/cljsjs/react

Note that the download counts provided by the API are calculated once daily, at 07:00 UTC.

JSON search results

Add &format=json to the end of the normal search URL to get a JSON version of the results (click to see the format):

https://clojars.org/search?q=incanter&format=json

If the search query is invalid, the response will have a status of 400, and have only an error key (example).

XML search results

Add &format=xml to the end of the normal search URL to get a XML version of the results (click to see the format):

https://clojars.org/search?q=incanter&format=xml

If the search query is invalid, the response will have a status of 400, and have only an error element (example).

List of POM files

http://clojars.org/repo/all-poms.txt
http://clojars.org/repo/all-poms.txt.gz

It looks like this:

./compojure/compojure/0.3.2/compojure-0.3.2.pom
./compojure/compojure/0.4.0-SNAPSHOT/compojure-0.4.0-20100308.145053-1.pom
./compojure/compojure/0.4.0-SNAPSHOT/compojure-0.4.0-20100308.145053-2.pom

This list is regenerated once per hour.

List of jars and versions (in Leiningen syntax)

http://clojars.org/repo/all-jars.clj
http://clojars.org/repo/all-jars.clj.gz

It looks like this:

[compojure "0.3.2"]
[compojure "0.4.0-SNAPSHOT"]
[concurrent "1.3.4"]

This list is regenerated once per hour.

Useful extracts from the POMs

http://clojars.org/repo/feed.clj.gz

The format is Clojure maps, one per line. It looks like this:

{:versions ["1.1.0" "1.1.0-SNAPSHOT"], :scm {:connection "scm:git:git://github.com/technomancy/swank-clojure.git", :developer-connection "scm:git:ssh://git@github.com/technomancy/swank-clojure.git", :tag "da6cb50944ba95940559a249c9659f71747312fb", :url "http://github.com/technomancy/swank-clojure"}, :description "Swank server connecting Clojure to Emacs SLIME", :artifact-id "swank-clojure", :group-id "swank-clojure"}
{:versions ["1.0.0-SNAPSHOT"], :scm {:connection "scm:git:git://github.com/timothypratley/tjpext.git", :developer-connection "scm:git:ssh://git@github.com/timothypratley/tjpext.git", :tag "0afe7de60218d2a7fbf5ddd510a3644350b06cf6", :url "http://github.com/timothypratley/tjpext"}, :description "TJP Extensions for Clojure", :artifact-id "tjpext", :group-id "tjpext"}

This list is regenerated once per hour.

Lucene index of Maven artifacts

Generated by the Nexus Indexer. This should be usable by M2eclipse, Netbeans, Nexus etc for searching jars.

http://clojars.org/repo/.index/

See Sonatype’s blog post for more about how to use this.

Download stats

Daily download stats are calculated and stored at

“https://clojars.org/stats/downloads-yyyyMMdd.edn”

There is a combined file of downloads for all time at

https://clojars.org/stats/all.edn.

New stat files and the combined all.edn file are generated daily. Older stat files are immutable and can be cached indefinitely. N.B. There are some gaps in the date range. You can see a list of all stat files at

https://clojars.org/stats/.

Rsync the whole repository

If you want an offline copy of the entire repository, you can grab it or update it with rsync:

rsync -av --delete clojars.org::clojars my-wonderful-copy-of-clojars