Switch branches/tags
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
71 lines (51 sloc) 3.04 KB

Overview document set visualization system -- prototype

Welcome! This is a prototype, pre-alpha distribution of the Overview open source document set visualization and exploration tool. As of 2013 this system is no longer maintained. See the production instead.

Getting started


The quick version:

  • You will need Ruby and Java installed.
  • Overview requires the full text of all input documents in a single CSV file, myfile.csv (see format below)
  • There are sample document set CSV files available from
  • First, preprocess the CSV file: myfile
  • Then fire up the visualization: myfile

For Mac, Windows, Linux

  • Mac: should just work
  • Windows: use the batch files something.bat instead of the shell scripts
  • Linux: replace visualization/lib/swt.jar with the appropriate version for your operating system.

Loading your own documents

Again, see

  • If you have a folder full of .pdf or .txt files, use the script
  • If you want to upload that folder to DocumentCloud first, use the ````` script
  • If you can convert your documents to the .csv format below, you can use Overview on just about anything

CSV file format

Overview takes a csv of the document text as input, one document per row. The simplest possible format that Overview will read has exactly one column named "text":

this is the content of document the first
and here is the text of document the second

This will work, but if you later add documents to this file, your saved tags will break, because the tags are based on row numbers if you don't have a "uid" field like this:

UNIQUEID_AAA, this is the content of document the first
UNIQUEID_BBB,and here is the text of document the second

The uid field can be any unique identifier, such as a hash of the document text. Finally, if you want Overview to display the document in its embedded browser instead of just showing the text, you can add a URL field.

UNIQUEID_AAA, this is the content of document the first,
UNIQUEID_BBB,and here is the text of document the second,

Over view does not do any sort of web scraping with this URL, it just uses it to display the document.

The "text" field for each document has to be quoted and escaped according to the normal CSV rules if the document text runs more than one line or has commas in it. HTML text is fine, because Overview simply strips all tags before processing. There is no hard upper limit on the number of documents, but the current UI gets a bit bogged down at about the 10,000 to 20,000 range.




need help? ask!