db/describe.sh
db/join_schema.txt
db/main_schema.txt
db/primary_schema.txt
db/tables.graffle
db/tables.pdf
scratch.R
setup.sql
.DS_Store
@@ -1,21 +1,45 @@
ACRP Notes
==========
+This project contains the analysis of data from the Australian Common
+Reader Project in the form of snippets of R and SQL code as well as some
+resulting graphs and tables.
-Methodology
------------
+Importantly, the actual data used in this analysis is *NOT* included in this
+project as I do not have the rights to redistribute it. The references to
+SQL databases and tables are for a local copy of the MySQL database I was given
+access to by the ACRP.
-A simple first question that can be asked of the data is whether there exist
-pairs of books (A,B) such that both A and B alone have a relatively large
-number of readers but there are few readers which have read both A and B. Pairs
-of this type can be further analysed to see whether certain books predict the
-reading of A or B and whether reading is clustered that can be understood
-via reader's occupation, genre, gender, etc.
+The rest of this file is some notes I made for myself while learning about
+the database and analysing it.
+
+Database Structure
+------------------
+The files `*_schema.txt` in the `db` directory are dumps of the MySQL database
+by the query in the `describe.sh` shell script.
+
+I turned the important parts of these schema into a PDF file
+-- `db/table.pdf` -- using OmniGraffle 5. The `.graffle` file is also
+available.
Set up
------
Several tables are created during analysis so as to speed things up. Once the
-original ACRP database is loaded, run `split_pairs.sql` to build the extra
-tables required before running `analysis.sql`.
+original ACRP database is loaded, run `setup.sql` to build the extra
+tables required before running and of the R scripts for analysis.
+
+Results
+-------
+The file `RESULTS.markdown` contains some useful counts and IDs collected
+for reference as well as some notes on some early plots.
+
+Seminar May 2008
+----------------
+The directory `seminarMay08` contains R analysis scripts and the resulting
+PDF plots used in the seminar Julieanne gave to her faculty in May 2008.
+
+Notice that as I learned more about RMySQL I began moving more and more of the
+database-related code into R in an effort to make the R scripts completely
+self-contained.
Configuration
-------------
README.markdown
vis/.DS_Store
vis/applet/.DS_Store
@@ -5,16 +5,16 @@
<!-- charset must remain utf-8 to be handled properly by Processing -->
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
- <title>vis : Built with Processing</title>
+ <title>Visualisation of Lambton Miners' and Mechanics' Institute</title>
<style type="text/css">
/* <![CDATA[ */
body {
- margin: 60px 0px 0px 55px;
+ width: 100%;
font-family: verdana, geneva, arial, helvetica, sans-serif;
font-size: 11px;
- background-color: #ddddcc;
+ background-color: white;
text-decoration: none;
font-weight: normal;
line-height: normal;
@@ -26,12 +26,20 @@
a:active { color: #3399cc; text-decoration: underline; }
a:hover { color: #3399cc; text-decoration: underline; }
+ #content {
+ margin: 0 auto 0 auto;
+ width: 600px;
+ }
+ h1 { text-align: center;}
+ dt { font-weight: bold; }
+ address { color: #555555; text-align: center;}
/* ]]> */
- </style>
-
+ </style>
+
</head>
<body>
<div id="content">
+ <h1>Book Similarity Visualisation</h1>
<div id="vis_container">
<!--[if !IE]> -->
@@ -87,18 +95,59 @@
<!--<![endif]-->
</div>
-
+ <h2>Layout</h2>
<p>
-
+ Each circle represents a book found in the Lambton Miners'
+ and Mechanics' Institute.
+ Only books that were borrowed by at least 20 people across
+ all libraries in the ACRP database are shown.
+ <p>
+ The size of a circle is indicative of the number of its
+ borrowers.
</p>
-
<p>
- Source code: <a href="vis.pde">vis</a> <a href="Book.pde">Book</a> <a href="Graph.pde">Graph</a> <a href="PlotView.pde">PlotView</a>
+ The similarity of any two books is defined to be the proportion
+ of their borrowers they have in common.
+ The books are arranged so that similar book are placed closer
+ together than books that are less similar.
</p>
-
+ <h2>Controls</h2>
<p>
- Built with <a href="http://processing.org" title="Processing.org">Processing</a>
+ Hovering over a book's circle will reveal its title.
+ The number in brackets is the number of borrowers.
</p>
+ <p>
+ Clicking on a circle will show its <quo;neighbours&rtquo;
+ — those books that are most similar to it.
+ </p>
+ <p>
+ The number of books and neighbours displayed are controlled
+ by the two sliders at the bottom of the visualisation.
+ <dl>
+ <dt>Borrowers</dt>
+ <dd>
+ Only books with at least this many borrowers are displayed.
+ </dd>
+ <dt>Similarity</dt>
+ <dd>
+ Links to neighbouring books are only shown if the two books
+ share at least this proportion of their borrowers.
+ </dd>
+ </dl>
+ </p>
+ <h2>More Information</h2>
+ <p>
+ A longer description of this visualisation and the data it is
+ based on can be found at
+ <a href="http://conflate.net/inductio/2008/06/visualising-reading/"
+ title="Technical details about this visualisation">
+ <i>Inductio Ex Machina</i>
+ </a>
+ </p>
+ <address>
+ Built with <a href="http://processing.org" title="Processing.org">Processing</a>
+ by <a href="http://mark.reid.name" title="Mark Reid's Home Page">Mark Reid</a>.
+ </address>
</div>
</body>
</html>
vis/applet/index.html
vis/applet/vis.jar
@@ -31,7 +31,7 @@ public void setup() {
20,20,width-40,height-70,
graph.xmin, graph.ymin, graph.xmin + vwidth, graph.ymin + vheight
);
- view.setTitle("Books from the Lambton Library");
+ view.setTitle("Books from the Lambton Miners' and Mechanics' Institute");
for(int i=0; i < graph.books.size(); i++) {
view.add((Drawable) graph.books.get(i));
@@ -42,7 +42,7 @@ public void setup() {
controlP5.addSlider("weightThreshold", 0.0f, 1.0f, 0.25f, width-220, height-30, 200, 20);
controlP5.addTextlabel("sim", "Similarity", width-120, height-30);
controlP5.addSlider("readerThreshold", 0, 100, 0, 20, height-30, 200, 20);
- controlP5.addTextlabel("read", "Readers", 100, height-30);
+ controlP5.addTextlabel("read", "Borrowers", 100, height-30);
smooth();
}
@@ -76,12 +76,12 @@ public void draw() {
}
}
- drawTitles(active);
if(selected != null) {
+ drawNeighbours(selected, false);
drawTitle(selected, 0);
- drawNeighbours(selected, true);
}
+ drawTitles(active);
}
public void drawTitle(Book book, int offset) {
vis/applet/vis.java
@@ -31,7 +31,7 @@ void setup() {
20,20,width-40,height-70,
graph.xmin, graph.ymin, graph.xmin + vwidth, graph.ymin + vheight
);
- view.setTitle("Books from the Lambton Library");
+ view.setTitle("Books from the Lambton Miners' and Mechanics' Institute");
for(int i=0; i < graph.books.size(); i++) {
view.add((Drawable) graph.books.get(i));
@@ -42,7 +42,7 @@ void setup() {
controlP5.addSlider("weightThreshold", 0.0, 1.0, 0.25, width-220, height-30, 200, 20);
controlP5.addTextlabel("sim", "Similarity", width-120, height-30);
controlP5.addSlider("readerThreshold", 0, 100, 0, 20, height-30, 200, 20);
- controlP5.addTextlabel("read", "Readers", 100, height-30);
+ controlP5.addTextlabel("read", "Borrowers", 100, height-30);
smooth();
}
@@ -76,12 +76,12 @@ void draw() {
}
}
- drawTitles(active);
if(selected != null) {
+ drawNeighbours(selected, false);
drawTitle(selected, 0);
- drawNeighbours(selected, true);
}
+ drawTitles(active);
}
void drawTitle(Book book, int offset) {
vis/applet/vis.pde
@@ -31,7 +31,7 @@ void setup() {
20,20,width-40,height-70,
graph.xmin, graph.ymin, graph.xmin + vwidth, graph.ymin + vheight
);
- view.setTitle("Books from the Lambton Library");
+ view.setTitle("Books from the Lambton Miners' and Mechanics' Institute");
for(int i=0; i < graph.books.size(); i++) {
view.add((Drawable) graph.books.get(i));
@@ -42,7 +42,7 @@ void setup() {
controlP5.addSlider("weightThreshold", 0.0, 1.0, 0.25, width-220, height-30, 200, 20);
controlP5.addTextlabel("sim", "Similarity", width-120, height-30);
controlP5.addSlider("readerThreshold", 0, 100, 0, 20, height-30, 200, 20);
- controlP5.addTextlabel("read", "Readers", 100, height-30);
+ controlP5.addTextlabel("read", "Borrowers", 100, height-30);
smooth();
}
@@ -76,12 +76,12 @@ void draw() {
}
}
- drawTitles(active);
if(selected != null) {
+ drawNeighbours(selected, false);
drawTitle(selected, 0);
- drawNeighbours(selected, true);
}
+ drawTitles(active);
}
void drawTitle(Book book, int offset) {
vis/vis.pde
TODO.markdown
analysis.out
analysis.sql
create_matrix.rb
describe.sh
feature_analysis.R
important_schema.txt
join_schema.txt
loanview.sql
plots/works-linPCA-colLibrary.sql
primary_schema.txt
split_pairs.sql
tables.graffle
tables.pdf
test.csv
aba37f4dec38a8503c400fba834246f3b6d7c571
Mark
mark@threewordslong.com
http://github.com/mreid/acrp/commit/9e706f97c6948987c547cf262af3bba0b34529c5
9e706f97c6948987c547cf262af3bba0b34529c5
2008-06-10T04:05:56-07:00
2008-06-10T04:05:56-07:00
Restructure and clean up
6a6d53611c3a8e13391272aed3a7eecdb2dfff76
Mark
mark@threewordslong.com