Get out of Excel free.
R Makefile
Latest commit 2fbec5f Aug 18, 2016 @jennybc jennybc committed on GitHub Merge pull request #6 from seaaan/patch-1
Fixed a few typos.

README.md

jailbreakr

Warning: This project is in the early scoping stages; do not use for anything other than amusement/frustration purposes

Data Liberator. To extract tabular data people put in nontabular structures in a program designed to hold tables.

Installation

Requires the development version of xml2 (for xml_find_lgl) as well as cellrangr and linen. Chances are you'll want rexcel too.

devtools::install_github(c("hadley/xml2",
                           "rsheets/linen",
                           "rsheets/cellranger",
                           "rsheets/rexcel",
                           "rsheets/jailbreakr"))

Goals

There are two large excel spreadsheet corpora; it would be nice to use these to get a feel for what fraction of spreadsheets we can handle or the range of non-table-like data out there.

the things people do to data

The first is the EUSES corpus of 4,447 spreadsheets (16,853 worksheets). This is all xls files (rather than xlsx) and therefore need either an xls -> xlsx conversion or support in jailbreakr for xls files.

The second, larger, one is the Enron corpus of 15,770 spreadsheets (79,983)

Roadmap

  • data structure package:

    • linen? General representation of spreadsheet data, plus some limited low-level operations on that data
    • depends on cell ranger, tibble
    • constructor function
    • print methods
    • subsetting, range extraction etc.
    • plot method - for quickly getting a feel for structure, or a shiny app
    • summary: this has n sheets, no formulae, 3 plots, etc, things about the references between the sheets?
    • where it came from (excel, googlesheet, etc), with filenames, reference ids etc.
    • probably needs references to handle multiple sheets and formulae within them, definitely if we need to do things with plots, but make them immutable at first?
    • md5 or other "id" so that we can see if the upstream source has changed. This is different for googlesheets where the id is properly baked into the sheet
  • low level packages:

    • googlesheets
    • rexcel
    • these depend on linen, and will have to provide things like ids and filenames to satisfy all the features that linen will do.
  • jailbreakr

    • uses output in linen format that is provided by googlesheets or rexcel

Ideas

Can we feed things through openrefine or something?