HTRC Workset Toolkit
HTRC Workset Toolkit provides tools for interacting with and analyzing volumes in the HathiTrust Digital Library:
- Volume Download (
- Metadata Download (
- Pre-built Analysis Workflows (
- Export of volume lists (
Each tool operates on a workset, which is a collection of volumes, pages, or catalog records.
The tools also assist with the HTRC Data Capsule, enabling you to download volumes to the secure mode of the capsule for analysis.
For usage instructions and documentation see [https://htrc.github.io/HTRC-WorksetToolkit/cli.html].
For developers, the Workset Toolkit provides ways to test algorithms that will be run in the secure mode of the Data Capsule. It also provides methods for accessing the bibliographic records for HathiTrust volumes and ways to resolve catalog records for multivolume collections. It has the following components:
- An access layer for the Bibliographic API (
- An access layer for the Data API (
- Pre-built analysis workflows (
- Provenance tracking for verification of non-consumptive exports (
- Mock testing interface for user-machine or maintenance-mode testing of
secure-mode commands (
- Utilities for record and volume resolution (
For documentation of the development libraries see [https://htrc.github.io/HTRC-WorksetToolkit/sdk.html].
Data Capsule usage
The HTRC Data Capsule allows for analysis of HathiTrust volumes. It is the only way to perform analysis on the raw OCR text of in-copyright works.
New users can register and configure a data capsule by following the HTRC Data Capsule Tutorial.
The HTRC Workset Toolkit will be pre-installed on Data Capsule images in the near future. Current data capsules will need to follow the installation instructions.
Download and install Anaconda Python. The HTRC Workset Toolkit is compatible with both Python 2.7 and 3.6, but we recommend using the 3.6 version for future compatibility.
After installing Anaconda, open a new terminal and type
pip install htrcto install the SDK.
git clone https://github.com/htrc/HTRC-WorksetToolkit.git
python setup.py develop
htrccommand will now refer to the code in this local repository.
- Run the unit tests with the command:
python setup.py test
- To revert to the PyPI version:
pip uninstall htrc pip install htrc
In order to update PyPI, you will need owner permissions, which are currently held by Samitha Liyanage and Jaimie Murdock.
- Create a
.pypirccontaining your username and password:
[distutils] index-servers = pypi [pypi] repository=https://upload.pypi.org/legacy/ username:USERNAME password:PASSWORD
python setup.py sdist uploadto upload the tarball.
python setup.py bdist_egg uploadto upload the egg file.
For usage instructions and documentation please see: [https://htrc.github.io/HTRC-WorksetToolkit/]
For a more detailed guide please see: [https://wiki.htrc.illinois.edu/x/NQBTAw.]