BitCurator Access Webtools: Browse disk images and file system metadata in a web service
The BitCurator Access Webtools project allows users to browse file systems contained within disk images using a web browser. It is intended to support access requirements in libraries, archives, and museums preserving born-digital materials extracted from source media as raw or forensically-packaged disk images.
The service uses open source libraries and toolkits including The Sleuth Kit, PyTSK, and the Flask web microservices framework. It uses PyLucene along with format-specific text-extraction tools to index the contents of files contained in disk images, allowing users to search for relevant content without individually inspecting files.
This repository includes a simple build script that deploys the web service as in a VirtualBox VM using Vagrant. It includes several sample images (in the "disk-images" directory) to get you started.
Find out more at https://github.com/BitCurator/bitcurator-access/wiki
This software uses Vagrant to provision a virtual machine in which bitcurator-access-webtools runs. To start, make sure you have VirtualBox and Vagrant installed on your Windows, Mac, or Linux host:
To clone out this repository from git, you'll also need git installed.
On Windows, download and install git from:
On MacOS, run the following command in a terminal and click through the prompts:
On Debian-based variants of Linux (including Ubuntu), run the following from a terminal:
sudo apt-get install git
IMPORTANT: On Windows, you must make Git check out files with Unix-style line endings in order for the VM to run properly. After installing git, run the following in a console (cmd prompt):
git config --global core.autocrlf false
Check out the bitcurator-access-webtools software from git:
git clone https://github.com/bitcurator/bitcurator-access-webtools
Once you have the source code (and you have unzipped it if you're using a release), change directory into bitcurator-access-webtools, and make sure the associated Vagrant box (bentu/ubuntu-18.04) is added:
cd bitcurator-access-webtools vagrant box add bento/ubuntu-18.04
You will be prompted for a provider. Select 2) virtualbox by typing '2' and hitting enter.
The first time you run vagrant box add bento/ubuntu-18.04 may take some time. Updates to the base vagrant box (a headless Ubuntu environment) are generally issued every couple of weeks. Vagrant will warn you when your box is out of date. (Note: You only need to run the "vagrant box add" the first time after installing Vagrant. You may be promted to run the command "vagrant box update" in future sessions, however, in order to keep the box up to date).
The bitcurator-access-webtools application can be used to view raw (.dd) and EWF (.E01) images containing FAT16, FAT32, NTFS, ext2/3/4, and HFS+ file systems. The application includes several sample images (in the bitcurator-access-webtools/disk-images directory) for testing. You can place additional images in this directory (and remove these test images) as needed.
Once the base box is downloaded, you can start the service by running the command:
from within the bitcurator-access-webtools directory. This step can take a long time the first time you run the software (15-30 minutes depending on your computer). The installation script will provide feedback in the console as it installs each package. Once the virtual machine has been provisioned, open a web browser on your host and navigate to:
to see the bitcurator-access-webtools service running. The first time you bring the service up, you must follow the steps in the next section to enable all browsing and search features.
Terminating the bitcurator-access-webtools service and virtual machine
If you need to stop the service, you can type:
in the bitcurator-access-webtools directory in the console or terminal on your host machine. The next time you issue the "vagrant up" command, the VM will restart in its previous state.
If you need to delete the VM entirely, you can the the following command after halting the VM:
If you wish to build a new VM with updated sources, simply delete the bitcurator-access-webtools directory after halting and destroying the previous VM, and clone or download the current sources from GitHub.
This is a Flask application that is deployed automatically into an appropriately configured Ubuntu 18.04 virtual machine. It has been tested with Python 2.7.3, Flask 0.11, Jinja2, and Postgres 9.3 (but will likely work with other versions). Python 3 should also work. Several other libraries and tools are required, including AFFLIB (v3.7.4 or later), libewf (20140427 or later), The Sleuth Kit (4.1.3 or later), and PyTSK.
Some of these dependencies have existing apt or pip packages in Ubuntu. Others do not. To simplify the process, we've written a bootstrap script that updates and upgrades all the necessary packages, compiles and installs the necessary source packages, and sets up the database. This script is located in the provision directory, and is only run the first time you execute the vagrant up command.
The latest documentation can be found on the BitCurator Access wiki page at https://github.com/BitCurator/bitcurator-access/wiki.
Or, follow this link to a direct download of the quick start guide:
The BitCurator logo, BitCurator project documentation, and other non-software products of the BitCurator team are subject to the the Creative Commons Attribution 4.0 Generic license (CC By 4.0).
Unless otherwise indicated, software items in this repository are distributed under the terms of the GNU General Public License, Version 3. See the text file "COPYING" for further details about the terms of this license.
In addition to software produced by the BitCurator team, BitCurator packages and modifies open source software produced by other developers. Licenses and attributions are retained here where applicable.