pdftables - a library for extracting tables from PDF files
This post on the ScraperWiki blog describes the algorithms used in pdftables, and something of its genesis. This README gives more technical information.
pdftables uses pdfminer to get information on the locations of text elements in a PDF document. pdfminer was chosen as a base because it provides information on the full range of page elements in PDF files, including graphical elements such as lines. Although the algorithms currently used do not use these elements they are planned for future work. As a purely Python library, pdfminer is very portable. The downside of pdfminer is that it is slow, perhaps an order of magnitude slower than alternative C based libraries.
You need poppler and Cairo. On a Ubuntu and friends you can go:
sudo apt-get -y install python-poppler python-cairo
Then we can install the
pip-able requirements from the
pip install -r requirements.txt
First we get a file object to a PDF:
filepath = 'example.pdf' fileobj = open(filepath,'rb')
Then we create a PDF element from the file object:
from pdftables.pdf_document import PDFDocument doc = PDFDocument.from_fileobj(fileobj)
Then we use the
get_page() method to select a single page from the document:
from pdftables.pdftables import page_to_tables page = doc.get_page(pagenumber) tables = page_to_tables(page)
You can also loop over all pages in the PDF using
from pdftables.pdftables import page_to_tables for page_number, page in enumerate(doc.get_pages()): tables = page_to_tables(page)
Now you have a TableContainer object, you can convert it to ASCII for quick previewing:
from pdftables.display import to_string for table in tables: print to_string(table.data)
table.data is a table that has been found, in the form of a list of lists of strings
(ie: a list of rows, each containing the same number of cells).
Command line tool
pdftables includes a command line tool for diagnostic rendering of pages and tables, called
This is installed if you
pip install pdftables, or you manually run
$ pdftables-render example.pdf
This creates separate PNG and SVG files for each page of the specified PDF, in
svg/, with three disagnostic displays per page.
Files and folders:
. |-fixtures | |-sample_data |-pdftables |-test
fixtures contains test fixtures, in particular the sample_data directory
contains PDF files which are installed from a different repository by running
The fixtures are currently unavailable as they are held on a private repository
We're also using data from http://www.tamirhassan.com/competition/dataset-tools.html which is also installed by the download script.
pdftables contains the core code files
test contains tests
pdftables.py - this is the core of the pdftables library
counter.py - implements collections.Counter for the benefit of Python 2.6
display.py - prettily prints a table by implementing the
numpy_subset.py - partially implements
numpy.average to avoid a large dependency on numpy.
pdf_document.py - implements PDFDocument to abstract away the underlying PDF class, and ease any conversion to a different underlying PDF library to replace PDFminer