Tools

This page documents the tools to be found in the tools/ directory.

pdf2text.py

From docs/index.html:

pdf2txt.py extracts text contents from a PDF file. It extracts all the texts that are to be rendered programmatically, It cannot recognize texts drawn as images that would require optical character recognition. It also extracts the corresponding locations, font names, font sizes, writing direction (horizontal or vertical) for each text portion. You need to provide a password for protected PDF documents when its access is restricted. You cannot extract any text from a PDF document which does not have extraction permission.

For non-ASCII languages, you can specify the output encoding (such as UTF-8).

Note: Not all characters in a PDF can be safely converted to Unicode.

Examples

$ pdf2txt.py samples/naacl06-shinyama.pdf -o output.html
(extract text as an HTML file whose filename is output.html)

$ pdf2txt.py -c euc-jp samples/jo.pdf -D V -o output.html
(extract a Japanese HTML file in vertical writing, CMap is required)

$ pdf2txt.py -P mypassword secret.pdf -o output.txt
(extract a text from an encrypted PDF file)

Options

-o filename

Specifies the output file name. By default, it prints the extracted contents to stdout in text format.

-p pageno[,pageno,...]

Specifies the comma-separated list of the page numbers to be extracted. Page numbers are starting from one. By default, it extracts texts from all the pages.

-c codec

Specifies the output codec for non-ASCII texts.

-t type

Specifies the output format. The following formats are currently supported.

* html: HTML format. (Default)
* text: TEXT format.
* sgml : SGML format.
* tag : “Tagged PDF” format. A tagged PDF has its own contents annotated with HTML-like tags. pdf2txt tries to extract its content streams rather than inferring its text locations. Tags used here are defined in the PDF specification (See §10.7 “Tagged PDF”).

-D direction

-M char_margin

-L line_margin

-W word_margin

These are the parameters used for layout analysis. In an actual PDF file, texts might be split into several chunks in the middle of its running, depending on the authoring software. Therefore, text extraction needs to splice text chunks. In the figure below, two text chunks whose distance is closer than the char_margin (shown as M) is considered continuous and get grouped into one. Also, two lines whose distance is closer than the line_margin (L) is grouped as a text box, which is a rectangular area that contains a “cluster” of texts. Furthermore, it may be required to insert blank characters (spaces) as necessary if the distance between two words is greater than the word_margin (W), as a blank between words might not be represented as a space, but indicated by the positioning of each word.

Each value is specified not as an actual length, but as a proportion of the length to the size of each character in question. The default values are M = 1.0, L = 0.3, and W = 0.2, respectively.

-s scale

Specifies the output scale. Can be used in HTML format only.

-m maxpages

Specifies the maximum number of pages to extract. By default, it extracts all the pages in a document.

-P password

Provides the user password to access PDF contents.

-C CMap directory

Specifies the path of CMap directory. CMap is needed when extracting non-ASCII texts (especially in Asian languages). The CMap location can be also specified with CMAP_PATH environment variable.

-d

Increases the debug level.

dumppdf.py

From docs/index.html:

dumppdf.py dumps the internal contents of a PDF file in pseudo-XML format. This program is primarily for debugging purpose, but it’s also possible to extract some meaningful contents (such as images).

Examples

$ dumppdf.py -a foo.pdf
(dump all the headers and contents, except stream objects)

$ dumppdf.py -T foo.pdf
(dump the table of contents)

$ dumppdf.py -r -i6 foo.pdf > pic.jpeg
(extract a JPEG image)

Options

-a

Instructs to dump all the objects. By default, it only prints the document trailer (like a header).

-i objno,objno, ...

Specifies PDF object IDs to display. Comma-separated IDs, or multiple -i options are accepted.

-p pageno,pageno, ...

Specifies the page number to be extracted. Comma-separated page numbers, or multiple -p options are accepted. Note that page numbers start from one, not zero.

-r (raw)

-b (binary)

-t (text)

Specifies the output format of stream contents. Because the contents of stream objects can be very large, they are omitted when none of the options above is specified.

With -r option, the “raw” stream contents are dumped without decompression. With -b option, the decompressed contents are dumped as a binary blob. With -t option, the decompressed contents are dumped in a text format, similar to repr() manner. When -r or -b option is given, no stream header is displayed for the ease of saving it to a file.

-T

Shows the table of contents.

-P password
Provides the user password to access PDF contents.

-d

Increases the debug level.