Repository navigation
PDF Spec 07 Filters
Part of: PDF-Spec-07-Syntax
Navigation: PDF-Spec-07-Objects | PDF-Spec-Index | PDF-Spec-07-File-Structure
Stream filters are introduced in 7.3.8, "Stream Objects." An option when reading stream data is to decode it using a filter to produce the original non-encoded data. Whether to do so and which decoding filter or filters to use may be specified in the stream dictionary.
- EXAMPLE 1 If a stream dictionary specifies the use of an ASCIIHexDecode filter, an application reading the data in that stream should transform the ASCII hexadecimal-encoded data in that stream in order to obtain the
original binary data.
A conforming writer may encode data in a stream (for example, data for sampled images) to compress it or to convert it to a portable ASCII representation (or both). A conforming reader shall invoke the corresponding decoding filter or filters to convert the information back to its original form.
The filter or filters for a stream shall be specified by the Filter entry in the stream's dictionary (or the FFilter entry if the stream is external). Filters may be cascaded to form a pipeline that passes the stream through two or more decoding transformations in sequence. For example, data encoded using LZW and ASCII base-85 encoding (in that order) shall be decoded using the following entry in the stream dictionary:
EXAMPLE 2 /Filter [ /ASCII85Decode /LZWDecode ]
Some filters may take parameters to control how they operate. These optional parameters shall be specified by the DecodeParms entry in the stream's dictionary (or the FDecodeParms entry if the stream is external).
PDF supports a standard set of filters that fall into two main categories:
-
ASCII filters enable decoding of arbitrary binary data that has been encoded as ASCII text (see 7.2, "Lexical Conventions," for an explanation of why this type of encoding might be useful).
-
Decompression filters enable decoding of data that has been compressed. The compressed data shall be in binary format, even if the original data is ASCII text.
-
NOTE 1 ASCII filters serve no useful purpose in a PDF file that is encrypted; see 7.6, 'Encryption'.
-
NOTE 2 Compression is particularly valuable for large sampled images, since it reduces storage requirements and transmission time. Some types of compression are lossy, meaning that some data is lost during the encoding, resulting in a loss of quality when the data is decompressed. Compression in which no loss of data occurs is called lossless. Though somehow obvious it might be worth pointing out that lossy compression can only be applied to sampled image data (and only certain types of lossy compression for certain types of images). Lossless compression on the other hand can be used for any kind of stream.
The standard filters are summarized in Table 6, which also indicates whether they accept any optional parameters. The following sub-clauses describe these filters and their parameters (if any) in greater detail, including specifications of encoding algorithms for some filters.
Table 6 - Standard filters
| FILTER name | Parameters | Description |
|---|---|---|
| ASCIIHexDecode | no | Decodes data encoded in an ASCII hexadecimal representation, reproducing the original binary data. |
| ASCII85Decode | no | Decodes data encoded in an ASCII base-85 representation, reproducing the original binary data. |
| LZWDecode | yes | Decompresses data encoded using the LZW (Lempel-Ziv- Welch) adaptive compression method, reproducing the original text or binary data. |
| FlateDecode | yes | (PDF 1.2) Decompresses data encoded using the zlib/deflate compression method, reproducing the original text or binary data. |
| RunLengthDecode | no | Decompresses data encoded using a byte-oriented run-length encoding algorithm, reproducing the original text or binary data (typically monochrome image data, or any data that contains frequent long runs of a single byte value). |
| CCITTFaxDecode | yes | Decompresses data encoded using the CCITT facsimile standard, reproducing the original data (typically monochrome image data at 1 bit per pixel). |
| JBIG2Decode | yes | (PDF 1.4) Decompresses data encoded using the JBIG2 standard, reproducing the original monochrome (1 bit per pixel) image data (or an approximation of that data). |
| DCTDecode | yes | Decompresses data encoded using a DCT (discrete cosine transform) technique based on the JPEG standard, reproducing image sample data that approximates the original data. |
| JPXDecode | no | (PDF 1.5) Decompresses data encoded using the wavelet- based JPEG2000 standard, reproducing the original image data. |
| Crypt | yes | (PDF 1.5) Decrypts data encrypted by a security handler, reproducing the data as it was before encryption. |
- EXAMPLE 3 The following example shows a stream, containing the marking instructions for a page, that was compressed using the LZW compression method and then encoded in ASCII base-85 representation.
1 0 obj << /Length 534 /Filter [ /ASCII85Decode /LZWDecode ] >>
stream J..)6T?p&<!J9%\_[umg"B7/Z7KNXbN'S+,*Q/&"OLT'F LIDK#!n$"<Atdi\Vn%b%)&'cA*VnK\CJY(sF>c!Jnl@ RM]WM;jjH6Gnc75idkL5]+cPZKEBPWdR>FF(kj1\_R%W\_d &/jS!;iuad7h?[L-F$+]]0A3Ck*$I0KZ?;<)CJtqi65Xb Vc3\n5ua:Q/=0$W<#N3U;H,MQKqfg1?:lUpR;6oN[C2E4 ZNr8Udn.'p+?#X+1>0Kuk$bCDF/(3fL5]Oq)^kJZ!C2H1 'TO]Rl?Q:&'<5&iP!$Rq;BXRecDN[IJB,)o8XJOSJ9sD S]hQ;Rj@!ND)bD_q&C\g:inYC%)&u#:u,M6Bm%IY!Kb1+ ":aAa'S`ViJglLb8<W9k6Yl\0McJQkDeLWdPN?9A'jX* al>iG1p&i;eVoK&juJHs9%;Xomop"5KatWRT"JQ#qYuL, JD?M$0QP)lKn06l1apKDC@\qJ4B!!(5m+j.7F790m(Vj8 8l8Q:_CZ(Gm1%X\N1&u!FKHMB~> endstream endobj
EXAMPLE 4 The following shows the same stream without any filters applied to it. (The stream's contents are explained in 7.8.2, "Content Streams," and the operators used there are further described in clause 9, "Text".)
1 0 obj << /Length 568 >> stream 2 J BT /F1 12 Tf 0 Tc 0 Tw 72.5 712 TD [ ( Unfiltered streams can be read easily ) 65 ( , ) ] TJ 0 -14 TD [ ( b ) 20 ( ut generally tak ) 10 ( e more space than \311 ) ] TJ T* ( compressed streams . ) Tj 0 -28 TD [ ( Se ) 25 ( v ) 15 ( eral encoding methods are a ) 20 ( v) 25 ( ailable in PDF ) 80 ( . ) ] TJ 0 -14 TD ( Some are used for compression and others simply ) Tj T* [ ( to represent binary data in an ) 55 ( ASCII format . ) ] TJ T* ( Some of the compression filters are \ suitable ) Tj T* ( for both data and images , while others are \ suitable only ) Tj T* ( for continuous-tone images . ) Tj ET endstream endobj
The ASCIIHexDecode filter decodes data that has been encoded in ASCII hexadecimal form. ASCII hexadecimal encoding and ASCII base-85 encoding (7.4.3, "ASCII85Decode Filter") convert binary data, such as image data or previously compressed data, to 7-bit ASCII characters.
NOTE ASCII base-85 encoding is preferred to ASCII hexadecimal encoding. Base-85 encoding is preferred because it is more compact: it expands the data by a factor of 4 : 5, compared with 1 : 2 for ASCII hexadecimal encoding.
The ASCIIHexDecode filter shall produce one byte of binary data for each pair of ASCII hexadecimal digits (0-9 and A-F or a-f). All white-space characters (see 7.2, "Lexical Conventions") shall be ignored. A GREATER-THAN SIGN (3Eh) indicates EOD. Any other characters shall cause an error. If the filter encounters the EOD marker after reading an odd number of hexadecimal digits, it shall behave as if a 0 (zero) followed the last digit.
The ASCII85Decode filter decodes data that has been encoded in ASCII base-85 encoding and produces binary data. The following paragraphs describe the process for encoding binary data in ASCII base-85; the ASCII85Decode filter reverses this process.
The ASCII base-85 encoding shall use the ASCII characters ! through u ((21h) - (75h)) and the character z (7Ah), with the 2-character sequence ~> (7Eh)(3Eh) as its EOD marker. The ASCII85Decode filter shall ignore all white-space characters (see 7.2, "Lexical Conventions"). Any other characters, and any character sequences that represent impossible combinations in the ASCII base-85 encoding shall cause an error.
Specifically, ASCII base-85 encoding shall produce 5 ASCII characters for every 4 bytes of binary data. Each group of 4 binary input bytes, ( b 1 b 2 b 3 b 4 ), shall be converted to a group of 5 output bytes, ( c 1 c 2 c 3 c 4 c 5 ), using the relation
In other words, 4 bytes of binary data shall be interpreted as a base-256 number and then shall be converted to a base-85 number. The five bytes of the base-85 number shall then be converted to ASCII characters by adding 33 (the ASCII code for the character ! ) to each. The resulting encoded data shall contain only printable ASCII characters with codes in the range 33 (!) to 117 (u). As a special case, if all five bytes are 0, they shall be represented by the character with code 122 (z) instead of by five exclamation points ( ! ! ! ! ! ).
If the length of the data to be encoded is not a multiple of 4 bytes, the last, partial group of 4 shall be used to produce a last, partial group of 5 output characters. Given n (1, 2, or 3) bytes of binary data, the encoder shall first append 4 n zero bytes to make a complete group of 4. It shall encode this group in the usual way, but shall not apply the special z case. Finally, it shall write only the first n + 1 characters of the resulting group of 5. These characters shall be immediately followed by the ~> EOD marker.
The following conditions shall never occur in a correctly encoded byte sequence:
- The value represented by a group of 5 characters is greater than 2 32 - 1.
- A z character occurs in the middle of a group.
- A final partial group contains only one character.
The LZWDecode and ( PDF 1.2 ) FlateDecode filters have much in common and are discussed together in this sub-clause. They decode data that has been encoded using the LZW or Flate data compression method, respectively:
- LZW (Lempel-Ziv-Welch) is a variable-length, adaptive compression method that has been adopted as one of the standard compression methods in the Tag Image File Format (TIFF) standard. For details on LZW encoding see 7.4.4.2, "Details of LZW Encoding."
- The Flate method is based on the public-domain zlib/deflate compression method, which is a variablelength Lempel-Ziv adaptive compression method cascaded with adaptive Huffman coding. It is fully defined in Internet RFCs 1950, ZLIB Compressed Data Format Specification, and 1951, DEFLATE Compressed Data Format Specification (see the Bibliography).
Both of these methods compress either binary data or ASCII text but (like all compression methods) always produce binary data, even if the original data was text.
The LZW and Flate compression methods can discover and exploit many patterns in the input data, whether the data is text or images. As described later, both filters support optional transformation by a predictor function , which improves the compression of sampled image data.
- NOTE 1 Because of its cascaded adaptive Huffman coding, Flate-encoded output is usually much more compact than LZW-encoded output for the same input. Flate and LZW decoding speeds are comparable, but Flate encoding is considerably slower than LZW encoding.
- NOTE 2 Usually, both Flate and LZW encodings compress their input substantially. However, in the worst case (in which no pair of adjacent bytes appears twice), Flate encoding expands its input by no more than 11 bytes or a factor of 1.003 (whichever is larger), plus the effects of algorithm tags added by PNG predictors. For LZW encoding, the best case (all zeros) provides a compression approaching 1365 : 1 for long files, but the worstcase expansion is at least a factor of 1.125, which can increase to nearly 1.5 in some implementations, plus the effects of PNG tags as with Flate encoding.
Data encoded using the LZW compression method shall consist of a sequence of codes that are 9 to 12 bits long. Each code shall represent a single character of input data (0-255), a clear-table marker (256), an EOD marker (257), or a table entry representing a multiple-character sequence that has been encountered previously in the input (258 or greater).
Initially, the code length shall be 9 bits and the LZW table shall contain only entries for the 258 fixed codes. As encoding proceeds, entries shall be appended to the table, associating new codes with longer and longer sequences of input characters. The encoder and the decoder shall maintain identical copies of this table.
Whenever both the encoder and the decoder independently (but synchronously) realize that the current code length is no longer sufficient to represent the number of entries in the table, they shall increase the number of bits per code by 1. The first output code that is 10 bits long shall be the one following the creation of table entry 511, and similarly for 11 (1023) and 12 (2047) bits. Codes shall never be longer than 12 bits; therefore, entry 4095 is the last entry of the LZW table.
The encoder shall execute the following sequence of steps to generate each output code:
- a) Accumulate a sequence of one or more input characters matching a sequence already present in the table. For maximum compression, the encoder looks for the longest such sequence.
- b) Emit the code corresponding to that sequence.
- c) Create a new table entry for the first unused code. Its value is the sequence found in step (a) followed by the next input character.
- EXAMPLE 1 Suppose the input consists of the following sequence of ASCII character codes: 45 45 45 45 45 65 45 45 45 66
Starting with an empty table, the encoder proceeds as shown in Table 7.
Table 7 - Typical LZW encoding sequence
| Input sequence | Output code | Code added to table | Sequence represented by new code |
|---|---|---|---|
| - | 256 (clear-table) | - | - |
| 45 | 45 | 258 | 45 45 |
| 45 45 | 258 | 259 | 45 45 45 |
Table 7 - Typical LZW encoding sequence (continued)
| Input sequence | Output code | Code added to table | Sequence represented by new code |
|---|---|---|---|
| 45 45 | 258 | 260 | 45 45 65 |
| 65 | 65 | 261 | 65 45 |
| 45 45 45 | 259 | 262 | 45 45 45 66 |
| 66 | 66 | - | - |
| - | 257 (EOD) | - | - |
Codes shall be packed into a continuous bit stream, high-order bit first. This stream shall then be divided into bytes, high-order bit first. Thus, codes may straddle byte boundaries arbitrarily. After the EOD marker (code value 257), any leftover bits in the final byte shall be set to 0.
In the example above, all the output codes are 9 bits long; they would pack into bytes as follows (represented in hexadecimal):
To adapt to changing input sequences, the encoder may at any point issue a clear-table code, which causes both the encoder and the decoder to restart with initial tables and a 9-bit code length. The encoder shall begin by issuing a clear-table code. It shall issue a clear-table code when the table becomes full; it may do so sooner.
The LZWDecode and FlateDecode filters shall accept optional parameters to control the decoding process.
- NOTE Most of these parameters are related to techniques that reduce the size of compressed sampled images (rectangular arrays of colour values, described in 8.9, "Images"). For example, image data typically changes very little from sample to sample. Therefore, subtracting the values of adjacent samples (a process called differencing), and encoding the differences rather than the raw sample values, can reduce the size of the output data. Furthermore, when the image data contains several colour components (red-green-blue or cyanmagenta-yellow-black) per sample, taking the difference between the values of corresponding components in adjacent samples, rather than between different colour components in the same sample, often reduces the output data size.
Table 8 shows the parameters that may optionally be specified for LZWDecode and FlateDecode filters. Except where otherwise noted, all values supplied to the decoding filter for any optional parameters shall match those used when the data was encoded.
Table 8 - Optional parameters for LZWDecode and FlateDecode filters
| Key | Type | Value |
|---|---|---|
| Predictor | integer | A code that selects the predictor algorithm, if any. If the value of this entry is 1, the filter shall assume that the normal algorithm was used to encode the data, without prediction. If the value is greater than 1, the filter shall assume that the data was differenced before being encoded, and Predictor selects the predictor algorithm. For more information regarding Predictor values greater than 1, see 7.4.4.4, "LZW and Flate Predictor Functions." Default value: 1. |
| Colors | integer | (May be used only if Predictor is greater than 1) The number of interleaved colour components per sample. Valid values are 1 to 4 ( PDF 1.0 ) and 1 or greater ( PDF 1.3 ). Default value: 1. |
Table 8 - Optional parameters for LZWDecode and FlateDecode filters (continued)
| Key | Type | Value |
|---|---|---|
| BitsPerComponent | integer | (May be used only if Predictor is greater than 1) The number of bits used to represent each colour component in a sample. Valid values are 1, 2, 4, 8, and ( PDF 1.5 ) 16. Default value: 8. |
| Columns | integer | (May be used only if Predictor is greater than 1) The number of samples in each row. Default value: 1. |
| EarlyChange | integer | ( LZWDecode only) An indication of when to increase the code length. If the value of this entry is 0, code length increases shall be postponed as long as possible. If the value is 1, code length increases shall occur one code early. This parameter is included because LZW sample code distributed by some vendors increases the code length one code earlier than necessary. Default value: 1. |
LZW and Flate encoding compress more compactly if their input data is highly predictable. One way of increasing the predictability of many continuous-tone sampled images is to replace each sample with the difference between that sample and a predictor function applied to earlier neighboring samples. If the predictor function works well, the postprediction data clusters toward 0.
PDF supports two groups of predictor functions. The first, the TIFF group, consists of the single function that is Predictor 2 in the TIFF 6.0 specification.
NOTE 1 (In the TIFF 6.0 specification, Predictor 2 applies only to LZW compression, but here it applies to Flate compression as well.) TIFF Predictor 2 predicts that each colour component of a sample is the same as the corresponding colour component of the sample immediately to its left.
The second supported group of predictor functions, the PNG group, consists of the filters of the World Wide Web Consortium's Portable Network Graphics recommendation, documented in Internet RFC 2083, PNG (Portable Network Graphics) Specification (see the Bibliography).
The term predictors is used here instead of filters to avoid confusion.
There are five basic PNG predictor algorithms (and a sixth that chooses the optimum predictor function separately for each row).
Table 9 - PNG predictor algorithms
| PNG Predictor Algorithms | Description |
|---|---|
| None | No prediction |
| Sub | Predicts the same as the sample to the left |
| Up | Predicts the same as the sample above |
| Average | Predicts the average of the sample to the left and the sample above |
| Paeth | A nonlinear function of the sample above, the sample to the left, and the sample to the upper left |
The predictor algorithm to be used, if any, shall be indicated by the Predictor filter parameter (see Table 8), whose value shall be one of those listed in Table 10.
For LZWDecode and FlateDecode , a Predictor value greater than or equal to 10 shall indicate that a PNG predictor is in use; the specific predictor function used shall be explicitly encoded in the incoming data. The value of Predictor supplied by the decoding filter need not match the value used when the data was encoded if they are both greater than or equal to 10.
Table 10 - Predictor values
| Value | Meaning |
|---|---|
| 1 | No prediction (the default value) |
| 2 | TIFF Predictor 2 |
| 10 | PNG prediction (on encoding, PNG None on all rows) |
| 11 | PNG prediction (on encoding, PNG Sub on all rows) |
| 12 | PNG prediction (on encoding, PNG Up on all rows) |
| 13 | PNG prediction (on encoding, PNG Average on all rows) |
| 14 | PNG prediction (on encoding, PNG Paeth on all rows) |
| 15 | PNG prediction (on encoding, PNG optimum) |
The two groups of predictor functions have some commonalities. Both make the following assumptions:
- Data shall be presented in order, from the top row to the bottom row and, within a row, from left to right.
- A row shall occupy a whole number of bytes, rounded up if necessary.
- Samples and their components shall be packed into bytes from high-order to low-order bits.
- All colour components of samples outside the image (which are necessary for predictions near the boundaries) shall be 0.
The predictor function groups also differ in significant ways:
Navigation: PDF-Spec-07-Objects | PDF-Spec-Index | PDF-Spec-07-File-Structure