114 changes: 90 additions & 24 deletions doc/lz4_manual.html
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>1.9.1 Manual</title>
<title>1.9.2 Manual</title>
</head>
<body>
<h1>1.9.1 Manual</h1>
<h1>1.9.2 Manual</h1>
<hr>
<a name="Contents"></a><h2>Contents</h2>
<ol>
Expand All @@ -21,7 +21,7 @@ <h1>1.9.1 Manual</h1>
</ol>
<hr>
<a name="Chapter1"></a><h2>Introduction</h2><pre>
LZ4 is lossless compression algorithm, providing compression speed at 500 MB/s per core,
LZ4 is lossless compression algorithm, providing compression speed >500 MB/s per core,
scalable with multi-cores CPU. It features an extremely fast decoder, with speed in
multiple GB/s per core, typically reaching RAM speed limits on multi-core systems.

Expand All @@ -33,16 +33,19 @@ <h1>1.9.1 Manual</h1>
- unbounded multiple steps (described as Streaming compression)

lz4.h generates and decodes LZ4-compressed blocks (doc/lz4_Block_format.md).
Decompressing a block requires additional metadata, such as its compressed size.
Decompressing such a compressed block requires additional metadata.
Exact metadata depends on exact decompression function.
For the typical case of LZ4_decompress_safe(),
metadata includes block's compressed size, and maximum bound of decompressed size.
Each application is free to encode and pass such metadata in whichever way it wants.

lz4.h only handle blocks, it can not generate Frames.

Blocks are different from Frames (doc/lz4_Frame_format.md).
Frames bundle both blocks and metadata in a specified manner.
This are required for compressed data to be self-contained and portable.
Embedding metadata is required for compressed data to be self-contained and portable.
Frame format is delivered through a companion API, declared in lz4frame.h.
Note that the `lz4` CLI can only manage frames.
The `lz4` CLI can only manage frames.
<BR></pre>

<a name="Chapter2"></a><h2>Version</h2><pre></pre>
Expand All @@ -66,27 +69,35 @@ <h1>1.9.1 Manual</h1>
<a name="Chapter4"></a><h2>Simple Functions</h2><pre></pre>

<pre><b>int LZ4_compress_default(const char* src, char* dst, int srcSize, int dstCapacity);
</b><p> Compresses 'srcSize' bytes from buffer 'src'
into already allocated 'dst' buffer of size 'dstCapacity'.
Compression is guaranteed to succeed if 'dstCapacity' >= LZ4_compressBound(srcSize).
It also runs faster, so it's a recommended setting.
If the function cannot compress 'src' into a more limited 'dst' budget,
compression stops *immediately*, and the function result is zero.
In which case, 'dst' content is undefined (invalid).
srcSize : max supported value is LZ4_MAX_INPUT_SIZE.
dstCapacity : size of buffer 'dst' (which must be already allocated)
@return : the number of bytes written into buffer 'dst' (necessarily <= dstCapacity)
or 0 if compression fails
Note : This function is protected against buffer overflow scenarios (never writes outside 'dst' buffer, nor read outside 'source' buffer).
</b><p> Compresses 'srcSize' bytes from buffer 'src'
into already allocated 'dst' buffer of size 'dstCapacity'.
Compression is guaranteed to succeed if 'dstCapacity' >= LZ4_compressBound(srcSize).
It also runs faster, so it's a recommended setting.
If the function cannot compress 'src' into a more limited 'dst' budget,
compression stops *immediately*, and the function result is zero.
In which case, 'dst' content is undefined (invalid).
srcSize : max supported value is LZ4_MAX_INPUT_SIZE.
dstCapacity : size of buffer 'dst' (which must be already allocated)
@return : the number of bytes written into buffer 'dst' (necessarily <= dstCapacity)
or 0 if compression fails
Note : This function is protected against buffer overflow scenarios (never writes outside 'dst' buffer, nor read outside 'source' buffer).

</p></pre><BR>

<pre><b>int LZ4_decompress_safe (const char* src, char* dst, int compressedSize, int dstCapacity);
</b><p> compressedSize : is the exact complete size of the compressed block.
dstCapacity : is the size of destination buffer, which must be already allocated.
@return : the number of bytes decompressed into destination buffer (necessarily <= dstCapacity)
If destination buffer is not large enough, decoding will stop and output an error code (negative value).
If the source stream is detected malformed, the function will stop decoding and return a negative result.
Note : This function is protected against malicious data packets (never writes outside 'dst' buffer, nor read outside 'source' buffer).
</b><p> compressedSize : is the exact complete size of the compressed block.
dstCapacity : is the size of destination buffer (which must be already allocated), presumed an upper bound of decompressed size.
@return : the number of bytes decompressed into destination buffer (necessarily <= dstCapacity)
If destination buffer is not large enough, decoding will stop and output an error code (negative value).
If the source stream is detected malformed, the function will stop decoding and return a negative result.
Note 1 : This function is protected against malicious data packets :
it will never writes outside 'dst' buffer, nor read outside 'source' buffer,
even if the compressed block is maliciously modified to order the decoder to do these actions.
In such case, the decoder stops immediately, and considers the compressed block malformed.
Note 2 : compressedSize and dstCapacity must be provided to the function, the compressed block does not contain them.
The implementation is free to send / store / derive this information in whichever way is most beneficial.
If there is a need for a different format which bundles together both compressed data and its metadata, consider looking at lz4frame.h instead.

</p></pre><BR>

<a name="Chapter5"></a><h2>Advanced Functions</h2><pre></pre>
Expand Down Expand Up @@ -357,6 +368,61 @@ <h1>1.9.1 Manual</h1>

</p></pre><BR>

<pre><b></b><p>
It's possible to have input and output sharing the same buffer,
for highly contrained memory environments.
In both cases, it requires input to lay at the end of the buffer,
and decompression to start at beginning of the buffer.
Buffer size must feature some margin, hence be larger than final size.

|<------------------------buffer--------------------------------->|
|<-----------compressed data--------->|
|<-----------decompressed size------------------>|
|<----margin---->|

This technique is more useful for decompression,
since decompressed size is typically larger,
and margin is short.

In-place decompression will work inside any buffer
which size is >= LZ4_DECOMPRESS_INPLACE_BUFFER_SIZE(decompressedSize).
This presumes that decompressedSize > compressedSize.
Otherwise, it means compression actually expanded data,
and it would be more efficient to store such data with a flag indicating it's not compressed.
This can happen when data is not compressible (already compressed, or encrypted).

For in-place compression, margin is larger, as it must be able to cope with both
history preservation, requiring input data to remain unmodified up to LZ4_DISTANCE_MAX,
and data expansion, which can happen when input is not compressible.
As a consequence, buffer size requirements are much higher,
and memory savings offered by in-place compression are more limited.

There are ways to limit this cost for compression :
- Reduce history size, by modifying LZ4_DISTANCE_MAX.
Note that it is a compile-time constant, so all compressions will apply this limit.
Lower values will reduce compression ratio, except when input_size < LZ4_DISTANCE_MAX,
so it's a reasonable trick when inputs are known to be small.
- Require the compressor to deliver a "maximum compressed size".
This is the `dstCapacity` parameter in `LZ4_compress*()`.
When this size is < LZ4_COMPRESSBOUND(inputSize), then compression can fail,
in which case, the return code will be 0 (zero).
The caller must be ready for these cases to happen,
and typically design a backup scheme to send data uncompressed.
The combination of both techniques can significantly reduce
the amount of margin required for in-place compression.

In-place compression can work in any buffer
which size is >= (maxCompressedSize)
with maxCompressedSize == LZ4_COMPRESSBOUND(srcSize) for guaranteed compression success.
LZ4_COMPRESS_INPLACE_BUFFER_SIZE() depends on both maxCompressedSize and LZ4_DISTANCE_MAX,
so it's possible to reduce memory requirements by playing with them.

</p></pre><BR>

<pre><b>#define LZ4_DECOMPRESS_INPLACE_BUFFER_SIZE(decompressedSize) ((decompressedSize) + LZ4_DECOMPRESS_INPLACE_MARGIN(decompressedSize)) </b>/**< note: presumes that compressedSize < decompressedSize. note2: margin is overestimated a bit, since it could use compressedSize instead */<b>
</b></pre><BR>
<pre><b>#define LZ4_COMPRESS_INPLACE_BUFFER_SIZE(maxCompressedSize) ((maxCompressedSize) + LZ4_COMPRESS_INPLACE_MARGIN) </b>/**< maxCompressedSize is generally LZ4_COMPRESSBOUND(inputSize), but can be set to any lower value, with the risk that compression can fail (return code 0(zero)) */<b>
</b></pre><BR>
<a name="Chapter9"></a><h2>PRIVATE DEFINITIONS</h2><pre>
Do not use these definitions directly.
They are only exposed to allow static allocation of `LZ4_stream_t` and `LZ4_streamDecode_t`.
Expand Down
4 changes: 2 additions & 2 deletions doc/lz4frame_manual.html
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>1.9.1 Manual</title>
<title>1.9.2 Manual</title>
</head>
<body>
<h1>1.9.1 Manual</h1>
<h1>1.9.2 Manual</h1>
<hr>
<a name="Contents"></a><h2>Contents</h2>
<ol>
Expand Down
9 changes: 6 additions & 3 deletions examples/HCStreaming_ringBuffer.c
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <assert.h>

enum {
MESSAGE_MAX_BYTES = 1024,
Expand All @@ -39,15 +40,17 @@ size_t write_int32(FILE* fp, int32_t i) {
}

size_t write_bin(FILE* fp, const void* array, int arrayBytes) {
return fwrite(array, 1, arrayBytes, fp);
assert(arrayBytes >= 0);
return fwrite(array, 1, (size_t)arrayBytes, fp);
}

size_t read_int32(FILE* fp, int32_t* i) {
return fread(i, sizeof(*i), 1, fp);
}

size_t read_bin(FILE* fp, void* array, int arrayBytes) {
return fread(array, 1, arrayBytes, fp);
assert(arrayBytes >= 0);
return fread(array, 1, (size_t)arrayBytes, fp);
}


Expand Down Expand Up @@ -174,7 +177,7 @@ int main(int argc, const char** argv)
return 0;
}

if (!strcmp(argv[1], "-p")) pause = 1, fileID = 2;
if (!strcmp(argv[1], "-p")) { pause = 1; fileID = 2; }

snprintf(inpFilename, 256, "%s", argv[fileID]);
snprintf(lz4Filename, 256, "%s.lz4s-%d", argv[fileID], 9);
Expand Down
6 changes: 3 additions & 3 deletions examples/blockStreaming_doubleBuffer.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# LZ4 Streaming API Example : Double Buffer
by *Takayuki Matsuoka*

`blockStreaming_doubleBuffer.c` is LZ4 Straming API example which implements double buffer (de)compression.
`blockStreaming_doubleBuffer.c` is LZ4 Streaming API example which implements double buffer (de)compression.

Please note :

Expand Down Expand Up @@ -76,10 +76,10 @@ so it just compress the line without dependencies and generates compressed block
After that, write {Out#1} to the file.

Next, read second block to double buffer's second page. And compress it.
In this time, LZ4 can use dependency to Block#1 to improve compression ratio.
This time, LZ4 can use dependency to Block#1 to improve compression ratio.
This dependency is called "Prefix mode".

Next, read third block to double buffer's *first* page. And compress it.
Next, read third block to double buffer's *first* page, and compress it.
Also this time, LZ4 can use dependency to Block#2.
This dependency is called "External Dictonaly mode".

Expand Down
50 changes: 28 additions & 22 deletions examples/simple_buffer.c
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,18 @@
* Copyright : Kyle Harper
* License : Follows same licensing as the lz4.c/lz4.h program at any given time. Currently, BSD 2.
* Description: Example program to demonstrate the basic usage of the compress/decompress functions within lz4.c/lz4.h.
* The functions you'll likely want are LZ4_compress_default and LZ4_decompress_safe. Both of these are documented in
* the lz4.h header file; I recommend reading them.
* The functions you'll likely want are LZ4_compress_default and LZ4_decompress_safe.
* Both of these are documented in the lz4.h header file; I recommend reading them.
*/

/* Includes, for Power! */
#include "lz4.h" // This is all that is required to expose the prototypes for basic compression and decompression.
/* Dependencies */
#include <stdio.h> // For printf()
#include <string.h> // For memcmp()
#include <stdlib.h> // For exit()
#include "lz4.h" // This is all that is required to expose the prototypes for basic compression and decompression.

/*
* Easy show-error-and-bail function.
* Simple show-error-and-bail function.
*/
void run_screaming(const char* message, const int code) {
printf("%s \n", message);
Expand All @@ -32,41 +32,47 @@ int main(void) {
// 1) The return codes of LZ4_ functions are important.
// Read lz4.h if you're unsure what a given code means.
// 2) LZ4 uses char* pointers in all LZ4_ functions.
// This is baked into the API and probably not going to change.
// If your program uses pointers that are unsigned char*, void*, or otherwise different,
// you may need to do some casting or set the right -W compiler flags to ignore those warnings (e.g.: -Wno-pointer-sign).
// This is baked into the API and not going to change, for consistency.
// If your program uses different pointer types,
// you may need to do some casting or set the right -Wno compiler flags to ignore those warnings (e.g.: -Wno-pointer-sign).

/* Compression */
// We'll store some text into a variable pointed to by *src to be compressed later.
const char* const src = "Lorem ipsum dolor sit amet, consectetur adipiscing elit.";
const char* const src = "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Lorem ipsum dolor site amat.";
// The compression function needs to know how many bytes exist. Since we're using a string, we can use strlen() + 1 (for \0).
const int src_size = (int)(strlen(src) + 1);
// LZ4 provides a function that will tell you the maximum size of compressed output based on input data via LZ4_compressBound().
const int max_dst_size = LZ4_compressBound(src_size);
// We will use that size for our destination boundary when allocating space.
char* compressed_data = malloc(max_dst_size);
char* compressed_data = malloc((size_t)max_dst_size);
if (compressed_data == NULL)
run_screaming("Failed to allocate memory for *compressed_data.", 1);
// That's all the information and preparation LZ4 needs to compress *src into *compressed_data.
// Invoke LZ4_compress_default now with our size values and pointers to our memory locations.
// Save the return value for error checking.
const int compressed_data_size = LZ4_compress_default(src, compressed_data, src_size, max_dst_size);
// Check return_value to determine what happened.
if (compressed_data_size < 0)
run_screaming("A negative result from LZ4_compress_default indicates a failure trying to compress the data. See exit code (echo $?) for value returned.", compressed_data_size);
if (compressed_data_size == 0)
run_screaming("A result of 0 means compression worked, but was stopped because the destination buffer couldn't hold all the information.", 1);
if (compressed_data_size <= 0)
run_screaming("A 0 or negative result from LZ4_compress_default() indicates a failure trying to compress the data. ", 1);
if (compressed_data_size > 0)
printf("We successfully compressed some data!\n");
printf("We successfully compressed some data! Ratio: %.2f\n",
(float) compressed_data_size/src_size);
// Not only does a positive return_value mean success, the value returned == the number of bytes required.
// You can use this to realloc() *compress_data to free up memory, if desired. We'll do so just to demonstrate the concept.
compressed_data = (char *)realloc(compressed_data, compressed_data_size);
compressed_data = (char *)realloc(compressed_data, (size_t)compressed_data_size);
if (compressed_data == NULL)
run_screaming("Failed to re-alloc memory for compressed_data. Sad :(", 1);


/* Decompression */
// Now that we've successfully compressed the information from *src to *compressed_data, let's do the opposite! We'll create a
// *new_src location of size src_size since we know that value.
// Now that we've successfully compressed the information from *src to *compressed_data, let's do the opposite!
// The decompression will need to know the compressed size, and an upper bound of the decompressed size.
// In this example, we just re-use this information from previous section,
// but in a real-world scenario, metadata must be transmitted to the decompression side.
// Each implementation is in charge of this part. Oftentimes, it adds some header of its own.
// Sometimes, the metadata can be extracted from the local context.

// First, let's create a *new_src location of size src_size since we know that value.
char* const regen_buffer = malloc(src_size);
if (regen_buffer == NULL)
run_screaming("Failed to allocate memory for *regen_buffer.", 1);
Expand All @@ -77,17 +83,17 @@ int main(void) {
free(compressed_data); /* no longer useful */
if (decompressed_size < 0)
run_screaming("A negative result from LZ4_decompress_safe indicates a failure trying to decompress the data. See exit code (echo $?) for value returned.", decompressed_size);
if (decompressed_size == 0)
run_screaming("I'm not sure this function can ever return 0. Documentation in lz4.h doesn't indicate so.", 1);
if (decompressed_size > 0)
if (decompressed_size >= 0)
printf("We successfully decompressed some data!\n");
// Not only does a positive return value mean success,
// value returned == number of bytes regenerated from compressed_data stream.
if (decompressed_size != src_size)
run_screaming("Decompressed data is different from original! \n", 1);

/* Validation */
// We should be able to compare our original *src with our *new_src and be byte-for-byte identical.
if (memcmp(src, regen_buffer, src_size) != 0)
run_screaming("Validation failed. *src and *new_src are not identical.", 1);
printf("Validation done. The string we ended up with is:\n%s\n", regen_buffer);
printf("Validation done. The string we ended up with is:\n%s\n", regen_buffer);
return 0;
}
5 changes: 2 additions & 3 deletions lib/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,11 @@ FLAGS = $(CPPFLAGS) $(CFLAGS) $(LDFLAGS)

SRCFILES := $(sort $(wildcard *.c))

include ../Makefile.inc

# OS X linker doesn't support -soname, and use different extension
# see : https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/DynamicLibraryDesignGuidelines.html
ifeq ($(OS), Darwin)
ifeq ($(TARGET_OS), Darwin)
SHARED_EXT = dylib
SHARED_EXT_MAJOR = $(LIBVER_MAJOR).$(SHARED_EXT)
SHARED_EXT_VER = $(LIBVER).$(SHARED_EXT)
Expand All @@ -70,8 +71,6 @@ else
SHARED_EXT_VER = $(SHARED_EXT).$(LIBVER)
endif

include ../Makefile.inc

.PHONY: default
default: lib-release

Expand Down
8 changes: 3 additions & 5 deletions lib/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,8 @@ The following build macro can be selected at compilation time :
- `LZ4_DISTANCE_MAX` : control the maximum offset that the compressor will allow.
Set to 65535 by default, which is the maximum value supported by lz4 format.
Reducing maximum distance will reduce opportunities for LZ4 to find matches,
hence will produce worse the compression ratio.
However, a smaller max distance may allow compatibility with specific decoders using limited memory budget.
hence will produce a worse compression ratio.
However, a smaller max distance can allow compatibility with specific decoders using limited memory budget.
This build macro only influences the compressed output of the compressor.

- `LZ4_DISABLE_DEPRECATE_WARNINGS` : invoking a deprecated function will make the compiler generate a warning.
Expand All @@ -74,9 +74,7 @@ The following build macro can be selected at compilation time :
lz4 source code can be amalgamated into a single file.
One can combine all source code into `lz4_all.c` by using following command:
```
cat lz4.c > lz4_all.c
cat lz4hc.c >> lz4_all.c
cat lz4frame.c >> lz4_all.c
cat lz4.c lz4hc.c lz4frame.c > lz4_all.c
```
(`cat` file order is important) then compile `lz4_all.c`.
All `*.h` files present in `/lib` remain necessary to compile `lz4_all.c`.
Expand Down
425 changes: 262 additions & 163 deletions lib/lz4.c

Large diffs are not rendered by default.

142 changes: 112 additions & 30 deletions lib/lz4.h
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ extern "C" {
/**
Introduction
LZ4 is lossless compression algorithm, providing compression speed at 500 MB/s per core,
LZ4 is lossless compression algorithm, providing compression speed >500 MB/s per core,
scalable with multi-cores CPU. It features an extremely fast decoder, with speed in
multiple GB/s per core, typically reaching RAM speed limits on multi-core systems.
Expand All @@ -58,16 +58,19 @@ extern "C" {
- unbounded multiple steps (described as Streaming compression)
lz4.h generates and decodes LZ4-compressed blocks (doc/lz4_Block_format.md).
Decompressing a block requires additional metadata, such as its compressed size.
Decompressing such a compressed block requires additional metadata.
Exact metadata depends on exact decompression function.
For the typical case of LZ4_decompress_safe(),
metadata includes block's compressed size, and maximum bound of decompressed size.
Each application is free to encode and pass such metadata in whichever way it wants.
lz4.h only handle blocks, it can not generate Frames.
Blocks are different from Frames (doc/lz4_Frame_format.md).
Frames bundle both blocks and metadata in a specified manner.
This are required for compressed data to be self-contained and portable.
Embedding metadata is required for compressed data to be self-contained and portable.
Frame format is delivered through a companion API, declared in lz4frame.h.
Note that the `lz4` CLI can only manage frames.
The `lz4` CLI can only manage frames.
*/

/*^***************************************************************
Expand Down Expand Up @@ -97,7 +100,7 @@ extern "C" {
/*------ Version ------*/
#define LZ4_VERSION_MAJOR 1 /* for breaking interface changes */
#define LZ4_VERSION_MINOR 9 /* for new (non-breaking) interface capabilities */
#define LZ4_VERSION_RELEASE 1 /* for tweaks, bug-fixes, or development */
#define LZ4_VERSION_RELEASE 2 /* for tweaks, bug-fixes, or development */

#define LZ4_VERSION_NUMBER (LZ4_VERSION_MAJOR *100*100 + LZ4_VERSION_MINOR *100 + LZ4_VERSION_RELEASE)

Expand Down Expand Up @@ -129,29 +132,35 @@ LZ4LIB_API const char* LZ4_versionString (void); /**< library version string;
* Simple Functions
**************************************/
/*! LZ4_compress_default() :
Compresses 'srcSize' bytes from buffer 'src'
into already allocated 'dst' buffer of size 'dstCapacity'.
Compression is guaranteed to succeed if 'dstCapacity' >= LZ4_compressBound(srcSize).
It also runs faster, so it's a recommended setting.
If the function cannot compress 'src' into a more limited 'dst' budget,
compression stops *immediately*, and the function result is zero.
In which case, 'dst' content is undefined (invalid).
srcSize : max supported value is LZ4_MAX_INPUT_SIZE.
dstCapacity : size of buffer 'dst' (which must be already allocated)
@return : the number of bytes written into buffer 'dst' (necessarily <= dstCapacity)
or 0 if compression fails
Note : This function is protected against buffer overflow scenarios (never writes outside 'dst' buffer, nor read outside 'source' buffer).
*/
* Compresses 'srcSize' bytes from buffer 'src'
* into already allocated 'dst' buffer of size 'dstCapacity'.
* Compression is guaranteed to succeed if 'dstCapacity' >= LZ4_compressBound(srcSize).
* It also runs faster, so it's a recommended setting.
* If the function cannot compress 'src' into a more limited 'dst' budget,
* compression stops *immediately*, and the function result is zero.
* In which case, 'dst' content is undefined (invalid).
* srcSize : max supported value is LZ4_MAX_INPUT_SIZE.
* dstCapacity : size of buffer 'dst' (which must be already allocated)
* @return : the number of bytes written into buffer 'dst' (necessarily <= dstCapacity)
* or 0 if compression fails
* Note : This function is protected against buffer overflow scenarios (never writes outside 'dst' buffer, nor read outside 'source' buffer).
*/
LZ4LIB_API int LZ4_compress_default(const char* src, char* dst, int srcSize, int dstCapacity);

/*! LZ4_decompress_safe() :
compressedSize : is the exact complete size of the compressed block.
dstCapacity : is the size of destination buffer, which must be already allocated.
@return : the number of bytes decompressed into destination buffer (necessarily <= dstCapacity)
If destination buffer is not large enough, decoding will stop and output an error code (negative value).
If the source stream is detected malformed, the function will stop decoding and return a negative result.
Note : This function is protected against malicious data packets (never writes outside 'dst' buffer, nor read outside 'source' buffer).
*/
* compressedSize : is the exact complete size of the compressed block.
* dstCapacity : is the size of destination buffer (which must be already allocated), presumed an upper bound of decompressed size.
* @return : the number of bytes decompressed into destination buffer (necessarily <= dstCapacity)
* If destination buffer is not large enough, decoding will stop and output an error code (negative value).
* If the source stream is detected malformed, the function will stop decoding and return a negative result.
* Note 1 : This function is protected against malicious data packets :
* it will never writes outside 'dst' buffer, nor read outside 'source' buffer,
* even if the compressed block is maliciously modified to order the decoder to do these actions.
* In such case, the decoder stops immediately, and considers the compressed block malformed.
* Note 2 : compressedSize and dstCapacity must be provided to the function, the compressed block does not contain them.
* The implementation is free to send / store / derive this information in whichever way is most beneficial.
* If there is a need for a different format which bundles together both compressed data and its metadata, consider looking at lz4frame.h instead.
*/
LZ4LIB_API int LZ4_decompress_safe (const char* src, char* dst, int compressedSize, int dstCapacity);


Expand Down Expand Up @@ -388,6 +397,8 @@ LZ4LIB_API int LZ4_decompress_safe_continue (LZ4_streamDecode_t* LZ4_streamDecod
*/
LZ4LIB_API int LZ4_decompress_safe_usingDict (const char* src, char* dst, int srcSize, int dstCapcity, const char* dictStart, int dictSize);

#endif /* LZ4_H_2983827168210 */


/*^*************************************
* !!!!!! STATIC LINKING ONLY !!!!!!
Expand All @@ -413,14 +424,17 @@ LZ4LIB_API int LZ4_decompress_safe_usingDict (const char* src, char* dst, int sr
* define LZ4_PUBLISH_STATIC_FUNCTIONS when building the LZ4 library.
******************************************************************************/

#ifdef LZ4_STATIC_LINKING_ONLY

#ifndef LZ4_STATIC_3504398509
#define LZ4_STATIC_3504398509

#ifdef LZ4_PUBLISH_STATIC_FUNCTIONS
#define LZ4LIB_STATIC_API LZ4LIB_API
#else
#define LZ4LIB_STATIC_API
#endif

#ifdef LZ4_STATIC_LINKING_ONLY


/*! LZ4_compress_fast_extState_fastReset() :
* A variant of LZ4_compress_fast_extState().
Expand Down Expand Up @@ -462,8 +476,75 @@ LZ4LIB_STATIC_API int LZ4_compress_fast_extState_fastReset (void* state, const c
*/
LZ4LIB_STATIC_API void LZ4_attach_dictionary(LZ4_stream_t* workingStream, const LZ4_stream_t* dictionaryStream);


/*! In-place compression and decompression
*
* It's possible to have input and output sharing the same buffer,
* for highly contrained memory environments.
* In both cases, it requires input to lay at the end of the buffer,
* and decompression to start at beginning of the buffer.
* Buffer size must feature some margin, hence be larger than final size.
*
* |<------------------------buffer--------------------------------->|
* |<-----------compressed data--------->|
* |<-----------decompressed size------------------>|
* |<----margin---->|
*
* This technique is more useful for decompression,
* since decompressed size is typically larger,
* and margin is short.
*
* In-place decompression will work inside any buffer
* which size is >= LZ4_DECOMPRESS_INPLACE_BUFFER_SIZE(decompressedSize).
* This presumes that decompressedSize > compressedSize.
* Otherwise, it means compression actually expanded data,
* and it would be more efficient to store such data with a flag indicating it's not compressed.
* This can happen when data is not compressible (already compressed, or encrypted).
*
* For in-place compression, margin is larger, as it must be able to cope with both
* history preservation, requiring input data to remain unmodified up to LZ4_DISTANCE_MAX,
* and data expansion, which can happen when input is not compressible.
* As a consequence, buffer size requirements are much higher,
* and memory savings offered by in-place compression are more limited.
*
* There are ways to limit this cost for compression :
* - Reduce history size, by modifying LZ4_DISTANCE_MAX.
* Note that it is a compile-time constant, so all compressions will apply this limit.
* Lower values will reduce compression ratio, except when input_size < LZ4_DISTANCE_MAX,
* so it's a reasonable trick when inputs are known to be small.
* - Require the compressor to deliver a "maximum compressed size".
* This is the `dstCapacity` parameter in `LZ4_compress*()`.
* When this size is < LZ4_COMPRESSBOUND(inputSize), then compression can fail,
* in which case, the return code will be 0 (zero).
* The caller must be ready for these cases to happen,
* and typically design a backup scheme to send data uncompressed.
* The combination of both techniques can significantly reduce
* the amount of margin required for in-place compression.
*
* In-place compression can work in any buffer
* which size is >= (maxCompressedSize)
* with maxCompressedSize == LZ4_COMPRESSBOUND(srcSize) for guaranteed compression success.
* LZ4_COMPRESS_INPLACE_BUFFER_SIZE() depends on both maxCompressedSize and LZ4_DISTANCE_MAX,
* so it's possible to reduce memory requirements by playing with them.
*/

#define LZ4_DECOMPRESS_INPLACE_MARGIN(compressedSize) (((compressedSize) >> 8) + 32)
#define LZ4_DECOMPRESS_INPLACE_BUFFER_SIZE(decompressedSize) ((decompressedSize) + LZ4_DECOMPRESS_INPLACE_MARGIN(decompressedSize)) /**< note: presumes that compressedSize < decompressedSize. note2: margin is overestimated a bit, since it could use compressedSize instead */

#ifndef LZ4_DISTANCE_MAX /* history window size; can be user-defined at compile time */
# define LZ4_DISTANCE_MAX 65535 /* set to maximum value by default */
#endif

#define LZ4_COMPRESS_INPLACE_MARGIN (LZ4_DISTANCE_MAX + 32) /* LZ4_DISTANCE_MAX can be safely replaced by srcSize when it's smaller */
#define LZ4_COMPRESS_INPLACE_BUFFER_SIZE(maxCompressedSize) ((maxCompressedSize) + LZ4_COMPRESS_INPLACE_MARGIN) /**< maxCompressedSize is generally LZ4_COMPRESSBOUND(inputSize), but can be set to any lower value, with the risk that compression can fail (return code 0(zero)) */

#endif /* LZ4_STATIC_3504398509 */
#endif /* LZ4_STATIC_LINKING_ONLY */



#ifndef LZ4_H_98237428734687
#define LZ4_H_98237428734687

/*-************************************************************
* PRIVATE DEFINITIONS
Expand Down Expand Up @@ -567,6 +648,7 @@ union LZ4_streamDecode_u {
} ; /* previously typedef'd to LZ4_streamDecode_t */



/*-************************************
* Obsolete Functions
**************************************/
Expand Down Expand Up @@ -601,8 +683,8 @@ union LZ4_streamDecode_u {
#endif /* LZ4_DISABLE_DEPRECATE_WARNINGS */

/* Obsolete compression functions */
LZ4_DEPRECATED("use LZ4_compress_default() instead") LZ4LIB_API int LZ4_compress (const char* source, char* dest, int sourceSize);
LZ4_DEPRECATED("use LZ4_compress_default() instead") LZ4LIB_API int LZ4_compress_limitedOutput (const char* source, char* dest, int sourceSize, int maxOutputSize);
LZ4_DEPRECATED("use LZ4_compress_default() instead") LZ4LIB_API int LZ4_compress (const char* src, char* dest, int srcSize);
LZ4_DEPRECATED("use LZ4_compress_default() instead") LZ4LIB_API int LZ4_compress_limitedOutput (const char* src, char* dest, int srcSize, int maxOutputSize);
LZ4_DEPRECATED("use LZ4_compress_fast_extState() instead") LZ4LIB_API int LZ4_compress_withState (void* state, const char* source, char* dest, int inputSize);
LZ4_DEPRECATED("use LZ4_compress_fast_extState() instead") LZ4LIB_API int LZ4_compress_limitedOutput_withState (void* state, const char* source, char* dest, int inputSize, int maxOutputSize);
LZ4_DEPRECATED("use LZ4_compress_fast_continue() instead") LZ4LIB_API int LZ4_compress_continue (LZ4_stream_t* LZ4_streamPtr, const char* source, char* dest, int inputSize);
Expand Down Expand Up @@ -674,7 +756,7 @@ LZ4LIB_API int LZ4_decompress_fast_usingDict (const char* src, char* dst, int or
LZ4LIB_API void LZ4_resetStream (LZ4_stream_t* streamPtr);


#endif /* LZ4_H_2983827168210 */
#endif /* LZ4_H_98237428734687 */


#if defined (__cplusplus)
Expand Down
28 changes: 25 additions & 3 deletions lib/lz4frame.c
Original file line number Diff line number Diff line change
Expand Up @@ -213,8 +213,8 @@ static void LZ4F_writeLE64 (void* dst, U64 value64)

static const size_t minFHSize = LZ4F_HEADER_SIZE_MIN; /* 7 */
static const size_t maxFHSize = LZ4F_HEADER_SIZE_MAX; /* 19 */
static const size_t BHSize = 4; /* block header : size, and compress flag */
static const size_t BFSize = 4; /* block footer : checksum (optional) */
static const size_t BHSize = LZ4F_BLOCK_HEADER_SIZE; /* block header : size, and compress flag */
static const size_t BFSize = LZ4F_BLOCK_CHECKSUM_SIZE; /* block footer : checksum (optional) */


/*-************************************
Expand Down Expand Up @@ -327,6 +327,7 @@ static size_t LZ4F_compressBound_internal(size_t srcSize,
{
LZ4F_preferences_t prefsNull = LZ4F_INIT_PREFERENCES;
prefsNull.frameInfo.contentChecksumFlag = LZ4F_contentChecksumEnabled; /* worst case */
prefsNull.frameInfo.blockChecksumFlag = LZ4F_blockChecksumEnabled; /* worst case */
{ const LZ4F_preferences_t* const prefsPtr = (preferencesPtr==NULL) ? &prefsNull : preferencesPtr;
U32 const flush = prefsPtr->autoFlush | (srcSize==0);
LZ4F_blockSizeID_t const blockID = prefsPtr->frameInfo.blockSizeID;
Expand Down Expand Up @@ -1130,8 +1131,10 @@ static size_t LZ4F_decodeHeader(LZ4F_dctx* dctx, const void* src, size_t srcSize
}

/* control magic number */
#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
if (LZ4F_readLE32(srcPtr) != LZ4F_MAGICNUMBER)
return err0r(LZ4F_ERROR_frameType_unknown);
#endif
dctx->frameInfo.frameType = LZ4F_frame;

/* Flags */
Expand Down Expand Up @@ -1170,10 +1173,12 @@ static size_t LZ4F_decodeHeader(LZ4F_dctx* dctx, const void* src, size_t srcSize

/* check header */
assert(frameHeaderSize > 5);
#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
{ BYTE const HC = LZ4F_headerChecksum(srcPtr+4, frameHeaderSize-5);
if (HC != srcPtr[frameHeaderSize-1])
return err0r(LZ4F_ERROR_headerChecksum_invalid);
}
#endif

/* save */
dctx->frameInfo.blockMode = (LZ4F_blockMode_t)blockMode;
Expand Down Expand Up @@ -1210,8 +1215,10 @@ size_t LZ4F_headerSize(const void* src, size_t srcSize)
return 8;

/* control magic number */
#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
if (LZ4F_readLE32(src) != LZ4F_MAGICNUMBER)
return err0r(LZ4F_ERROR_frameType_unknown);
#endif

/* Frame Header Size */
{ BYTE const FLG = ((const BYTE*)src)[4];
Expand Down Expand Up @@ -1493,7 +1500,7 @@ size_t LZ4F_decompress(LZ4F_dctx* dctx,
/* next block is a compressed block */
dctx->tmpInTarget = nextCBlockSize + crcSize;
dctx->dStage = dstage_getCBlock;
if (dstPtr==dstEnd) {
if (dstPtr==dstEnd || srcPtr==srcEnd) {
nextSrcSizeHint = BHSize + nextCBlockSize + crcSize;
doAnotherStage = 0;
}
Expand Down Expand Up @@ -1554,8 +1561,13 @@ size_t LZ4F_decompress(LZ4F_dctx* dctx,
}
{ U32 const readCRC = LZ4F_readLE32(crcSrc);
U32 const calcCRC = XXH32_digest(&dctx->blockChecksum);
#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
if (readCRC != calcCRC)
return err0r(LZ4F_ERROR_blockChecksum_invalid);
#else
(void)readCRC;
(void)calcCRC;
#endif
} }
dctx->dStage = dstage_getBlockHeader; /* new block */
break;
Expand Down Expand Up @@ -1594,8 +1606,13 @@ size_t LZ4F_decompress(LZ4F_dctx* dctx,
assert(selectedIn != NULL); /* selectedIn is defined at this stage (either srcPtr, or dctx->tmpIn) */
{ U32 const readBlockCrc = LZ4F_readLE32(selectedIn + dctx->tmpInTarget);
U32 const calcBlockCrc = XXH32(selectedIn, dctx->tmpInTarget, 0);
#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
if (readBlockCrc != calcBlockCrc)
return err0r(LZ4F_ERROR_blockChecksum_invalid);
#else
(void)readBlockCrc;
(void)calcBlockCrc;
#endif
} }

if ((size_t)(dstEnd-dstPtr) >= dctx->maxBlockSize) {
Expand Down Expand Up @@ -1723,8 +1740,13 @@ size_t LZ4F_decompress(LZ4F_dctx* dctx,
/* case dstage_checkSuffix: */ /* no direct entry, avoid initialization risks */
{ U32 const readCRC = LZ4F_readLE32(selectedIn);
U32 const resultCRC = XXH32_digest(&(dctx->xxh));
#ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
if (readCRC != resultCRC)
return err0r(LZ4F_ERROR_contentChecksum_invalid);
#else
(void)readCRC;
(void)resultCRC;
#endif
nextSrcSizeHint = 0;
LZ4F_resetDecompressionContext(dctx);
doAnotherStage = 0;
Expand Down
9 changes: 9 additions & 0 deletions lib/lz4frame.h
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,15 @@ LZ4FLIB_API LZ4F_errorCode_t LZ4F_freeCompressionContext(LZ4F_cctx* cctx);
#define LZ4F_HEADER_SIZE_MIN 7 /* LZ4 Frame header size can vary, depending on selected paramaters */
#define LZ4F_HEADER_SIZE_MAX 19

/* Size in bytes of a block header in little-endian format. Highest bit indicates if block data is uncompressed */
#define LZ4F_BLOCK_HEADER_SIZE 4

/* Size in bytes of a block checksum footer in little-endian format. */
#define LZ4F_BLOCK_CHECKSUM_SIZE 4

/* Size in bytes of the content checksum. */
#define LZ4F_CONTENT_CHECKSUM_SIZE 4

/*! LZ4F_compressBegin() :
* will write the frame header into dstBuffer.
* dstCapacity must be >= LZ4F_HEADER_SIZE_MAX bytes.
Expand Down
120 changes: 91 additions & 29 deletions lib/lz4hc.c
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,21 @@ int LZ4HC_countBack(const BYTE* const ip, const BYTE* const match,
return back;
}

#if defined(_MSC_VER)
# define LZ4HC_rotl32(x,r) _rotl(x,r)
#else
# define LZ4HC_rotl32(x,r) ((x << r) | (x >> (32 - r)))
#endif


static U32 LZ4HC_rotatePattern(size_t const rotate, U32 const pattern)
{
size_t const bitsToRotate = (rotate & (sizeof(pattern) - 1)) << 3;
if (bitsToRotate == 0)
return pattern;
return LZ4HC_rotl32(pattern, (int)bitsToRotate);
}

/* LZ4HC_countPattern() :
* pattern32 must be a sample of repetitive pattern of length 1, 2 or 4 (but not 3!) */
static unsigned
Expand Down Expand Up @@ -203,6 +218,16 @@ LZ4HC_reverseCountPattern(const BYTE* ip, const BYTE* const iLow, U32 pattern)
return (unsigned)(iStart - ip);
}

/* LZ4HC_protectDictEnd() :
* Checks if the match is in the last 3 bytes of the dictionary, so reading the
* 4 byte MINMATCH would overflow.
* @returns true if the match index is okay.
*/
static int LZ4HC_protectDictEnd(U32 const dictLimit, U32 const matchIndex)
{
return ((U32)((dictLimit - 1) - matchIndex) >= 3);
}

typedef enum { rep_untested, rep_not, rep_confirmed } repeat_state_e;
typedef enum { favorCompressionRatio=0, favorDecompressionSpeed } HCfavor_e;

Expand All @@ -228,7 +253,7 @@ LZ4HC_InsertAndGetWiderMatch (
const U32 dictLimit = hc4->dictLimit;
const BYTE* const lowPrefixPtr = base + dictLimit;
const U32 ipIndex = (U32)(ip - base);
const U32 lowestMatchIndex = (hc4->lowLimit + 64 KB > ipIndex) ? hc4->lowLimit : ipIndex - LZ4_DISTANCE_MAX;
const U32 lowestMatchIndex = (hc4->lowLimit + (LZ4_DISTANCE_MAX + 1) > ipIndex) ? hc4->lowLimit : ipIndex - LZ4_DISTANCE_MAX;
const BYTE* const dictBase = hc4->dictBase;
int const lookBackLength = (int)(ip-iLowLimit);
int nbAttempts = maxNbAttempts;
Expand Down Expand Up @@ -287,14 +312,21 @@ LZ4HC_InsertAndGetWiderMatch (
if (chainSwap && matchLength==longest) { /* better match => select a better chain */
assert(lookBackLength==0); /* search forward only */
if (matchIndex + (U32)longest <= ipIndex) {
int const kTrigger = 4;
U32 distanceToNextMatch = 1;
int const end = longest - MINMATCH + 1;
int step = 1;
int accel = 1 << kTrigger;
int pos;
for (pos = 0; pos <= longest - MINMATCH; pos++) {
for (pos = 0; pos < end; pos += step) {
U32 const candidateDist = DELTANEXTU16(chainTable, matchIndex + (U32)pos);
step = (accel++ >> kTrigger);
if (candidateDist > distanceToNextMatch) {
distanceToNextMatch = candidateDist;
matchChainPos = (U32)pos;
} }
accel = 1 << kTrigger;
}
}
if (distanceToNextMatch > 1) {
if (distanceToNextMatch > matchIndex) break; /* avoid overflow */
matchIndex -= distanceToNextMatch;
Expand All @@ -313,34 +345,61 @@ LZ4HC_InsertAndGetWiderMatch (
} else {
repeat = rep_not;
} }
if ( (repeat == rep_confirmed)
&& (matchCandidateIdx >= dictLimit) ) { /* same segment only */
const BYTE* const matchPtr = base + matchCandidateIdx;
if ( (repeat == rep_confirmed) && (matchCandidateIdx >= lowestMatchIndex)
&& LZ4HC_protectDictEnd(dictLimit, matchCandidateIdx) ) {
const int extDict = matchCandidateIdx < dictLimit;
const BYTE* const matchPtr = (extDict ? dictBase : base) + matchCandidateIdx;
if (LZ4_read32(matchPtr) == pattern) { /* good candidate */
size_t const forwardPatternLength = LZ4HC_countPattern(matchPtr+sizeof(pattern), iHighLimit, pattern) + sizeof(pattern);
const BYTE* const lowestMatchPtr = (lowPrefixPtr + LZ4_DISTANCE_MAX >= ip) ? lowPrefixPtr : ip - LZ4_DISTANCE_MAX;
size_t const backLength = LZ4HC_reverseCountPattern(matchPtr, lowestMatchPtr, pattern);
size_t const currentSegmentLength = backLength + forwardPatternLength;

if ( (currentSegmentLength >= srcPatternLength) /* current pattern segment large enough to contain full srcPatternLength */
&& (forwardPatternLength <= srcPatternLength) ) { /* haven't reached this position yet */
matchIndex = matchCandidateIdx + (U32)forwardPatternLength - (U32)srcPatternLength; /* best position, full pattern, might be followed by more match */
} else {
matchIndex = matchCandidateIdx - (U32)backLength; /* farthest position in current segment, will find a match of length currentSegmentLength + maybe some back */
if (lookBackLength==0) { /* no back possible */
size_t const maxML = MIN(currentSegmentLength, srcPatternLength);
if ((size_t)longest < maxML) {
assert(base + matchIndex < ip);
if (ip - (base+matchIndex) > LZ4_DISTANCE_MAX) break;
assert(maxML < 2 GB);
longest = (int)maxML;
*matchpos = base + matchIndex; /* virtual pos, relative to ip, to retrieve offset */
*startpos = ip;
const BYTE* const dictStart = dictBase + hc4->lowLimit;
const BYTE* const iLimit = extDict ? dictBase + dictLimit : iHighLimit;
size_t forwardPatternLength = LZ4HC_countPattern(matchPtr+sizeof(pattern), iLimit, pattern) + sizeof(pattern);
if (extDict && matchPtr + forwardPatternLength == iLimit) {
U32 const rotatedPattern = LZ4HC_rotatePattern(forwardPatternLength, pattern);
forwardPatternLength += LZ4HC_countPattern(lowPrefixPtr, iHighLimit, rotatedPattern);
}
{ const BYTE* const lowestMatchPtr = extDict ? dictStart : lowPrefixPtr;
size_t backLength = LZ4HC_reverseCountPattern(matchPtr, lowestMatchPtr, pattern);
size_t currentSegmentLength;
if (!extDict && matchPtr - backLength == lowPrefixPtr && hc4->lowLimit < dictLimit) {
U32 const rotatedPattern = LZ4HC_rotatePattern((U32)(-(int)backLength), pattern);
backLength += LZ4HC_reverseCountPattern(dictBase + dictLimit, dictStart, rotatedPattern);
}
/* Limit backLength not go further than lowestMatchIndex */
backLength = matchCandidateIdx - MAX(matchCandidateIdx - (U32)backLength, lowestMatchIndex);
assert(matchCandidateIdx - backLength >= lowestMatchIndex);
currentSegmentLength = backLength + forwardPatternLength;
/* Adjust to end of pattern if the source pattern fits, otherwise the beginning of the pattern */
if ( (currentSegmentLength >= srcPatternLength) /* current pattern segment large enough to contain full srcPatternLength */
&& (forwardPatternLength <= srcPatternLength) ) { /* haven't reached this position yet */
U32 const newMatchIndex = matchCandidateIdx + (U32)forwardPatternLength - (U32)srcPatternLength; /* best position, full pattern, might be followed by more match */
if (LZ4HC_protectDictEnd(dictLimit, newMatchIndex))
matchIndex = newMatchIndex;
else {
/* Can only happen if started in the prefix */
assert(newMatchIndex >= dictLimit - 3 && newMatchIndex < dictLimit && !extDict);
matchIndex = dictLimit;
}
{ U32 const distToNextPattern = DELTANEXTU16(chainTable, matchIndex);
if (distToNextPattern > matchIndex) break; /* avoid overflow */
matchIndex -= distToNextPattern;
} } }
} else {
U32 const newMatchIndex = matchCandidateIdx - (U32)backLength; /* farthest position in current segment, will find a match of length currentSegmentLength + maybe some back */
if (!LZ4HC_protectDictEnd(dictLimit, newMatchIndex)) {
assert(newMatchIndex >= dictLimit - 3 && newMatchIndex < dictLimit && !extDict);
matchIndex = dictLimit;
} else {
matchIndex = newMatchIndex;
if (lookBackLength==0) { /* no back possible */
size_t const maxML = MIN(currentSegmentLength, srcPatternLength);
if ((size_t)longest < maxML) {
assert(base + matchIndex < ip);
if (ip - (base+matchIndex) > LZ4_DISTANCE_MAX) break;
assert(maxML < 2 GB);
longest = (int)maxML;
*matchpos = base + matchIndex; /* virtual pos, relative to ip, to retrieve offset */
*startpos = ip;
}
{ U32 const distToNextPattern = DELTANEXTU16(chainTable, matchIndex);
if (distToNextPattern > matchIndex) break; /* avoid overflow */
matchIndex -= distToNextPattern;
} } } } }
continue;
} }
} } /* PA optimization */
Expand Down Expand Up @@ -1005,6 +1064,9 @@ static void LZ4HC_setExternalDict(LZ4HC_CCtx_internal* ctxPtr, const BYTE* newBl
ctxPtr->base = newBlock - ctxPtr->dictLimit;
ctxPtr->end = newBlock;
ctxPtr->nextToUpdate = ctxPtr->dictLimit; /* match referencing will resume from there */

/* cannot reference an extDict and a dictCtx at the same time */
ctxPtr->dictCtx = NULL;
}

static int LZ4_compressHC_continue_generic (LZ4_streamHC_t* LZ4_streamHCPtr,
Expand Down
3 changes: 3 additions & 0 deletions lib/lz4hc.h
Original file line number Diff line number Diff line change
Expand Up @@ -336,6 +336,9 @@ LZ4LIB_API void LZ4_resetStreamHC (LZ4_streamHC_t* streamHCPtr, int compressionL
#ifndef LZ4_HC_SLO_098092834
#define LZ4_HC_SLO_098092834

#define LZ4_STATIC_LINKING_ONLY /* LZ4LIB_STATIC_API */
#include "lz4.h"

#if defined (__cplusplus)
extern "C" {
#endif
Expand Down
74 changes: 74 additions & 0 deletions ossfuzz/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# ##########################################################################
# LZ4 oss fuzzer - Makefile
#
# GPL v2 License
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
#
# You can contact the author at :
# - LZ4 homepage : http://www.lz4.org
# - LZ4 source repository : https://github.com/lz4/lz4
# ##########################################################################
# compress_fuzzer : OSS Fuzz test tool
# decompress_fuzzer : OSS Fuzz test tool
# ##########################################################################

LZ4DIR := ../lib
LIB_FUZZING_ENGINE ?= standaloneengine.o

DEBUGLEVEL?= 1
DEBUGFLAGS = -g -DLZ4_DEBUG=$(DEBUGLEVEL)

LZ4_CFLAGS = $(CFLAGS) $(DEBUGFLAGS) $(MOREFLAGS)
LZ4_CXXFLAGS = $(CXXFLAGS) $(DEBUGFLAGS) $(MOREFLAGS)
LZ4_CPPFLAGS = $(CPPFLAGS) -I$(LZ4DIR) -DXXH_NAMESPACE=LZ4_ \
-DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION

FUZZERS := \
compress_fuzzer \
decompress_fuzzer \
round_trip_fuzzer \
round_trip_stream_fuzzer \
compress_hc_fuzzer \
round_trip_hc_fuzzer \
compress_frame_fuzzer \
round_trip_frame_fuzzer \
decompress_frame_fuzzer

all: $(FUZZERS)

# Include a rule to build the static library if calling this target
# directly.
$(LZ4DIR)/liblz4.a:
$(MAKE) -C $(LZ4DIR) CFLAGS="$(LZ4_CFLAGS)" liblz4.a

%.o: %.c
$(CC) -c $(LZ4_CFLAGS) $(LZ4_CPPFLAGS) $< -o $@

# Generic rule for generating fuzzers
%_fuzzer: %_fuzzer.o lz4_helpers.o $(LZ4DIR)/liblz4.a
# Compile the standalone code just in case. The OSS-Fuzz code might
# override the LIB_FUZZING_ENGINE value to "-fsanitize=fuzzer"
$(CC) -c $(LZ4_CFLAGS) $(LZ4_CPPFLAGS) standaloneengine.c -o standaloneengine.o

# Now compile the actual fuzzer.
$(CXX) $(LZ4_CXXFLAGS) $(LZ4_CPPFLAGS) $(LDFLAGS) $(LIB_FUZZING_ENGINE) $^ -o $@$(EXT)

%_fuzzer_clean:
$(RM) $*_fuzzer $*_fuzzer.o standaloneengine.o

.PHONY: clean
clean: compress_fuzzer_clean decompress_fuzzer_clean
$(MAKE) -C $(LZ4DIR) clean
42 changes: 42 additions & 0 deletions ossfuzz/compress_frame_fuzzer.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
/**
* This fuzz target attempts to compress the fuzzed data with the simple
* compression function with an output buffer that may be too small to
* ensure that the compressor never crashes.
*/

#include <stddef.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>

#include "fuzz_helpers.h"
#include "lz4.h"
#include "lz4frame.h"
#include "lz4_helpers.h"

int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
uint32_t seed = FUZZ_seed(&data, &size);
LZ4F_preferences_t const prefs = FUZZ_randomPreferences(&seed);
size_t const compressBound = LZ4F_compressFrameBound(size, &prefs);
size_t const dstCapacity = FUZZ_rand32(&seed, 0, compressBound);
char* const dst = (char*)malloc(dstCapacity);
char* const rt = (char*)malloc(size);

FUZZ_ASSERT(dst);
FUZZ_ASSERT(rt);

/* If compression succeeds it must round trip correctly. */
size_t const dstSize =
LZ4F_compressFrame(dst, dstCapacity, data, size, &prefs);
if (!LZ4F_isError(dstSize)) {
size_t const rtSize = FUZZ_decompressFrame(rt, size, dst, dstSize);
FUZZ_ASSERT_MSG(rtSize == size, "Incorrect regenerated size");
FUZZ_ASSERT_MSG(!memcmp(data, rt, size), "Corruption!");
}

free(dst);
free(rt);

return 0;
}
51 changes: 51 additions & 0 deletions ossfuzz/compress_fuzzer.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
/**
* This fuzz target attempts to compress the fuzzed data with the simple
* compression function with an output buffer that may be too small to
* ensure that the compressor never crashes.
*/

#include <stddef.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>

#include "fuzz_helpers.h"
#include "lz4.h"

int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
uint32_t seed = FUZZ_seed(&data, &size);
size_t const dstCapacity = FUZZ_rand32(&seed, 0, LZ4_compressBound(size));
char* const dst = (char*)malloc(dstCapacity);
char* const rt = (char*)malloc(size);

FUZZ_ASSERT(dst);
FUZZ_ASSERT(rt);

/* If compression succeeds it must round trip correctly. */
{
int const dstSize = LZ4_compress_default((const char*)data, dst,
size, dstCapacity);
if (dstSize > 0) {
int const rtSize = LZ4_decompress_safe(dst, rt, dstSize, size);
FUZZ_ASSERT_MSG(rtSize == size, "Incorrect regenerated size");
FUZZ_ASSERT_MSG(!memcmp(data, rt, size), "Corruption!");
}
}

if (dstCapacity > 0) {
/* Compression succeeds and must round trip correctly. */
int compressedSize = size;
int const dstSize = LZ4_compress_destSize((const char*)data, dst,
&compressedSize, dstCapacity);
FUZZ_ASSERT(dstSize > 0);
int const rtSize = LZ4_decompress_safe(dst, rt, dstSize, size);
FUZZ_ASSERT_MSG(rtSize == compressedSize, "Incorrect regenerated size");
FUZZ_ASSERT_MSG(!memcmp(data, rt, compressedSize), "Corruption!");
}

free(dst);
free(rt);

return 0;
}
57 changes: 57 additions & 0 deletions ossfuzz/compress_hc_fuzzer.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
/**
* This fuzz target attempts to compress the fuzzed data with the simple
* compression function with an output buffer that may be too small to
* ensure that the compressor never crashes.
*/

#include <stddef.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>

#include "fuzz_helpers.h"
#include "lz4.h"
#include "lz4hc.h"

int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
uint32_t seed = FUZZ_seed(&data, &size);
size_t const dstCapacity = FUZZ_rand32(&seed, 0, LZ4_compressBound(size));
char* const dst = (char*)malloc(dstCapacity);
char* const rt = (char*)malloc(size);
int const level = FUZZ_rand32(&seed, LZ4HC_CLEVEL_MIN, LZ4HC_CLEVEL_MAX);

FUZZ_ASSERT(dst);
FUZZ_ASSERT(rt);

/* If compression succeeds it must round trip correctly. */
{
int const dstSize = LZ4_compress_HC((const char*)data, dst, size,
dstCapacity, level);
if (dstSize > 0) {
int const rtSize = LZ4_decompress_safe(dst, rt, dstSize, size);
FUZZ_ASSERT_MSG(rtSize == size, "Incorrect regenerated size");
FUZZ_ASSERT_MSG(!memcmp(data, rt, size), "Corruption!");
}
}

if (dstCapacity > 0) {
/* Compression succeeds and must round trip correctly. */
void* state = malloc(LZ4_sizeofStateHC());
FUZZ_ASSERT(state);
int compressedSize = size;
int const dstSize = LZ4_compress_HC_destSize(state, (const char*)data,
dst, &compressedSize,
dstCapacity, level);
FUZZ_ASSERT(dstSize > 0);
int const rtSize = LZ4_decompress_safe(dst, rt, dstSize, size);
FUZZ_ASSERT_MSG(rtSize == compressedSize, "Incorrect regenerated size");
FUZZ_ASSERT_MSG(!memcmp(data, rt, compressedSize), "Corruption!");
free(state);
}

free(dst);
free(rt);

return 0;
}
67 changes: 67 additions & 0 deletions ossfuzz/decompress_frame_fuzzer.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
/**
* This fuzz target attempts to decompress the fuzzed data with the simple
* decompression function to ensure the decompressor never crashes.
*/

#include <stddef.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>

#include "fuzz_helpers.h"
#include "lz4.h"
#define LZ4F_STATIC_LINKING_ONLY
#include "lz4frame.h"
#include "lz4_helpers.h"

static void decompress(LZ4F_dctx* dctx, void* dst, size_t dstCapacity,
const void* src, size_t srcSize,
const void* dict, size_t dictSize,
const LZ4F_decompressOptions_t* opts)
{
LZ4F_resetDecompressionContext(dctx);
if (dictSize == 0)
LZ4F_decompress(dctx, dst, &dstCapacity, src, &srcSize, opts);
else
LZ4F_decompress_usingDict(dctx, dst, &dstCapacity, src, &srcSize,
dict, dictSize, opts);
}

int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{

uint32_t seed = FUZZ_seed(&data, &size);
size_t const dstCapacity = FUZZ_rand32(&seed, 0, 4 * size);
size_t const largeDictSize = 64 * 1024;
size_t const dictSize = FUZZ_rand32(&seed, 0, largeDictSize);
char* const dst = (char*)malloc(dstCapacity);
char* const dict = (char*)malloc(dictSize);
LZ4F_decompressOptions_t opts;
LZ4F_dctx* dctx;
LZ4F_createDecompressionContext(&dctx, LZ4F_VERSION);

FUZZ_ASSERT(dctx);
FUZZ_ASSERT(dst);
FUZZ_ASSERT(dict);

/* Prepare the dictionary. The data doesn't matter for decompression. */
memset(dict, 0, dictSize);


/* Decompress using multiple configurations. */
memset(&opts, 0, sizeof(opts));
opts.stableDst = 0;
decompress(dctx, dst, dstCapacity, data, size, NULL, 0, &opts);
opts.stableDst = 1;
decompress(dctx, dst, dstCapacity, data, size, NULL, 0, &opts);
opts.stableDst = 0;
decompress(dctx, dst, dstCapacity, data, size, dict, dictSize, &opts);
opts.stableDst = 1;
decompress(dctx, dst, dstCapacity, data, size, dict, dictSize, &opts);

LZ4F_freeDecompressionContext(dctx);
free(dst);
free(dict);

return 0;
}
58 changes: 58 additions & 0 deletions ossfuzz/decompress_fuzzer.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
/**
* This fuzz target attempts to decompress the fuzzed data with the simple
* decompression function to ensure the decompressor never crashes.
*/

#include <stddef.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>

#include "fuzz_helpers.h"
#include "lz4.h"

int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{

uint32_t seed = FUZZ_seed(&data, &size);
size_t const dstCapacity = FUZZ_rand32(&seed, 0, 4 * size);
size_t const smallDictSize = size + 1;
size_t const largeDictSize = 64 * 1024 - 1;
size_t const dictSize = MAX(smallDictSize, largeDictSize);
char* const dst = (char*)malloc(dstCapacity);
char* const dict = (char*)malloc(dictSize + size);
char* const largeDict = dict;
char* const dataAfterDict = dict + dictSize;
char* const smallDict = dataAfterDict - smallDictSize;

FUZZ_ASSERT(dst);
FUZZ_ASSERT(dict);

/* Prepare the dictionary. The data doesn't matter for decompression. */
memset(dict, 0, dictSize);
memcpy(dataAfterDict, data, size);

/* Decompress using each possible dictionary configuration. */
/* No dictionary. */
LZ4_decompress_safe_usingDict((char const*)data, dst, size,
dstCapacity, NULL, 0);
/* Small external dictonary. */
LZ4_decompress_safe_usingDict((char const*)data, dst, size,
dstCapacity, smallDict, smallDictSize);
/* Large external dictionary. */
LZ4_decompress_safe_usingDict((char const*)data, dst, size,
dstCapacity, largeDict, largeDictSize);
/* Small prefix. */
LZ4_decompress_safe_usingDict((char const*)dataAfterDict, dst, size,
dstCapacity, smallDict, smallDictSize);
/* Large prefix. */
LZ4_decompress_safe_usingDict((char const*)data, dst, size,
dstCapacity, largeDict, largeDictSize);
/* Partial decompression. */
LZ4_decompress_safe_partial((char const*)data, dst, size,
dstCapacity, dstCapacity);
free(dst);
free(dict);

return 0;
}
48 changes: 48 additions & 0 deletions ossfuzz/fuzz.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
/**
* Fuzz target interface.
* Fuzz targets have some common parameters passed as macros during compilation.
* Check the documentation for each individual fuzzer for more parameters.
*
* @param FUZZ_RNG_SEED_SIZE:
* The number of bytes of the source to look at when constructing a seed
* for the deterministic RNG. These bytes are discarded before passing
* the data to lz4 functions. Every fuzzer initializes the RNG exactly
* once before doing anything else, even if it is unused.
* Default: 4.
* @param LZ4_DEBUG:
* This is a parameter for the lz4 library. Defining `LZ4_DEBUG=1`
* enables assert() statements in the lz4 library. Higher levels enable
* logging, so aren't recommended. Defining `LZ4_DEBUG=1` is
* recommended.
* @param LZ4_FORCE_MEMORY_ACCESS:
* This flag controls how the zstd library accesses unaligned memory.
* It can be undefined, or 0 through 2. If it is undefined, it selects
* the method to use based on the compiler. If testing with UBSAN set
* MEM_FORCE_MEMORY_ACCESS=0 to use the standard compliant method.
* @param FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
* This is the canonical flag to enable deterministic builds for fuzzing.
* Changes to zstd for fuzzing are gated behind this define.
* It is recommended to define this when building zstd for fuzzing.
*/

#ifndef FUZZ_H
#define FUZZ_H

#ifndef FUZZ_RNG_SEED_SIZE
# define FUZZ_RNG_SEED_SIZE 4
#endif

#include <stddef.h>
#include <stdint.h>

#ifdef __cplusplus
extern "C" {
#endif

int LLVMFuzzerTestOneInput(const uint8_t *src, size_t size);

#ifdef __cplusplus
}
#endif

#endif
94 changes: 94 additions & 0 deletions ossfuzz/fuzz_helpers.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
/*
* Copyright (c) 2016-present, Facebook, Inc.
* All rights reserved.
*
* This source code is licensed under both the BSD-style license (found in the
* LICENSE file in the root directory of this source tree) and the GPLv2 (found
* in the COPYING file in the root directory of this source tree).
*/

/**
* Helper functions for fuzzing.
*/

#ifndef FUZZ_HELPERS_H
#define FUZZ_HELPERS_H

#include "fuzz.h"
#include "xxhash.h"
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>

#ifdef __cplusplus
extern "C" {
#endif

#define LZ4_COMMONDEFS_ONLY
#ifndef LZ4_SRC_INCLUDED
#include "lz4.c" /* LZ4_count, constants, mem */
#endif

#define MIN(a,b) ( (a) < (b) ? (a) : (b) )
#define MAX(a,b) ( (a) > (b) ? (a) : (b) )

#define FUZZ_QUOTE_IMPL(str) #str
#define FUZZ_QUOTE(str) FUZZ_QUOTE_IMPL(str)

/**
* Asserts for fuzzing that are always enabled.
*/
#define FUZZ_ASSERT_MSG(cond, msg) \
((cond) ? (void)0 \
: (fprintf(stderr, "%s: %u: Assertion: `%s' failed. %s\n", __FILE__, \
__LINE__, FUZZ_QUOTE(cond), (msg)), \
abort()))
#define FUZZ_ASSERT(cond) FUZZ_ASSERT_MSG((cond), "");

#if defined(__GNUC__)
#define FUZZ_STATIC static __inline __attribute__((unused))
#elif defined(__cplusplus) || \
(defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */)
#define FUZZ_STATIC static inline
#elif defined(_MSC_VER)
#define FUZZ_STATIC static __inline
#else
#define FUZZ_STATIC static
#endif

/**
* Deterministically constructs a seed based on the fuzz input.
* Consumes up to the first FUZZ_RNG_SEED_SIZE bytes of the input.
*/
FUZZ_STATIC uint32_t FUZZ_seed(uint8_t const **src, size_t* size) {
uint8_t const *data = *src;
size_t const toHash = MIN(FUZZ_RNG_SEED_SIZE, *size);
*size -= toHash;
*src += toHash;
return XXH32(data, toHash, 0);
}

#define FUZZ_rotl32(x, r) (((x) << (r)) | ((x) >> (32 - (r))))

FUZZ_STATIC uint32_t FUZZ_rand(uint32_t *state) {
static const uint32_t prime1 = 2654435761U;
static const uint32_t prime2 = 2246822519U;
uint32_t rand32 = *state;
rand32 *= prime1;
rand32 += prime2;
rand32 = FUZZ_rotl32(rand32, 13);
*state = rand32;
return rand32 >> 5;
}

/* Returns a random numer in the range [min, max]. */
FUZZ_STATIC uint32_t FUZZ_rand32(uint32_t *state, uint32_t min, uint32_t max) {
uint32_t random = FUZZ_rand(state);
return min + (random % (max - min + 1));
}

#ifdef __cplusplus
}
#endif

#endif
51 changes: 51 additions & 0 deletions ossfuzz/lz4_helpers.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
#include "fuzz_helpers.h"
#include "lz4_helpers.h"
#include "lz4hc.h"

LZ4F_frameInfo_t FUZZ_randomFrameInfo(uint32_t* seed)
{
LZ4F_frameInfo_t info = LZ4F_INIT_FRAMEINFO;
info.blockSizeID = FUZZ_rand32(seed, LZ4F_max64KB - 1, LZ4F_max4MB);
if (info.blockSizeID < LZ4F_max64KB) {
info.blockSizeID = LZ4F_default;
}
info.blockMode = FUZZ_rand32(seed, LZ4F_blockLinked, LZ4F_blockIndependent);
info.contentChecksumFlag = FUZZ_rand32(seed, LZ4F_noContentChecksum,
LZ4F_contentChecksumEnabled);
info.blockChecksumFlag = FUZZ_rand32(seed, LZ4F_noBlockChecksum,
LZ4F_blockChecksumEnabled);
return info;
}

LZ4F_preferences_t FUZZ_randomPreferences(uint32_t* seed)
{
LZ4F_preferences_t prefs = LZ4F_INIT_PREFERENCES;
prefs.frameInfo = FUZZ_randomFrameInfo(seed);
prefs.compressionLevel = FUZZ_rand32(seed, 0, LZ4HC_CLEVEL_MAX + 3) - 3;
prefs.autoFlush = FUZZ_rand32(seed, 0, 1);
prefs.favorDecSpeed = FUZZ_rand32(seed, 0, 1);
return prefs;
}

size_t FUZZ_decompressFrame(void* dst, const size_t dstCapacity,
const void* src, const size_t srcSize)
{
LZ4F_decompressOptions_t opts;
memset(&opts, 0, sizeof(opts));
opts.stableDst = 1;
LZ4F_dctx* dctx;
LZ4F_createDecompressionContext(&dctx, LZ4F_VERSION);
FUZZ_ASSERT(dctx);

size_t dstSize = dstCapacity;
size_t srcConsumed = srcSize;
size_t const rc =
LZ4F_decompress(dctx, dst, &dstSize, src, &srcConsumed, &opts);
FUZZ_ASSERT(!LZ4F_isError(rc));
FUZZ_ASSERT(rc == 0);
FUZZ_ASSERT(srcConsumed == srcSize);

LZ4F_freeDecompressionContext(dctx);

return dstSize;
}
13 changes: 13 additions & 0 deletions ossfuzz/lz4_helpers.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#ifndef LZ4_HELPERS
#define LZ4_HELPERS

#include "lz4frame.h"

LZ4F_frameInfo_t FUZZ_randomFrameInfo(uint32_t* seed);

LZ4F_preferences_t FUZZ_randomPreferences(uint32_t* seed);

size_t FUZZ_decompressFrame(void* dst, const size_t dstCapacity,
const void* src, const size_t srcSize);

#endif /* LZ4_HELPERS */
23 changes: 23 additions & 0 deletions ossfuzz/ossfuzz.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/bin/bash -eu

# This script is called by the oss-fuzz main project when compiling the fuzz
# targets. This script is regression tested by travisoss.sh.

# Save off the current folder as the build root.
export BUILD_ROOT=$PWD

echo "CC: $CC"
echo "CXX: $CXX"
echo "LIB_FUZZING_ENGINE: $LIB_FUZZING_ENGINE"
echo "CFLAGS: $CFLAGS"
echo "CXXFLAGS: $CXXFLAGS"
echo "OUT: $OUT"

export MAKEFLAGS+="-j$(nproc)"

pushd ossfuzz
make V=1 all
popd

# Copy the fuzzers to the target directory.
cp -v ossfuzz/*_fuzzer $OUT/
39 changes: 39 additions & 0 deletions ossfuzz/round_trip_frame_fuzzer.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
/**
* This fuzz target performs a lz4 round-trip test (compress & decompress),
* compares the result with the original, and calls abort() on corruption.
*/

#include <stddef.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>

#include "fuzz_helpers.h"
#include "lz4.h"
#include "lz4frame.h"
#include "lz4_helpers.h"

int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
uint32_t seed = FUZZ_seed(&data, &size);
LZ4F_preferences_t const prefs = FUZZ_randomPreferences(&seed);
size_t const dstCapacity = LZ4F_compressFrameBound(size, &prefs);
char* const dst = (char*)malloc(dstCapacity);
char* const rt = (char*)malloc(size);

FUZZ_ASSERT(dst);
FUZZ_ASSERT(rt);

/* Compression must succeed and round trip correctly. */
size_t const dstSize =
LZ4F_compressFrame(dst, dstCapacity, data, size, &prefs);
FUZZ_ASSERT(!LZ4F_isError(dstSize));
size_t const rtSize = FUZZ_decompressFrame(rt, size, dst, dstSize);
FUZZ_ASSERT_MSG(rtSize == size, "Incorrect regenerated size");
FUZZ_ASSERT_MSG(!memcmp(data, rt, size), "Corruption!");

free(dst);
free(rt);

return 0;
}
50 changes: 50 additions & 0 deletions ossfuzz/round_trip_fuzzer.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
/**
* This fuzz target performs a lz4 round-trip test (compress & decompress),
* compares the result with the original, and calls abort() on corruption.
*/

#include <stddef.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>

#include "fuzz_helpers.h"
#include "lz4.h"

int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
uint32_t seed = FUZZ_seed(&data, &size);
size_t const dstCapacity = LZ4_compressBound(size);
char* const dst = (char*)malloc(dstCapacity);
char* const rt = (char*)malloc(size);

FUZZ_ASSERT(dst);
FUZZ_ASSERT(rt);

/* Compression must succeed and round trip correctly. */
int const dstSize = LZ4_compress_default((const char*)data, dst,
size, dstCapacity);
FUZZ_ASSERT(dstSize > 0);

int const rtSize = LZ4_decompress_safe(dst, rt, dstSize, size);
FUZZ_ASSERT_MSG(rtSize == size, "Incorrect size");
FUZZ_ASSERT_MSG(!memcmp(data, rt, size), "Corruption!");

/* Partial decompression must succeed. */
{
size_t const partialCapacity = FUZZ_rand32(&seed, 0, size);
char* const partial = (char*)malloc(partialCapacity);
FUZZ_ASSERT(partial);
int const partialSize = LZ4_decompress_safe_partial(
dst, partial, dstSize, partialCapacity, partialCapacity);
FUZZ_ASSERT(partialSize >= 0);
FUZZ_ASSERT_MSG(partialSize == partialCapacity, "Incorrect size");
FUZZ_ASSERT_MSG(!memcmp(data, partial, partialSize), "Corruption!");
free(partial);
}

free(dst);
free(rt);

return 0;
}
39 changes: 39 additions & 0 deletions ossfuzz/round_trip_hc_fuzzer.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
/**
* This fuzz target performs a lz4 round-trip test (compress & decompress),
* compares the result with the original, and calls abort() on corruption.
*/

#include <stddef.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>

#include "fuzz_helpers.h"
#include "lz4.h"
#include "lz4hc.h"

int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
uint32_t seed = FUZZ_seed(&data, &size);
size_t const dstCapacity = LZ4_compressBound(size);
char* const dst = (char*)malloc(dstCapacity);
char* const rt = (char*)malloc(size);
int const level = FUZZ_rand32(&seed, LZ4HC_CLEVEL_MIN, LZ4HC_CLEVEL_MAX);

FUZZ_ASSERT(dst);
FUZZ_ASSERT(rt);

/* Compression must succeed and round trip correctly. */
int const dstSize = LZ4_compress_HC((const char*)data, dst, size,
dstCapacity, level);
FUZZ_ASSERT(dstSize > 0);

int const rtSize = LZ4_decompress_safe(dst, rt, dstSize, size);
FUZZ_ASSERT_MSG(rtSize == size, "Incorrect size");
FUZZ_ASSERT_MSG(!memcmp(data, rt, size), "Corruption!");

free(dst);
free(rt);

return 0;
}
302 changes: 302 additions & 0 deletions ossfuzz/round_trip_stream_fuzzer.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,302 @@
/**
* This fuzz target performs a lz4 streaming round-trip test
* (compress & decompress), compares the result with the original, and calls
* abort() on corruption.
*/

#include <stddef.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>

#include "fuzz_helpers.h"
#define LZ4_STATIC_LINKING_ONLY
#include "lz4.h"
#define LZ4_HC_STATIC_LINKING_ONLY
#include "lz4hc.h"

typedef struct {
char const* buf;
size_t size;
size_t pos;
} const_cursor_t;

typedef struct {
char* buf;
size_t size;
size_t pos;
} cursor_t;

typedef struct {
LZ4_stream_t* cstream;
LZ4_streamHC_t* cstreamHC;
LZ4_streamDecode_t* dstream;
const_cursor_t data;
cursor_t compressed;
cursor_t roundTrip;
uint32_t seed;
int level;
} state_t;

cursor_t cursor_create(size_t size)
{
cursor_t cursor;
cursor.buf = (char*)malloc(size);
cursor.size = size;
cursor.pos = 0;
FUZZ_ASSERT(cursor.buf);
return cursor;
}

typedef void (*round_trip_t)(state_t* state);

void cursor_free(cursor_t cursor)
{
free(cursor.buf);
}

state_t state_create(char const* data, size_t size, uint32_t seed)
{
state_t state;

state.seed = seed;

state.data.buf = (char const*)data;
state.data.size = size;
state.data.pos = 0;

/* Extra margin because we are streaming. */
state.compressed = cursor_create(1024 + 2 * LZ4_compressBound(size));
state.roundTrip = cursor_create(size);

state.cstream = LZ4_createStream();
FUZZ_ASSERT(state.cstream);
state.cstreamHC = LZ4_createStreamHC();
FUZZ_ASSERT(state.cstream);
state.dstream = LZ4_createStreamDecode();
FUZZ_ASSERT(state.dstream);

return state;
}

void state_free(state_t state)
{
cursor_free(state.compressed);
cursor_free(state.roundTrip);
LZ4_freeStream(state.cstream);
LZ4_freeStreamHC(state.cstreamHC);
LZ4_freeStreamDecode(state.dstream);
}

static void state_reset(state_t* state, uint32_t seed)
{
state->level = FUZZ_rand32(&seed, LZ4HC_CLEVEL_MIN, LZ4HC_CLEVEL_MAX);
LZ4_resetStream_fast(state->cstream);
LZ4_resetStreamHC_fast(state->cstreamHC, state->level);
LZ4_setStreamDecode(state->dstream, NULL, 0);
state->data.pos = 0;
state->compressed.pos = 0;
state->roundTrip.pos = 0;
state->seed = seed;
}

static void state_decompress(state_t* state, char const* src, int srcSize)
{
char* dst = state->roundTrip.buf + state->roundTrip.pos;
int const dstCapacity = state->roundTrip.size - state->roundTrip.pos;
int const dSize = LZ4_decompress_safe_continue(state->dstream, src, dst,
srcSize, dstCapacity);
FUZZ_ASSERT(dSize >= 0);
state->roundTrip.pos += dSize;
}

static void state_checkRoundTrip(state_t const* state)
{
char const* data = state->data.buf;
size_t const size = state->data.size;
FUZZ_ASSERT_MSG(size == state->roundTrip.pos, "Incorrect size!");
FUZZ_ASSERT_MSG(!memcmp(data, state->roundTrip.buf, size), "Corruption!");
}

/**
* Picks a dictionary size and trims the dictionary off of the data.
* We copy the dictionary to the roundTrip so our validation passes.
*/
static size_t state_trimDict(state_t* state)
{
/* 64 KB is the max dict size, allow slightly beyond that to test trim. */
uint32_t maxDictSize = MIN(70 * 1024, state->data.size);
size_t const dictSize = FUZZ_rand32(&state->seed, 0, maxDictSize);
DEBUGLOG(2, "dictSize = %zu", dictSize);
FUZZ_ASSERT(state->data.pos == 0);
FUZZ_ASSERT(state->roundTrip.pos == 0);
memcpy(state->roundTrip.buf, state->data.buf, dictSize);
state->data.pos += dictSize;
state->roundTrip.pos += dictSize;
return dictSize;
}

static void state_prefixRoundTrip(state_t* state)
{
while (state->data.pos != state->data.size) {
char const* src = state->data.buf + state->data.pos;
char* dst = state->compressed.buf + state->compressed.pos;
int const srcRemaining = state->data.size - state->data.pos;
int const srcSize = FUZZ_rand32(&state->seed, 0, srcRemaining);
int const dstCapacity = state->compressed.size - state->compressed.pos;
int const cSize = LZ4_compress_fast_continue(state->cstream, src, dst,
srcSize, dstCapacity, 0);
FUZZ_ASSERT(cSize > 0);
state->data.pos += srcSize;
state->compressed.pos += cSize;
state_decompress(state, dst, cSize);
}
}

static void state_extDictRoundTrip(state_t* state)
{
int i = 0;
cursor_t data2 = cursor_create(state->data.size);
memcpy(data2.buf, state->data.buf, state->data.size);
while (state->data.pos != state->data.size) {
char const* data = (i++ & 1) ? state->data.buf : data2.buf;
char const* src = data + state->data.pos;
char* dst = state->compressed.buf + state->compressed.pos;
int const srcRemaining = state->data.size - state->data.pos;
int const srcSize = FUZZ_rand32(&state->seed, 0, srcRemaining);
int const dstCapacity = state->compressed.size - state->compressed.pos;
int const cSize = LZ4_compress_fast_continue(state->cstream, src, dst,
srcSize, dstCapacity, 0);
FUZZ_ASSERT(cSize > 0);
state->data.pos += srcSize;
state->compressed.pos += cSize;
state_decompress(state, dst, cSize);
}
cursor_free(data2);
}

static void state_randomRoundTrip(state_t* state, round_trip_t rt0,
round_trip_t rt1)
{
if (FUZZ_rand32(&state->seed, 0, 1)) {
rt0(state);
} else {
rt1(state);
}
}

static void state_loadDictRoundTrip(state_t* state)
{
char const* dict = state->data.buf;
size_t const dictSize = state_trimDict(state);
LZ4_loadDict(state->cstream, dict, dictSize);
LZ4_setStreamDecode(state->dstream, dict, dictSize);
state_randomRoundTrip(state, state_prefixRoundTrip, state_extDictRoundTrip);
}

static void state_attachDictRoundTrip(state_t* state)
{
char const* dict = state->data.buf;
size_t const dictSize = state_trimDict(state);
LZ4_stream_t* dictStream = LZ4_createStream();
LZ4_loadDict(dictStream, dict, dictSize);
LZ4_attach_dictionary(state->cstream, dictStream);
LZ4_setStreamDecode(state->dstream, dict, dictSize);
state_randomRoundTrip(state, state_prefixRoundTrip, state_extDictRoundTrip);
LZ4_freeStream(dictStream);
}

static void state_prefixHCRoundTrip(state_t* state)
{
while (state->data.pos != state->data.size) {
char const* src = state->data.buf + state->data.pos;
char* dst = state->compressed.buf + state->compressed.pos;
int const srcRemaining = state->data.size - state->data.pos;
int const srcSize = FUZZ_rand32(&state->seed, 0, srcRemaining);
int const dstCapacity = state->compressed.size - state->compressed.pos;
int const cSize = LZ4_compress_HC_continue(state->cstreamHC, src, dst,
srcSize, dstCapacity);
FUZZ_ASSERT(cSize > 0);
state->data.pos += srcSize;
state->compressed.pos += cSize;
state_decompress(state, dst, cSize);
}
}

static void state_extDictHCRoundTrip(state_t* state)
{
int i = 0;
cursor_t data2 = cursor_create(state->data.size);
DEBUGLOG(2, "extDictHC");
memcpy(data2.buf, state->data.buf, state->data.size);
while (state->data.pos != state->data.size) {
char const* data = (i++ & 1) ? state->data.buf : data2.buf;
char const* src = data + state->data.pos;
char* dst = state->compressed.buf + state->compressed.pos;
int const srcRemaining = state->data.size - state->data.pos;
int const srcSize = FUZZ_rand32(&state->seed, 0, srcRemaining);
int const dstCapacity = state->compressed.size - state->compressed.pos;
int const cSize = LZ4_compress_HC_continue(state->cstreamHC, src, dst,
srcSize, dstCapacity);
FUZZ_ASSERT(cSize > 0);
DEBUGLOG(2, "srcSize = %d", srcSize);
state->data.pos += srcSize;
state->compressed.pos += cSize;
state_decompress(state, dst, cSize);
}
cursor_free(data2);
}

static void state_loadDictHCRoundTrip(state_t* state)
{
char const* dict = state->data.buf;
size_t const dictSize = state_trimDict(state);
LZ4_loadDictHC(state->cstreamHC, dict, dictSize);
LZ4_setStreamDecode(state->dstream, dict, dictSize);
state_randomRoundTrip(state, state_prefixHCRoundTrip,
state_extDictHCRoundTrip);
}

static void state_attachDictHCRoundTrip(state_t* state)
{
char const* dict = state->data.buf;
size_t const dictSize = state_trimDict(state);
LZ4_streamHC_t* dictStream = LZ4_createStreamHC();
LZ4_setCompressionLevel(dictStream, state->level);
LZ4_loadDictHC(dictStream, dict, dictSize);
LZ4_attach_HC_dictionary(state->cstreamHC, dictStream);
LZ4_setStreamDecode(state->dstream, dict, dictSize);
state_randomRoundTrip(state, state_prefixHCRoundTrip,
state_extDictHCRoundTrip);
LZ4_freeStreamHC(dictStream);
}

round_trip_t roundTrips[] = {
&state_prefixRoundTrip,
&state_extDictRoundTrip,
&state_loadDictRoundTrip,
&state_attachDictRoundTrip,
&state_prefixHCRoundTrip,
&state_extDictHCRoundTrip,
&state_loadDictHCRoundTrip,
&state_attachDictHCRoundTrip,
};

int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
uint32_t seed = FUZZ_seed(&data, &size);
state_t state = state_create((char const*)data, size, seed);
const int n = sizeof(roundTrips) / sizeof(round_trip_t);
int i;

for (i = 0; i < n; ++i) {
DEBUGLOG(2, "Round trip %d", i);
state_reset(&state, seed);
roundTrips[i](&state);
state_checkRoundTrip(&state);
}

state_free(state);

return 0;
}
74 changes: 74 additions & 0 deletions ossfuzz/standaloneengine.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>

#include "fuzz.h"

/**
* Main procedure for standalone fuzzing engine.
*
* Reads filenames from the argument array. For each filename, read the file
* into memory and then call the fuzzing interface with the data.
*/
int main(int argc, char **argv)
{
int ii;
for(ii = 1; ii < argc; ii++)
{
FILE *infile;
printf("[%s] ", argv[ii]);

/* Try and open the file. */
infile = fopen(argv[ii], "rb");
if(infile)
{
uint8_t *buffer = NULL;
size_t buffer_len;

printf("Opened.. ");

/* Get the length of the file. */
fseek(infile, 0L, SEEK_END);
buffer_len = ftell(infile);

/* Reset the file indicator to the beginning of the file. */
fseek(infile, 0L, SEEK_SET);

/* Allocate a buffer for the file contents. */
buffer = (uint8_t *)calloc(buffer_len, sizeof(uint8_t));
if(buffer)
{
/* Read all the text from the file into the buffer. */
fread(buffer, sizeof(uint8_t), buffer_len, infile);
printf("Read %zu bytes, fuzzing.. ", buffer_len);

/* Call the fuzzer with the data. */
LLVMFuzzerTestOneInput(buffer, buffer_len);

printf("complete !!");

/* Free the buffer as it's no longer needed. */
free(buffer);
buffer = NULL;
}
else
{
fprintf(stderr,
"[%s] Failed to allocate %zu bytes \n",
argv[ii],
buffer_len);
}

/* Close the file as it's no longer needed. */
fclose(infile);
infile = NULL;
}
else
{
/* Failed to open the file. Maybe wrong name or wrong permissions? */
fprintf(stderr, "[%s] Open failed. \n", argv[ii]);
}

printf("\n");
}
}
21 changes: 21 additions & 0 deletions ossfuzz/travisoss.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/bin/bash

set -ex

# Clone the oss-fuzz repository
git clone https://github.com/google/oss-fuzz.git /tmp/ossfuzz

if [[ ! -d /tmp/ossfuzz/projects/lz4 ]]
then
echo "Could not find the lz4 project in ossfuzz"
exit 1
fi

# Modify the oss-fuzz Dockerfile so that we're checking out the current branch on travis.
sed -i "s@https://github.com/lz4/lz4.git@-b $TRAVIS_BRANCH https://github.com/lz4/lz4.git@" /tmp/ossfuzz/projects/lz4/Dockerfile

# Try and build the fuzzers
pushd /tmp/ossfuzz
python infra/helper.py build_image --pull lz4
python infra/helper.py build_fuzzers lz4
popd
6 changes: 5 additions & 1 deletion programs/lz4.1
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
.
.TH "LZ4" "1" "April 2019" "lz4 1.9.1" "User Commands"
.TH "LZ4" "1" "July 2019" "lz4 1.9.2" "User Commands"
.
.SH "NAME"
\fBlz4\fR \- lz4, unlz4, lz4cat \- Compress or decompress \.lz4 files
Expand Down Expand Up @@ -120,6 +120,10 @@ Compression level, with # being any value from 1 to 12\. Higher values trade com
Switch to ultra\-fast compression levels\. The higher the value, the faster the compression speed, at the cost of some compression ratio\. If \fB=#\fR is not present, it defaults to \fB1\fR\. This setting overrides compression level if one was set previously\. Similarly, if a compression level is set after \fB\-\-fast\fR, it overrides it\.
.
.TP
\fB\-\-best\fR
Set highest compression level\. Same as -12\.
.
.TP
\fB\-\-favor\-decSpeed\fR
Generate compressed data optimized for decompression speed\. Compressed data will be larger as a consequence (typically by ~0\.5%), while decompression speed will be improved by 5\-20%, depending on use cases\. This option only works in combination with very high compression levels (>=10)\.
.
Expand Down
3 changes: 3 additions & 0 deletions programs/lz4.1.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,9 @@ only the latest one will be applied.
This setting overrides compression level if one was set previously.
Similarly, if a compression level is set after `--fast`, it overrides it.

* `--best`:
Set highest compression level. Same as -12.

* `--favor-decSpeed`:
Generate compressed data optimized for decompression speed.
Compressed data will be larger as a consequence (typically by ~0.5%),
Expand Down
43 changes: 31 additions & 12 deletions programs/lz4cli.c
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ static int g_lz4c_legacy_commands = 0;
/*-************************************
* Macros
***************************************/
#define DISPLAYOUT(...) fprintf(stdout, __VA_ARGS__)
#define DISPLAY(...) fprintf(stderr, __VA_ARGS__)
#define DISPLAYLEVEL(l, ...) if (displayLevel>=l) { DISPLAY(__VA_ARGS__); }
static unsigned displayLevel = 2; /* 0 : no display ; 1: errors only ; 2 : downgradable normal ; 3 : non-downgradable normal; 4 : + information */
Expand Down Expand Up @@ -141,10 +142,11 @@ static int usage_advanced(const char* exeName)
DISPLAY( " -BX : enable block checksum (default:disabled) \n");
DISPLAY( "--no-frame-crc : disable stream checksum (default:enabled) \n");
DISPLAY( "--content-size : compressed frame includes original size (default:not present)\n");
DISPLAY( "--list : lists information about .lz4 files. Useful if compressed with --content-size flag.\n");
DISPLAY( "--list FILE : lists information about .lz4 files (useful for files compressed with --content-size flag)\n");
DISPLAY( "--[no-]sparse : sparse mode (default:enabled on file, disabled on stdout)\n");
DISPLAY( "--favor-decSpeed: compressed files decompress faster, but are less compressed \n");
DISPLAY( "--fast[=#]: switch to ultra fast compression level (default: %i)\n", 1);
DISPLAY( "--best : same as -%d\n", LZ4HC_CLEVEL_MAX);
DISPLAY( "Benchmark arguments : \n");
DISPLAY( " -b# : benchmark file(s), using # compression level (default : 1) \n");
DISPLAY( " -e# : test all compression levels from -bX to # (default : 1)\n");
Expand Down Expand Up @@ -390,7 +392,7 @@ int main(int argc, const char** argv)
if (!strcmp(argument, "--favor-decSpeed")) { LZ4IO_favorDecSpeed(prefs, 1); continue; }
if (!strcmp(argument, "--verbose")) { displayLevel++; continue; }
if (!strcmp(argument, "--quiet")) { if (displayLevel) displayLevel--; continue; }
if (!strcmp(argument, "--version")) { DISPLAY(WELCOME_MESSAGE); return 0; }
if (!strcmp(argument, "--version")) { DISPLAYOUT(WELCOME_MESSAGE); return 0; }
if (!strcmp(argument, "--help")) { usage_advanced(exeName); goto _cleanup; }
if (!strcmp(argument, "--keep")) { LZ4IO_setRemoveSrcFile(prefs, 0); continue; } /* keep source file (default) */
if (!strcmp(argument, "--rm")) { LZ4IO_setRemoveSrcFile(prefs, 1); continue; }
Expand All @@ -413,6 +415,9 @@ int main(int argc, const char** argv)
}
continue;
}

/* For gzip(1) compatibility */
if (!strcmp(argument, "--best")) { cLevel=LZ4HC_CLEVEL_MAX; continue; }
}

while (argument[1]!=0) {
Expand All @@ -437,7 +442,7 @@ int main(int argc, const char** argv)
switch(argument[0])
{
/* Display help */
case 'V': DISPLAY(WELCOME_MESSAGE); goto _cleanup; /* Version */
case 'V': DISPLAYOUT(WELCOME_MESSAGE); goto _cleanup; /* Version */
case 'h': usage_advanced(exeName); goto _cleanup;
case 'H': usage_longhelp(exeName); goto _cleanup;

Expand Down Expand Up @@ -648,13 +653,19 @@ int main(int argc, const char** argv)
DISPLAYLEVEL(1, "refusing to read from a console\n");
exit(1);
}
/* if input==stdin and no output defined, stdout becomes default output */
if (!strcmp(input_filename, stdinmark) && !output_filename)
output_filename = stdoutmark;
if (!strcmp(input_filename, stdinmark)) {
/* if input==stdin and no output defined, stdout becomes default output */
if (!output_filename) output_filename = stdoutmark;
}
else{
if (!recursive && !UTIL_isRegFile(input_filename)) {
DISPLAYLEVEL(1, "%s: is not a regular file \n", input_filename);
exit(1);
}
}

/* No output filename ==> try to select one automatically (when possible) */
while ((!output_filename) && (multiple_inputs==0)) {

if (!IS_CONSOLE(stdout)) {
/* Default to stdout whenever stdout is not the console.
* Note : this policy may change in the future, therefore don't rely on it !
Expand Down Expand Up @@ -693,7 +704,19 @@ int main(int argc, const char** argv)
break;
}

if (multiple_inputs==0 && mode != om_list) assert(output_filename);
if (mode == om_list){
/* Exit if trying to read from stdin as this isn't supported in this mode */
if(!strcmp(input_filename, stdinmark)){
DISPLAYLEVEL(1, "refusing to read from standard input in --list mode\n");
exit(1);
}
if(!multiple_inputs){
inFileNames[ifnIdx++] = input_filename;
}
}
else{
if (multiple_inputs==0) assert(output_filename);
}
/* when multiple_inputs==1, output_filename may simply be useless,
* however, output_filename must be !NULL for next strcmp() tests */
if (!output_filename) output_filename = "*\\dummy^!//";
Expand Down Expand Up @@ -723,11 +746,7 @@ int main(int argc, const char** argv)
operationResult = DEFAULT_DECOMPRESSOR(prefs, input_filename, output_filename);
}
} else if (mode == om_list){
if(!multiple_inputs){
inFileNames[ifnIdx++] = input_filename;
}
operationResult = LZ4IO_displayCompressedFilesInfo(inFileNames, ifnIdx);
inFileNames=NULL;
} else { /* compression is default action */
if (legacy_format) {
DISPLAYLEVEL(3, "! Generating LZ4 Legacy format (deprecated) ! \n");
Expand Down
Loading