Permalink
Browse files

Initial import

  • Loading branch information...
1 parent 725e478 commit 6dcceb0b1b2bf47c538060dfa126d87dbfb4b650 @ckolivas committed Mar 28, 2010
Showing with 26,485 additions and 0 deletions.
  1. +16 −0 AUTHORS
  2. +4 −0 BUGS
  3. +339 −0 COPYING
  4. +328 −0 ChangeLog
  5. +126 −0 Makefile
  6. +118 −0 Makefile.in
  7. +92 −0 README-NOT-BACKWARD-COMPATIBLE
  8. +18 −0 TODO
  9. +123 −0 WHATS-NEW
  10. +76 −0 aclocal.m4
  11. +1,516 −0 config.guess
  12. +135 −0 config.h
  13. +134 −0 config.h.in
  14. +1,579 −0 config.sub
  15. +6,134 −0 configure
  16. +88 −0 configure.ac
  17. +1 −0 description-pak
  18. +44 −0 doc/README.Assembler
  19. +120 −0 doc/README.benchmarks
  20. +118 −0 doc/README.lzo_compresses.test.txt
  21. +45 −0 doc/lrzip.conf.example
  22. +41 −0 doc/magic.header.txt
  23. +238 −0 install-sh
  24. +48 −0 lrztar
  25. +194 −0 lzma/7zC.txt
  26. +471 −0 lzma/7zFormat.txt
  27. +102 −0 lzma/ASM/x86/7zCrcT8U.s
  28. +105 −0 lzma/ASM/x86_64/7zCrcT8U_64.s
  29. +35 −0 lzma/C/7zCrc.c
  30. +30 −0 lzma/C/7zCrc.h
  31. +43 −0 lzma/C/7zCrcT8.c
  32. +127 −0 lzma/C/Alloc.c
  33. +37 −0 lzma/C/Alloc.h
  34. +761 −0 lzma/C/LzFind.c
  35. +115 −0 lzma/C/LzFind.h
  36. +793 −0 lzma/C/LzFindMt.c
  37. +105 −0 lzma/C/LzFindMt.h
  38. +54 −0 lzma/C/LzHash.h
  39. +1,007 −0 lzma/C/LzmaDec.c
  40. +231 −0 lzma/C/LzmaDec.h
  41. +2,268 −0 lzma/C/LzmaEnc.c
  42. +80 −0 lzma/C/LzmaEnc.h
  43. +46 −0 lzma/C/LzmaLib.c
  44. +135 −0 lzma/C/LzmaLib.h
  45. +55 −0 lzma/C/MyGuidDef.h
  46. +226 −0 lzma/C/MyWindows.h
  47. +582 −0 lzma/C/Threads.c
  48. +121 −0 lzma/C/Threads.h
  49. +228 −0 lzma/C/Types.h
  50. +228 −0 lzma/C/Types.h~
  51. +19 −0 lzma/C/basetyps.h
  52. +194 −0 lzma/C/windows.h
  53. +141 −0 lzma/Methods.txt
  54. +31 −0 lzma/README
  55. +31 −0 lzma/README-Alloc
  56. +231 −0 lzma/history.txt
  57. +594 −0 lzma/lzma.txt
  58. +872 −0 main.c
  59. +83 −0 man/lrunzip.1.pod
  60. +276 −0 man/lrzip.1
  61. +59 −0 man/lrzip.conf.5
  62. +85 −0 man/lrztar.1.pod
  63. +66 −0 pod2man.mk
  64. +274 −0 runzip.c
  65. +659 −0 rzip.c
  66. +185 −0 rzip.h
  67. +1,050 −0 stream.c
  68. +204 −0 util.c
  69. +1,771 −0 zpipe.cpp
View
16 AUTHORS
@@ -0,0 +1,16 @@
+Original concept and programming by Con Kolivas
+
+Beginning with version 0.19, Peter Hyman submitted bug fixes,
+patches, multi-threading support, assembler integration,
+SDK updating, and autoconf improvements.
+
+Thanks to:
+Andrew Tridgell for rzip.
+Markus Oberhumer for lzo.
+Igor Pavlov for lzma and CRC Assembler code.
+Jean-loup Gailly and Mark Adler for the zlib compression library
+Christian Leber for lzma compat layer
+Lasse Collin for debugging the compat layer
+Michael J Cohen for Darwin support
+Jukka Laurila for newer Darwin support
+George Makrydakis for lrztar
View
4 BUGS
@@ -0,0 +1,4 @@
+BUGME November 2009
+
+MacOSX build is broken as of v0.41. No easy fix is planned
+at the moment.
View
339 COPYING

Large diffs are not rendered by default.

Oops, something went wrong.
View
328 ChangeLog
@@ -0,0 +1,328 @@
+lrzip ChangeLog
+DEC 2009, version 0.44, Con Kolivas, George Makrydakis
+* Added lrztar wrapper to manage whole directories.
+* Added -i option to provide information about a compressed file.
+* Fixed "nan" showing as Compression speed on very small files.
+* Fixed build for old bz library.
+* Avoid overwriting output file if input doesn't exist.
+* Implement signal handler to delete temporary files.
+
+
+DEC 2009, version 0.43, Con Kolivas, Jukka Laurila
+* Darwin support thanks to Jukka Laurila.
+* Finally added stdin/stdout support due to popular demand. This is done
+by basically using temporary files so is a low performance way of using
+lrzip.
+* Added test function. This just uses a temporary file during decompression.
+* Config files should now accept zpaq options.
+* Minor code style cleanups.
+* Updated benchmarks in docs.
+* Add a warning when attempting to decompress a file from a newer lrzip
+version.
+
+
+NOV 2009, version 0.42, Con Kolivas
+* Changed progress update to show which of 2 chunks are being compressed
+in zpaq.
+* Fixed progress update in ZPAQ to not update with each byte which was
+wasting heaps of CPU time.
+
+
+NOV 2009, version 0.41, Con Kolivas
+* Added zpaq compression backend for extremely good but extremely slow
+compression (incompatible with previous versions if used).
+* Limited chunk size passed to LZMA to 4GB to avoid library overflows.
+* Minor changes to the formatting output
+* Changed lower limit of -T threshhold to 0 to allow disabling it.
+* Added lzo_compresses check into zpaq and bzip2 as well since they're
+slow.
+
+
+NOV 2009, version 0.40, Con Kolivas
+* Massive core code rewrite.
+* All code moved to be 64bit based for compression block addressing and length
+allowing compression windows to be limited by ram only.
+* 64bit userspace should now have no restriction on compression window size,
+32bit is still limited to 2GB windows due to userspace limitations.
+* New file format using the new addressing and data types, incompatible with
+versions prior to 0.40.
+* Support for reading and decompressing older formats.
+* Minor speedups in read/write routines.
+* Countless minor code fixes throughout.
+* Code style cleanups and consistency changes in core code.
+* Configure script improvements.
+
+
+NOV 2009, version 0.31, Con Kolivas
+* Updated to be in sync with lzma SDK 9.07beta.
+* Cleanups and fixes of the configure scripts to use the correct package version
+name.
+* Massive fixes to the memory management code fixing lots of 32bit overflow
+errors. The window size limit is now 2GB on both 32bit and 64bit. While it
+appears to be smaller than the old windows, only 900MB was being used on .30
+even though it claimed to use more. This can cause huge improvements in the
+compression of very large files.
+* The offset when mmap()ing was not being set to a multiple of page size so
+it would fail if the window size was not a multiple of it.
+* Flushing of data to disk between compression windows was implemented to
+minimise disk thrashing of read vs write.
+
+
+NOV 2009, version 0.30, Con Kolivas
+* Numerous bugfixes to try and make the most of 64bit environments with huge
+memory and to barf less on 32bit environments.
+* Executable stacks were fixed.
+* Probably other weird and wonderful bugs have been introduced.
+* -P option to not set permissions on output files allowing you to write to
+braindead filesystems (eg fat32).
+
+
+JAN 2009, version 0.24, Peter Hyman, pete@peterhyman.com
+Happy New Year!
+* Upgrade LZMA SDK to 4.63. Use new C Wrapper. Invalidates
+ LZMA archives created earlier due to new Magic property
+ bytes.
+* New LZMA logic will automatically determine allow LZMA
+ code to determine optimal lc, lp, pb, fb, and dictionary
+ size settings. stream.c will only pass level and thread
+ information. Compress function will return encoded 5 byte
+ data with compression settings. This will be stored in lrz
+ file header.
+* add error messages during LZMA compression. There are some
+ edge cases where LZMA cannot allocate memory. These errors
+ are reported and the user will be advised to use a lower
+ compression window setting.
+* type changes in rzip_fd function for correctness.
+* remove function *Realloc() since it was never used. Cleaned
+ in rzip.h and util.c.
+* apply munmap prior to closing and compressing stream in
+ function rzip_chunk in rzip.c.
+* add realloc function in close_stream_out in stream.c
+ to reclaim some ram and try and allieviate out of memory
+ conditions in LZMA compression.
+* remove file acconfig.h and include DEFINE in configure.in.
+* add lrzip.conf capability.
+* add timer for compression including elapsed time and eta.
+* add compression and decompression MB/s calculation.
+* Updated WHATS-NEW, TODO and created BUGS file.
+* Updated lrzip.1 manpage and created lrzip.conf.5 manpage.
+* Added lrzip.conf.example file in doc directory.
+
+MAR 2008, Con Kolivas, kernel@kolivas.org
+* Numerous changes all over to place restrictions on window
+size to work with 32 bit limitations.
+* Various bugfixes with respect to detecting buffer sizes and
+likelihood of compressibility.
+* Fixed the inappropriate straight copying uncompressed data for
+files larger than 4GB.
+* Re-initiated the 10MB window limits for non-lzma compression.
+I was unable to reproduce any file size savings.
+* Allow compression windows larger than ramsize if people really
+really want them.
+* Decrease thresholds for the test function to a minimum of 5%
+compressibility since the hanging in lzma compression bug has been
+fixed.
+
+JAN 2008, version 0.22, Peter Hyman, pete@peterhyman.com
+* version update
+lzma/LZMALib.cpp
+ Thanks to Lasse Collin for debugging the problem LZMA
+ had with hanging on uncompressable files.
+ Update for control parameters to both compress and
+ decompress functions.
+Makefile.in
+* use of @top_srcdir@ (Lasse Collin). Also moved away
+ more cruft.
+main.c stream.c.rzip.h LZMALib.cpp lzmalib.h
+* addition of three new control structure members.
+ control.lc -- literal context bits
+ control.lp -- literal post state bits
+ control.pb -- post state bits
+ These are needed to ensure decompression will work.
+ These will now be stored along with control.compression_level
+ in the lrz file beginning at offset 0x16 for three bytes.
+ These will be passed to the functions lzma_compresses and
+ lzma_uncompress. Currently, only compression level is
+ needed or used, but the others are stored for possible future
+ use.
+ See magic file for more information.
+stream.c
+* Change to lzo_compresses function that will reject a chunk
+ without testing it if the size of the chunk is greater
+ than the compression window * threshold. This is to avoid
+ a low probability that lzma would still be passed a chunk
+ that contains uncompressible data or barely compressible
+ data. If after rzip hashing the chunk size is still close
+ to the window size, there is hardly anything worth
+ compressing. While there is no reason lzma cannot get the
+ chunk, this will save a lot of time.
+magic.headers.txt
+* updated file to show new layout that includes lzma
+ parameters.
+README-NOT-BACKWARD-COMPATIBLE
+* added warning about using lrzip-0.22 with earlier versions.
+WHATS-NEW
+* highlight of new features.
+
+DEC 2007, version 0.21. Peter Hyman, pete@peterhyman.com
+* version update.
+* Modified to use Assembler routines from lzma SDK for CRC
+ computation when hashing streams in rzip.c and runzip.c.
+ Added files 7zCrcT8.c and 7zCrcT8u.s to lzma tree.
+ Cleaned up source tree. Moved unused files out of the way.
+ Moved non-core docs to doc directory
+configure.in
+* correct AC_INIT to set program variables.
+* modified to add check for nasm assembler.
+* modified syntax of test for errno in error.h to use
+ echo $ECHO_N/$ECHO_C instead of $ac_n/$ac_c which
+ was incorrect.
+Makefile.in, lzma/Makefile
+* modified to add compile instructions for 7zCrcT8.c
+ and 7zCrcT8U.s and Assembler. Cleaned up to remove
+ targets that don't exist or sources that don't
+ exist.
+ Modified to properly set directories. Added doc install.
+ Add link command to symlink lrunzip to lrzip.
+*main.c
+ Add CrcGenerateTable() function to init CRC tables.
+ This is needed for all crc routines including those
+ in MatchFinderMT.
+rzip.c and runzip.c
+* Updated source to change call to crc32_buffer to call
+ CrcUpdate in the assembler code. Changed parameter order
+ to conform.
+stream.c
+* Removed 10MB limit on streams for bzip, gzip, and lzo.
+ This, to improve effeciency of long range analysis. For
+ some files, this could improve results.
+Current-Benchmarks.txt
+* Added file to keep benchmarks current to version.
+ (probably need to update README too).
+README.Assembler
+* Explain how to remove default compile of Assembler
+ modules.
+config.sub config.guess
+* added files for system detection.
+
+DEC 2007, version 0.20. Peter Hyman, pete@peterhyman.com
+
+* Updated to LZMA SDK 4.57.
+* Updated to p7zip POSIX version. (www.p7zip.org)
+* Added multi-threading support (up to 2x speed with LZMA).
+* Edited LZMADecompress.cpp for backward compatibility
+ with decompress function. Needed SetPropertiesRaw function.
+* Repopulated source tree for distribution.
+* Updated Makefile.in to reflect new source files.
+ Updated to include command to link lrunzip to lrzip because
+ lrzip will test if lrunzip was used on command line.
+* Updated Makefile.in for new compile time and linking options.
+* Updated LZMALibs.cpp to include new property members for
+ LZMAEncoders as well as changed default dictionaries to
+ level+16. This would make the default compression level
+ of 7 translate to a dictionary number of 23.
+* Added output to show Nice Level when verbose mode set
+ Initial add of support for zlib which seems to give quite
+ excellent performance.
+* configure.in added AC_CHECK for libz and libm.
+ Added AC_PROG_LN_S for Makefile symlink section.
+* lrzip.1 updated man page for -g option
+* main.c added option test for gzip
+ Added sysconf(_SC_NPROCESSORS_CONF) for CPU detection
+ for threading.
+ Updated verbose output to show whether or not
+ Threading will be used.
+ Added Timer for each file compressed.
+* rzip.h added flags for GZIP compression.
+ Added control member for threads. Arg passed to
+ lzma_conpress.
+* stream.c update to accomodate gzip compress and decompress
+ functions. Cleaned up file by rearranging functions into
+ groups.
+ Removed include of lzmalib.h since it was causing a
+ compile time warning with zlib.h. Prototyped functions
+ manually.
+ Cleanup output from lzo_compresses function so that
+ unnecessary linefeeds are eliminated.
+ lzma_compress function call now uses threads as argument.
+* Added README.benchmarks file to explain a method of
+ comparing results between different methods.
+* LZMALib.cpp, lzmalib.h. Adjust function lzma_compress
+ prototype and function to include new argument threads.
+ This parameter is now placed in properties.
+* lzma/Makefile. Updated to reflect new API library.
+ Updated to include Threading option.
+
+DEC 2007, version 0.19. Con Kolivas.
+* Added nice support, defaulting to nice 19.
+
+DEC 2007, version 0.19. Peter Hyman, pete@peterhyman.com
+
+* Major goal was to stop LZMA from hanging on some files.
+ Accomplished this with a threhold setting that is used by
+ the lzo_compresses function to better analyze chunk data.
+ Threshold makes it less likely that uncompressible data
+ will be passed to the LZMA compressor.
+
+main.c
+* Added Threshold option 1-10 to control LZMA compression attempt.
+ Default value=2. This means that anything over 10% compression
+ as reported by lzo_compresses will return a true value to
+ the LZMA compression function.
+* Added verbosity option and more verbosity option (-v[v]).
+* Added -O option to specify output directory.
+* Updated compress_file and decompress_file functions to handle.
+ output directories and better handle multi files and filename
+ extensions. Optimized some string handling routines.
+ Improved flexibility in determining location of output files
+ when using -O. Added fflush(stdout) to improve printf reliability.
+* decompress_file will accept any filename and will automatically
+ append .lrz if not present. Won't automatically fail.
+* Added logic to protect against conflicting options such as
+ -q and -v, -o and -O.
+* Added printout to screen of options selected. Will display
+ only when -v or -vv used.
+* Adjusted several printf statements to avoid compiler
+ warnings (use %ll for long long int types).
+
+runzip.c
+* Added decompression progress indicator.
+ Will show percent decompressed along with bytes decompressed
+ and total to be decompressed. Will show if -q option NOT used.
+
+rzip.h
+* Version incremented to 0.19.
+* Added flag DEFINESs for verbosity and more verbosity.
+* Updated control struct to include output directory and
+ threshold value. Removed verbosity member.
+
+rzip.c
+* Minor changes to handle display when verbosity set. Changed
+ number format in some printf statements to properly handle
+ unsigned data.
+
+stream.c
+* major overhaul of lzo_compresses function to use a threshold
+ value when testing a data chunk to see if it is suitable for
+ LZMA compression. Optimized test loop to improve performance
+ and reduce number of passes. Improved output reporting depending
+ on verbosity setting.
+* Added print controls for verbosity option.
+* Corrected if statements that tested for error condition of
+ some lzo functions that only return a true value regardless.
+
+lrzip.1
+* updated man page to show new options and explain -T threshold.
+
+README
+* updated README to explain -T threshold option.
+
+README.lzo_compresses.test.txt
+* Added this file to help explain the theory behind the rewrite
+ of the lzo_compresses function and how to use the -T option.
+
+TODO
+* wish list and future enhancements.
+
+ChangeLog
+* added file.
Oops, something went wrong.

0 comments on commit 6dcceb0

Please sign in to comment.