Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

Fix public issue 82: Stop distributing benchmark data files that have

unclear or unsuitable licensing.

In general, we replace the files we can with liberally licensed data,
and remove all the others (in particular all the parts of the Canterbury
corpus that are not clearly in the public domain). The replacements
do not always have the exact same characteristics as the original ones,
but they are more than good enough to be useful for benchmarking.
  • Loading branch information...
commit f1614929a748d3cc666d67311bb10babb43983e5 1 parent 0668fe9
snappy authored
26 COPYING
View
@@ -26,3 +26,29 @@ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+===
+
+Some of the benchmark data in util/zippy/testdata is licensed differently:
+
+ - fireworks.jpeg is Copyright 2013 Steinar H. Gunderson, and
+ is licensed under the Creative Commons Attribution 3.0 license
+ (CC-BY-3.0). See https://creativecommons.org/licenses/by/3.0/
+ for more information.
+
+ - kppkn.gtb is taken from the Gaviota chess tablebase set, and
+ is licensed under the MIT License. See
+ https://sites.google.com/site/gaviotachessengine/Home/endgame-tablebases-1
+ for more information.
+
+ - paper-100k.pdf is an excerpt (bytes 92160 to 194560) from the paper
+ “Combinatorial Modeling of Chromatin Features Quantitatively Predicts DNA
+ Replication Timing in _Drosophila_” by Federico Comoglio and Renato Paro,
+ which is licensed under the CC-BY license. See
+ http://www.ploscompbiol.org/static/license for more ifnormation.
+
+ - alice29.txt, asyoulik.txt, plrabn12.txt and lcet10.txt are from Project
+ Gutenberg. The first three have expired copyrights and are in the public
+ domain; the latter does not have expired copyright, but is still in the
+ public domain according to the license information
+ (http://www.gutenberg.org/ebooks/53).
2  Makefile.am
View
@@ -16,7 +16,7 @@ snappy_unittest_LDADD = libsnappy.la $(UNITTEST_LIBS) $(gflags_LIBS) $(GTEST_LIB
TESTS = snappy_unittest
noinst_PROGRAMS = $(TESTS)
-EXTRA_DIST = autogen.sh testdata/alice29.txt testdata/asyoulik.txt testdata/baddata1.snappy testdata/baddata2.snappy testdata/baddata3.snappy testdata/cp.html testdata/fields.c testdata/geo.protodata testdata/grammar.lsp testdata/house.jpg testdata/html testdata/html_x_4 testdata/kennedy.xls testdata/kppkn.gtb testdata/lcet10.txt testdata/mapreduce-osdi-1.pdf testdata/plrabn12.txt testdata/ptt5 testdata/sum testdata/urls.10K testdata/xargs.1
+EXTRA_DIST = autogen.sh testdata/alice29.txt testdata/asyoulik.txt testdata/baddata1.snappy testdata/baddata2.snappy testdata/baddata3.snappy testdata/geo.protodata testdata/fireworks.jpeg testdata/html testdata/html_x_4 testdata/kppkn.gtb testdata/lcet10.txt testdata/paper-100k.pdf testdata/plrabn12.txt testdata/urls.10K
dist_doc_DATA = ChangeLog COPYING INSTALL NEWS README format_description.txt framing_format.txt
libtool: $(LIBTOOL_DEPS)
16 snappy_unittest.cc
View
@@ -1191,23 +1191,14 @@ static struct {
} files[] = {
{ "html", "html", 0 },
{ "urls", "urls.10K", 0 },
- { "jpg", "house.jpg", 0 },
- { "jpg_200", "house.jpg", 200 },
- { "pdf", "mapreduce-osdi-1.pdf", 0 },
+ { "jpg", "fireworks.jpeg", 0 },
+ { "jpg_200", "fireworks.jpeg", 200 },
+ { "pdf", "paper-100k.pdf", 0 },
{ "html4", "html_x_4", 0 },
- { "cp", "cp.html", 0 },
- { "c", "fields.c", 0 },
- { "lsp", "grammar.lsp", 0 },
- { "xls", "kennedy.xls", 0 },
- { "xls_200", "kennedy.xls", 200 },
{ "txt1", "alice29.txt", 0 },
{ "txt2", "asyoulik.txt", 0 },
{ "txt3", "lcet10.txt", 0 },
{ "txt4", "plrabn12.txt", 0 },
- { "bin", "ptt5", 0 },
- { "bin_200", "ptt5", 200 },
- { "sum", "sum", 0 },
- { "man", "xargs.1", 0 },
{ "pb", "geo.protodata", 0 },
{ "gaviota", "kppkn.gtb", 0 },
};
@@ -1344,7 +1335,6 @@ BENCHMARK(BM_ZFlat)->DenseRange(0, ARRAYSIZE(files) - 1);
int main(int argc, char** argv) {
InitGoogle(argv[0], &argc, &argv, true);
- File::Init();
RunSpecifiedBenchmarks();
645 testdata/cp.html
View
@@ -1,645 +0,0 @@
-<head>
-<title>Compression Pointers</title>
-<META HTTP-EQUIV="Keywords" CONTENT="compression, compression, compression">
-</head>
-<body>
-<BODY BGCOLOR=#FFFFFF>
-
-<center>
-<H1> Compression Pointers </h1>
-
-
-
-</center>
-
-<p>
-<a href="#Resources">Compression resources</a>, <a href="#Conferences">conferences</a>, and some <a href="#Research">research
-groups and companies</a>, are listed towards the end of this page. <p>
-
-Use this <a href="form.html">handy form</a> to add something to this page, or to simply say you liked this page<i> <img src="new.gif">
-<p>
-
-</i>
-
-
-
-<p>
-
-<h2>What's New?</h2><p>
-
-
-<a href="http://www.teaser.fr/~jlgailly/">Jean-loup Gailly</a> -- <i>Mr. gzip,
-PNG, CCR (1996-06-10)</i>
-<img src="new.gif">
-<br>
-
-<a href="http://www.creative.net/~tristan/MPEG">MPEG Pointers and Resources</a>
-<img src="new.gif">
-<br>
-
-<a href="http://www-isl.stanford.edu/~gray/">Robert M. Gray</a> -- <i>Signal
-compression, VQ, image quality evaluation (1996-04-22)</i><img src="new.gif">
-<br>
-
-<a href="http://www-isl.stanford.edu/~gray/compression.html">Compression and
-Classification Group</a> -- <i>ISL,EE,Stanford (1996-04-22)</i><img src="new.gif">
-<br>
-
-<a href="http://www-isl.stanford.edu/~gray/iii.html">Signal Processing
-and the International Information Infrastructure</a> -- <i>Web sites (1996-04-22)</i> <img src="new.gif">
-<br>
-
-<a href="http://www.cs.sc.edu:80/~valenta/">Valenta, Vladimir</a> -- <i>(1996-04-22)</i>
-<br>
-
-
-<a href="http://patpwww.epfl.ch:80/~jordan/Subjects/Parallel/parallel.html">Jordan, Frederic</a> -- <i>Parallel Image Compression (1996-04-12)</i> <img src="new.gif">
-<br>
-<a href="http://www.garlic.com/biz/eotek">Electro-Optical Technologies, Inc.</a> -- <i>Consultant (1996-04-12)</i> <img src="new.gif">
-<br>
-<a href="http://quicktime.apple.com">quicktime.apple.com</a> -- <i>Quicktime site (1996-03-11)</i> <img src="new.gif">
-<br>
-<a href="http://www.chips.ibm.com/products/aldc/index.html">IBM Hardware</a> -- <i>Compression chips (1996-03-11)</i> <img src="new.gif">
-<br>
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-<!---
-
-<img src="new.gif"><i>Do you have, or know of, a research position where an image compression guru with a Ph.D. would feel at
-home? Let me know if you do...<a
-href="mailto:singlis@cs.waikato.ac.nz"><b>send me some email!</b></a>, or <a
-href="http://www.cs.waikato.ac.nz/~singlis/cv.html"><b>check out my
-c.v.</b>
-</a>
-</i>
-
--->
-
-<p>
-
-
-<p>
-
-
-<h2><i>People</i></h2>
-<h2>A</h2>
-
-<a href="http://quest.jpl.nasa.gov/Mark.Adler/">Adler, Mark</a> -- <i>Info-ZIP; Zip, UnZip, gzip and zlib co-author; PNG group</i>
-<p>
-
-<h2>B</h2>
-
-<a href="http://www.eese.qut.edu.au/~mbaker/">Baker, Matthew</a> -- <i>Region based video compression</i>
-<p>
-<a href="http://www.eecs.wsu.edu/~bamberg/">Bamberger, Roberto H.</a>
-<p>
-<a href="http://sutherland.eese.qut.edu.au/~dbell">Bell, Daniel</a> -- <i>Region based image compression</i>
-<p>
-<a href="http://www.cosc.canterbury.ac.nz/~tim">Bell, Tim</a> -- <i>compression, computer science for children, and computers and music.</i>
-<p>
-<a href="http://www.polytechnique.fr/poly/~bellard/">Bellard, Fabrice</a> -- <i>Author of LZEXE</I>
-<p>
-<a href="http://www.cevis.uni-bremen.de/~willy/">Berghorn, Willy </a>
-<p>
-<a href="http://www.best.com/~bhaskara">Bhaskaran, Vasudev</a> -- <i>Image and Video compression</i>
-<p>
-<a href="http://wwwvms.utexas.edu/~cbloom/index.html">Bloom, Charles</a> -- <i>text compression, LZ methods, PPM methods...</i>
-<p>
-<a href="http://www.ime.usp.br/~rbrito">Brito, Roger</a> -- <i>Arithmetic Coding, LZW, Text Compression</i>
-<p>
-<a href="http://www.engineering.usu.edu/ece/faculty/scottb.html">Budge, Scott E.</a> -- <i>medical/lossy image compression</i>
-<p>
-<a href="http://www.elec.uow.edu.au/people/staff/burnett/biography.html">Burnett, Ian</a> -- <i>Speech coding, prototype waveform techniques</I> <img src="new.gif">
-<p>
-
-
-<h2>C</h2>
-
-<a href="http://gabor.eecs.berkeley.edu/~wychan/">Chan, Christopher</a> -- <i>universal lossy source coding, adaptive VQ</i>
-<p>
-<a href="http://www.compsci.com/~chao">Chao, Hong-yang</a> -- <I>Lightning strike image compressor </i>
-<p>
-<a href="http://monet.uwaterloo.ca:80/schao/">Chao, Stewart</a>
-<p>
-<a href="http://diana.ecs.soton.ac.uk/~pjc94r/">Cherriman, Peter</a>
-<p>
-<a href="http://www-video.eecs.berkeley.edu/erl/sccheung.html">Cheung, S.C.</a> -- <i>Scalable video compression algorithms (18th Feb 1996)</i> <img src="new.gif">
-<p>
-
-<a href="http://etro.vub.ac.be/chchrist.html">Christopoulos, Charilaos</a> -- <i> Image and Video compression </i>
-<p>
-<a href="http://sipi.usc.edu/~chrysafi/">Chrysafis, Christos</a>
-<p>
-<a href="http://www.ee.gatech.edu/research/DSP/students/wchung/index.html">Chung, Wilson C.</a> -- <i>R-D image and video coding, subband/wavelet, filter banks</i>
-<p>
-<a href="mailto:aclark@hayes.com">Clark, Alan</a> -- <i>primary contact for V.42bis. Developed BTLZ variant of LZW</i>
-<p>
-<a href="http://www.cs.waikato.ac.nz/cs/Staff/jgc.html">Cleary, John</a> -- <i>PPM, PPM*, K*</i>
-<p>
-<a href="http://www.cs.brandeis.edu/dept/faculty/cohn/">Cohn, Martin</a>
-<p>
-
-<a href="http://plg.uwaterloo.ca/~gvcormac">Cormack, Gordon V.</a> -- <I>DMC</i>
-<p>
-<a href="http://www.ee.duke.edu/~cec/index.html">Cramer, Chris</a> -- <i>neural network image compression</i>
-<p>
-
-<h2>D</h2>
-<a href="http://www.cs.dartmouth.edu/~jmd/">Danskin, John</a> -- <i>Protocol compression, document compression (9th Feb 1996)</i>
-<p>
-<a href="http://www.cs.dartmouth.edu/~gdavis">Davis, Geoff</a> -- <i>wavelets, image compression, medical</i>
-<p>
-<a href="http://www-mddsp.enel.ucalgary.ca/People/adilger/">Dilger, Andreas</a> -- <i>fractal block coding of video sequences</i>
-<p>
-
-
-<h2>E</h2>
-<a href="http://info.cipic.ucdavis.edu/~estes/index.html">Estes, Robert</a>
-<p>
-
-
-
-<h2>F</h2>
-<a href="http://inls.ucsd.edu/y/Fractals/">Fisher, Yuval</a> -- <i>Fractal Image Compression</i>
-<p>
-<a href="http://www.wmin.ac.uk/~ajoec1/homepage.html">Ford, Adrian</a> -- <i>Subjective and Objective Compression quality (5th Mar 1996)</i> <img src="new.gif">
-<p>
-
-<h2>G</h2>
-<a href="http://www.teaser.fr/~jlgailly/">Jean-loup Gailly</a> --
-<i>Mr. gzip, PNG, CCR (1996-06-10)</i>
-<p>
-
-<a href="http://www.ece.ucsb.edu/faculty/gersho/default.html">Gersho, Allen</a> -- <i>Vector Quantisation, Image Compression</i>
-<p>
-<a href="http://life.anu.edu.au/ci/vol1/goertzel.html">Goertzel, Ben</a>
-<p>
-<a href="http://info.lut.ac.uk/departments/el/research/sys/compression.html">Gooch, Mark</a> -- <i>High Performance Hardware Compression (7th Feb 1996)</i>
-<p>
-<a href="http://robotics.eecs.berkeley.edu/~vkgoyal/">Goyal, Vivek</a> -- <i>overcomplete representations, adaptive transform coding, VQ</i>
-<p>
-<a href="http://www-isl.stanford.edu/~gray/">Robert M. Gray</a> -- <i>Signal
-compression, VQ, image quality evaluation</i><img src="new.gif">
-<p>
-
-<h2>H</h2>
-<a href="http://cork.informatik.uni-wuerzburg.de/mitarbeiter/ulli">Hafner, Ullrich</a> -- <i>WFA image compression</i>
-<p>
-<a href="http://www.ics.uci.edu/~dan/">Hirschberg, Dan</a> -- <i>algorithm theory, compression</i>
-<p>
-<a href="http://wolfpack.larc.nasa.gov/holland.html">Holland, Scott</a>
-<p>
-<a href="http://www.csc.uvic.ca/home/nigelh/nigelh.html">Horspool, R. Nigel</a> -- <i>text compression, ECG compression</i>
-
-<h2>I</h2>
-<a href="http://www.cs.waikato.ac.nz/~singlis">Inglis, Stuart</a> -- <i>image compression, OCR, lossy/lossless document compression</I>
-<p>
-
-<h2>J</h2>
-<a href="http://www.cs.uiowa.edu/~jones/compress/index.html">Jones, Douglas</a> -- <i>splay-tree based compression and encryption</i>
-<p>
-<a href="http://info.lut.ac.uk/departments/el/research/sys/compression.html">Jones, Simon</a> -- <i>Lossless compression, High Performance Hardware (7th Feb 1996) </i>
-<p>
-<a href="http://patpwww.epfl.ch:80/~jordan/Subjects/Parallel/parallel.html">Jordan, Frederic</a> -- <i>Parallel Image Compression (12th April 1996)</i> <img src="new.gif">
-<p>
-<a href="mailto:robjung@world.std.com">Jung, Robert K.</a> -- <i><a href="http://www.info.fundp.ac.be/~fta/arj.html">ARJ</a> </i>
-<p>
-<a href="http://www.cs.tu-berlin.de/~jutta/toast.html">jutta</a>
-<p>
-
-
-
-<h2>K</h2>
-<a href="http://links.uwaterloo.ca">Kominek, John</a> -- <I>Fractal and spline based compression</i>
-<p>
-<a href="http://info.lut.ac.uk/departments/el/research/sys/elmk3.html">Kjelso, Morten</a> -- <i>Main memory compression, High performance hardware</i>
-<p>
-<a href="http://wwwcip.informatik.uni-erlangen.de/user/mskuhn">Kuhn, Markus</a> -- <i>JBIG implementation</i> <img src="new.gif">
-<p>
-<a href="http://sipi.usc.edu/faculty/Kuo.html">Kuo, C.-C. Jay</a>
-<p>
-<a href="http://kaarna.cc.jyu.fi:80/~kuru/">Kuru, Esa</a>
-<p>
-<a href="http://www.prz.tu-berlin.de:80/~teo/">Kyfonidis, Theodoros</a>
-<p>
-
-<h2>L</h2>
-<a href="http://www.icsi.berkeley.edu/~lampart/">Lamparter, Bernd</a>
-<p>
-<a href="http://www.cse.ucsc.edu/cgi-bin/faculty?langdon">Langdon, Glen</a>
-<p>
-<a href="http://www.dna.lth.se/~jesper/">Larsson, Jesper</a> -- <i>algorithms and data structures, text compression</i> <img src="new.gif">
-<p>
-
-
-<h2>M</h2>
-<a href="http://glimpse.cs.arizona.edu:1994/udi.html">Manber, Udi</a>
-<p>
-<a href="http://www.elec.rma.ac.be/~jma/compression.html">Mangen, Jean-Michel</a> -- <i>image compression, wavelets, satellite</I>
-<p>
-<a href="http://www.cs.mu.oz.au/~alistair">Moffat, Alistair</a> -- <i>text and index compression, coding methods, information retrieval, document databases</i>
-<p>
-<a href="http://deskfish.cs.titech.ac.jp/squish/squish_index.html">Montgomery, Christopher</a> -- <i>OggSquish, audio compression</i>
-<p>
-<a href="mailto:umueller@amiga.physik.unizh.ch">Mueller, Urban Dominik</a> -- <i>XPK system</I>
-<p>
-<h2>N</h2>
-<a href="ftp://ai.toronto.edu/pub/radford/www/index.html">Neal, Radford</a>
-<p>
-<a href="http://www.cs.waikato.ac.nz/~cgn">Nevill-Manning, Craig</a>
-<p>
-<a href="http://phoenix.bath.ac.uk/jez/jez.html">Nicholls, Jeremy</a>
-<p>
-<a href="http://www.acti.com/matt/">Noah, Matt</a> -- <i>speech, ATC, ACELP and IMBE</i>
-<p>
-<a href="http://www.eecs.wsu.edu/~vnuri/">Nuri, Veyis</a> -- <i>wavelets, data compression, DSP (speech & image)</I>
-
-<h2>O</h2>
-<a href="ftp://replicant.csci.unt.edu/pub/oleg/README.html">Oleg</a> -- <i>Lots of code</i>
-<p>
-<a href="http://sipi.usc.edu/faculty/Ortega/Ortega.html">Ortega, Antonio</a> -- <i>Video compression, Packet video, Adaptive quantization</i>
-<p>
-<a href="http://www.cs.dartmouth.edu:80/~cowen/">Owen, Charles</a>
-<p>
-
-<h2>P</h2>
-<a
-href="http://www-mddsp.enel.ucalgary.ca/People/provine/compression.html">Provine,
-Joseph</a> -- <i>Model based coding</i>
-<p>
-
-<h2>R</h2>
-<a href="http://monet.uwaterloo.ca/~john/btpc.html">Robinson, John</a> -- <i>Binary Tree Predictive Coding</i>
-<p>
-<a href="http://quest.jpl.nasa.gov/Info-ZIP/people/greg/">Roelofs, Greg</a> -- <i>Info-ZIP; primary UnZip author; PNG group</i>
-<p>
-<a href="http://www.ece.ucsb.edu/faculty/rose/default.html">Rose, Kenneth</a>
-<p>
-<a href="http://www.nettuno.it/fiera/telvox/telvox.htm">Russo, Roberto Maria</a> -- <I>Multiplatform Data Compression</i>
-<p>
-
-<h2>S</h2>
-<a href="http://harc.edu:80/~schmidt/">Schmidt, Bill</a>
-<p>
-<a href="http://koa.ifa.hawaii.edu/~shaw/shaw.html">Shaw, Sandy C.</a>
-<p>
-<a href="http://www.comm.toronto.edu/~kamran/kamran.html">Sharifi, Kamran</a> -- <i>Video over ATM networks</i>
-<p>
-<a href="http://calypso.unl.edu/~sunil/">Shende, Sunil M.</a>
-<p>
-<a href="http://viewprint2.berkeley.edu/KleinLab/Amnon/Amnon.html">Silverstein, D. Amnon</a>
-<p>
-<a href="http://www-plateau.cs.berkeley.edu/people/smoot">Smoot, Steve</a> -- <i>MPEG</i>
-<p>
-<a href="http://www.cs.brandeis.edu/dept/faculty/storer/">Storer, James A.</a>
-<p>
-<a href="http://rice.ecs.soton.ac.uk/index.html">Streit, Jurgen</a> -- <i>Low bitrate coding</i>
-<p>
-<a href="http://www_nt.e-technik.uni-rostock.de/~ts/index.html">Strutz, Tilo</a> -- <i>Wavelet image compressor, demonstration execs</i> <img src="new.gif">
-<p>
-
-
-<h2>T</h2>
-<a href="http://zaphod.csci.unt.edu:80/~srt/">Tate, Steve</a>
-<p>
-<a href="http://www.cs.waikato.ac.nz/~wjt">Teahan, Bill</a> -- <i>PPM, PPM*, master of the Calgary Corpus</i>
-<p>
-<a href="http://lightning.eee.strath.ac.uk/~duncan/">Thomson, Duncan</a> -- <i>ECG compression with ANNs</i>
-<p>
-<a href="http://outside.gsfc.nasa.gov/GRSS/">Tilton, James C.</a>
-<p>
-<a href="http://www.uni-karlsruhe.de/~un55/">Tomczyk, Marek</a>
-<p>
-<a href="http://munkora.cs.mu.oz.au/~aht">Turpin, Andrew</a> -- <i>Prefix Codes</i>
-<p>
-
-<h2>V</h2>
-<a href="http://gabor.eecs.berkeley.edu/~martin/">Vetterli, Martin</a> -- <i>wavelets, subband coding, video compression, computational complexity</i>
-<p>
-<a href="http://www.cs.duke.edu/~jsv/HomePage.html">Vitter, Jeff</a>
-<p>
-<a href="http://www.eee.strath.ac.uk/~stathis/home.html">Voukelatos, Stathis</a>
-<p>
-<a href="http://www.xs4all.nl/~aipnl">de Vries, Nico</a> -- <i>AIP-NL, UltraCompressor II development</i> <img src="new.gif">
-<p>
-
-<h2>W</h2>
-
-<a href="http://ipcl.ee.queensu.ca/wareham/wareham.html">Wareham, Paul</a> -- <i>Region-oriented video coding</I>
-<p>
-<a href="mailto://Wegs59@aol.com">Wegener, Al</a> -- <i>DSP, lossless audio compression and AC-2</i>
-<p>
-<a href="http://www-dsp.rice.edu:80/~weid/">Wei, Dong</a> -- <i>Wavelet compression</i>
-<p>
-<a href="http://www.rocksoft.com/~ross">Williams, Ross</a>
-<p>
-<a href="http://www.cs.waikato.ac.nz/~ihw">Witten, Ian</a> -- <i>PPM, mg, arithmetic coding</i>
-<p>
-<a href="http://www.sees.bangor.ac.uk/~gerry/sp_summary.html">Wolff, Gerry</a>
-<p>
-<a href="http://sipi.usc.edu/~wwoo/">Woo, Woon-Tack</a> -- <i>stereo image and video compression (12th Feb 1996)</i>
-<p>
-<a href="http://www.csd.uwo.ca/faculty/wu/">Wu, Xiaolin</a> -- <i>CALIC</i>
-<P>
-
-
-<h2>Z</h2>
-<a href="http://www.cs.rmit.edu.au/~jz">Zobel, Justin</a> -- <i>index compression, database compression</i>
-<p>
-
-<br>
-<br>
-
-
-<a name="Conferences">
-
-<h2><i>Conferences</i></h2>
-<a href="http://www.cs.brandeis.edu:80/~dcc/index.html"><b>Data Compression Conference (DCC), Snowbird, Utah, USA.</b></a><br>
-<p>
-
-<a name="Resources">
-
-<h2><i>Resources</i></h2>
-<dt>
-<i><b>Where to get answers...</b></i>
-<dd>
-<dd><a href="http://info.itu.ch/">International Telecommunication Union (ITU)</a> -- <i>standards documents (CCITT)</i>
-<br>
-<dd><a href="http://www.cis.ohio-state.edu/hypertext/faq/usenet/compression-faq/top.html">comp.compression -- Frequently Asked Questions</a> -- <i>Come here 1st!</i>
-<dd>
-<a href="http://www.cis.ohio-state.edu/hypertext/faq/usenet/jpeg-faq/faq.html">JPEG - Frequently Asked Questions</a>
-<dd>
-<a href="http://www.crs4.it/HTML/LUIGI/MPEG/mpegfaq.html">MPEG - Frequently Asked Questions</a>
-<dd>
-<a href="ftp://rtfm.mit.edu:/pub/usenet/news.answers/standards-faq">Standards - Frequenty Asked Questions</a>
-<dd>
-<a href="news:comp.compression.research">usenet newsgroup: comp.compression.research</a>
-<br>
-<dd>
-<a href="http://www.mi.net/act/act.html">Archive Compression Test</a> -- <i>Summary of all archivers</i>
-<br>
-<dd>
-<a href="http://dip1.ee.uct.ac.za/fractal.bib.html">Fractal compression bibliography</a> -- <i>bibtex file</i>
-<br>
-<dd>
-<a href="http://goethe.ira.uka.de/edu/proseminar.html">LZ/Complexity Seminars</a> -- <i>(in German)</i>
-<br>
-<dd>
-<a href="http://www.cs.waikato.ac.nz/~singlis/ratios.html">Comparitive Compression Ratios</a> -- <i>Sofar...B&W, Gray Scale, Text Compression</i>
-<br>
-<dd>
-<a href="http://www.cs.ucl.ac.uk/mice/mpeggloss.html">VCN (Video, Compression, Networking)
-Glossary</a>
-<br>
-<dd>
-<a href="http://www.cs.waikato.ac.nz/~nzdl/">Digital Library -- search for compression</a>
-<br>
-</dt>
-
-<br>
-<dt>
-<i><b>Reports/Ph.D. Theses</b></i>
-<dd>
-<a href="http://www.cs.brown.edu/publications/techreports/reports/CS-93-28.html">Paul Howards Ph.D. thesis</a><br>
-<dd><i>The Design and Analysis of Efficient Lossless Data Compression Systems</i>
-<br>
-</dt>
-
-
-
-<br>
-<dt>
-<i><b>Source code</b></i>
-<dd>
-<a href="ftp://munnari.oz.au/pub/arith_coder">Arithmetic coding routines</a>
-<dd><i>&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp(from Moffat, Neal and Witten, Proc. DCC, April 1995)</i>
-<br>
-<dd>
-<a href="ftp://ftp.cpsc.ucalgary.ca/pub/projects/ar.cod">CACM Arithmetic coding package</a>
-<dd><i>&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp(Witten, Neal and Cleary, CACM 30:520-541, June 1987)</i>
-<br>
-<dd>
-<a href="ftp://nic.funet.fi/pub/graphics/misc/test-images/jbig.tar.gz">JBIG Source code</a> <i>Includes a Q-coder</i>
-<br>
-<dd><a href="http://wwwcip.informatik.uni-erlangen.de/user/mskuhn">Markus Kuhn's JBIG implementation</a>
-<br>
-<dd>
-<a href="ftp://media-lab.media.mit.edu/pub/k-arith-code/">k-arithmetic coder</a>
-<br>
-<dd>
-<a href="ftp://garbo.uwasa.fi/pc/programming/lds_11.zip">Lossless Data Compression toolkit 1.1</a>
-<br>
-<dd>
-<a href="http://www.cs.uiowa.edu/~jones/compress/index.html">Splay Trees Code</a> -- <i>by Douglas W. Jones</i>
-<br>
-<dd><a href="ftp://ftp.cl.cam.ac.uk/users/djw3">Block compression code</a> -- <i>Excellent text compressor</i>
-<br>
-
-</dt>
-
-<br>
-<dt>
-<i><b>Test Files</b></i>
-<dd>
-<a href="ftp://nic.funet.fi/pub/graphics/misc/test-images/">Test Images</a> -- <i>CCITT images (pbm), Lena etc. (Sun raster)</i>
-<br>
-<dd>
-Stockholm test images -- <i>You have to buy a CDROM, thats all I know! :-(</i>
-<dd>&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp mail me at <a href="mailto:singlis@cs.waikato.ac.nz">singlis@cs.waikato.ac.nz</a> if you know anything about them.
-<br>
-<dd>
-<a href="ftp://ftp.cpsc.ucalgary.ca/pub/projects/text.compression.corpus/">Calgary Text Compression Corpus</a><i> (Text Compression, Bell, Cleary and Witten, 1990)</i>
-<br>
-</dt>
-
-<br>
-
-<a name="Research">
-
-
-<h2><i>Research Projects, Standards & Companies</i></h2>
-
-<p>
-<dt>
-<i><b>Research/Free software Groups</b></i>
-<dd><a href="http://www-isl.stanford.edu/~gray/compression.html">Compression and
-Classification Group</a> -- <i>ISL,EE,Stanford (22th April 1996)</i><img src="new.gif">
-
-<dd><a href="http://www.bonzi.com">Voice email</a> -- <i>Lossless audio compression (18th Feb 1996)</i>
-
-
-<dd><a href="http://www.garlic.com/biz/eotek">Electro-Optical Technologies, Inc.</a> -- <i>Consultant (12th April 1996)</i> <img src="new.gif">
-
-<dd><a href="http://quicktime.apple.com">quicktime.apple.com</a> -- <i>Quicktime site (11th Mar 1996)</i>
-
-<dd><a href="http://www.atinternet.fr/image/">IMAGE etc.</a> -- <i>Commercial image compression software (Fractals/Wavelets) (29th Feb 1996)</i> <img src="new.gif">
-
-<dd><a href="http://re7alpha.epm.ornl.gov/CcmDemo/">NCAR CCM Compression and Visualisation</a> -- <i>(18th Feb 1996)</i> <img src="new.gif">
-<br>
-<dd><a href="http://ipl.rpi.edu/SPIHT/">SPIHT</a> -- <I>Wavelet Natural Image Compressor (12th Feb 1996)</i> <img src="new.gif">
-<br>
-<dd><a href="http://info.lut.ac.uk/departments/el/research/sys/compression.html">Real-Time Lossless Compression Systems</a> -- <i>Loughborough University</i> <img src="new.gif">
-<br>
-<dd><a href="http://saigon.ece.wisc.edu/~waveweb/QMF.html">Web site for Multirate Signal Processing</a> -- <i>University of Wisconsin, Madison</i>
-<br>
-<dd><a href="http://quest.jpl.nasa.gov/Info-ZIP/">Info-ZIP</a> -- <i>free,
-portable Zip and UnZip utilities</i>
-<br>
-<dd><a href="http://cesdis.gsfc.nasa.gov/">CEDIS</a> -- <i>NASA, Maryland, Image/data compression</i>
-<br>
-<dd><a href="http://www.c3.lanl.gov/~brislawn/ftp.html">CIC-3 Image Compression</a> -- <i>FBI Fingerprints</i>
-<br>
-<dd><a href="http://rainbow.ece.ucsb.edu/scl.html">Signal Compression Lab at UCSB</a>
-<br>
-<dd><a href="http://isdl.ee.washington.edu/COMPRESSION/homepage.html">University of
-Washington compression lab</a> -- <I>VQ, Wavelets, Shlomo</i>
-<br>
-</dt>
-
-
-<p>
-<dt>
-<i><b>Snippets</b></i>
-<dd><a href="http://www-isl.stanford.edu/~gray/iii.html">Signal Processing
-and the International Information Infrastructure</a> -- <i>Web sites (22th April 1996)</i> <img src="new.gif">
-<dd><a href="http://www.scu.edu.au/ausweb95/papers/management/vanzyl/">Increasing Web bandwith</a> -- <i>comparing GIF, JPEG, Fractal compression</i>
-<br>
-<dd><a href="http://www.yahoo.com/Art/Computer_Generated/Fractals">Fractal Links on Yahoo</a>
-<br>
-<dd><a href="http://www.ddj.com/old/ddjw001d.htm">Digital Speech Compression GSM 06.10 RPE-LTP</a> -- <i>DDJ</i>
-<br>
-<dd><a href="http://www.crc.ricoh.com/CREW/">CREW</a> -- <i>Continuous tone loss(y/less) wavelet compression</i>
-<br>
-<dd><a href="ftp://ftp.csd.uwo.ca/pub/from_wu/">CALIC -- Context-based adaptive lossless image compressor</a>
-<br>
-<dd><a href="http://www-plateau.cs.berkeley.edu/mpeg/index.html">Berkeley MPEG</a> -- <i>MPEG tools</i>
-<br>
-<dd><a href="http://www.creative.net/~tristan/MPEG">MPEG Pointers and Resources</a>
-<BR>
-</dt>
-
-
-
-
-
-<p>
-<dt>
-<i><b>Wavelets</b></i>
-<dd>
-<a href="http://www.cis.upenn.edu/~eero/epic.html">
- EPIC (Efficient Pyramid Image Coder)</a> -- <i>by Eero Simoncelli</i>
-<br>
-<dd>
-<a href="mailto://72234.2617@compuserve.com">Tucker, Michael</a> -- <i>FASTWAVE, audio/image compression</i>
-<br>
-<dd>
-<a href="http://www.harc.edu">HARC</a> -- <i>Lossy Wavelet Compression technology</i>
-<br>
-<dd>
-<a href="http://gabor.eecs.berkeley.edu/">UC Berkeley Wavelet Group</a>
-<br>
-<dd>
-<a href="http://www.c3.lanl.gov/~cjhamil/Wavelets/main.html">Khoros Wavetlet and Compression Toolbox</a>
-<br>
-<dd>
-<a href="http://arabigo.math.scarolina.edu:80/~wavelet/">The Wavelet Digest</a>
-<br>
-<dd>
-<a href="http://jazz.rice.edu/publications.html">Rice DSP Publications Archive</a> -- <i>Wavelets, Time Frequency/Scale</i>
-<br>
-</dt>
-
-<p>
-<dt>
-<i><b>Fractals</b></i>
-<dd>
-<a href="http://inls.ucsd.edu/y/Fractals/">Fractal Image Compression</a> -- <i>Software, Pointers, Conferences</i>
-<br>
-<dd>
-<a href="http://www.fractal.com/">Fractal Design Corporation</a>
-<br>
-<dd>
-<a href="http://www.webcom.com/~verrando/university/ifs.html">New Fractal Image Compression program</a>
-<br>
-</dt>
-
-<p>
-<dt>
-<i><b>Companies</b></i>
-<dd><a href="http://www.chips.ibm.com/products/aldc/index.html">IBM Hardware</a> -- <i>Compression chips (11th Mar 1996)</i> <img src="new.gif">
-
-<dd><a href="http://www.summus.com">Summus Wavelet Technology</a> -- <i>Wavelet image and video compressors (11th Mar 1996)</i> <img src="new.gif">
-<dd><a href="http://www.mitsubishi.co.jp/jp/fractal/fractal.html">Fractal Image Compression</a> -- <I>Mitsubishi (18th Feb 1996)</i> <img src="new.gif">
-
-<dd><a href="http://www.terran-int.com/">Terran Interactive</a> -- <i>Video compression for the Mac (1st Feb 1996)</i> <img src="new.gif">
-<br>
-
-<dd><a href="http://darvision.kaist.ac.kr/dvmpeg.html">DV Mpeg</a> -- <i>Windows drivers (18th Feb 1996)</I> <img src="new.gif">
-<br>
-
-<dd><a href="http://www.crawford.com/cs/welcome.html">Crawford Compression Services</a> -- <i>MPEG post-production (18th Feb 1996)</i> <img src="new.gif">
-<br>
-
-<dd><a href="http://www.shore.net/~ict">Intelligent Compression Technologies</a> <i>(18th Feb 1996)</i> <img src="new.gif">
-<br>
-
-<dd><a href="http://www.jpg.com">Pegasus Imaging</a> -- <i>Compression software/dev. kits (12th Feb 1996)</i> <img src="new.gif">
-<br>
-<dd><a href="http://www.aladdinsys.com/">Aladdin Systems</a> -- <i>StuffIt compression products</i>
-<br>
-<dd><a href="http://www.stac.com/">Stac Electronics</a>
-<br>
-<dd><a href="http://www.ccinet.ab.ca/dcp.html">DCP Research</a> -- <i>Hardware solutions</i>
-<br>
-<dd><a href="http://www.aware.com/product_info/compression_overview.html">Aware Inc.</a> -- <i>Specialised compression company</i>
-<br>
-<dd><a href="http://www.inria.fr/rodeo/ivs.html">IVS - INRIA Videoconferencing System</a>
-<br>
-<dd><a href="http://sp25.ianus.cineca.it/telvox/telvoxen.htm">Telvox Teleinformatica</a> -- <i>Multiplatform Data Compressor</i>
-<br>
-<dd><a href="http://www.compression.com">Compression Technologies,
-Inc</a> -- <i>Canada</i>
-<br>
-<dd><a href="http://www.optivision.com/">Optivision</a> -- <i>MPEG</i>
-<br>
-<dd><a href="http://www.pkware.com/">PKWARE</a> -- <i>Makers of PKZIP</i>
-<br>
-<dd><a href="http://www.infoanalytic.com/tayson/index.html">Multimedia Imaging Services</a>
-<br>
-</dt>
-
-<p>
-<dt>
-<i><b>Audio compression</b></i>
-<dd><a href="http://www.iis.fhg.de/departs/amm/index.html"> Fraunhofer Institut für Integrierte Schaltungen</a>
-<br>
-<dd><a href="http://svr-www.eng.cam.ac.uk/~ajr/speechCoding.html">Shorten</a>
-<br>
-<dd><a href="http://www.ddj.com/old/ddjw001d.htm">digital speech compression</a>
-<br>
-<dd><a href="http://www.cs.tu-berlin.de/~phade/audiowww.html">Audio compression references</a>
-<br>
-<dd><a href="http://www.fourmilab.ch/speakfree/windows/doc/compress.html">SpeakFreely - compression</a>
-<br>
-<dd><a href="http://www.fourmilab.ch/speakfree/windows/speak_freely.html#contents">SpeakFreely - Contents</a>
-<br>
431 testdata/fields.c
View
@@ -1,431 +0,0 @@
-#ifndef lint
-static char Rcs_Id[] =
- "$Id: fields.c,v 1.7 1994/01/06 05:26:37 geoff Exp $";
-#endif
-
-/*
- * $Log: fields.c,v $
- * Revision 1.7 1994/01/06 05:26:37 geoff
- * Get rid of all references to System V string routines, for portability
- * (sigh).
- *
- * Revision 1.6 1994/01/05 20:13:43 geoff
- * Add the maxf parameter
- *
- * Revision 1.5 1994/01/04 02:40:21 geoff
- * Make the increments settable (field_line_inc and field_field_inc).
- * Add support for the FLD_NOSHRINK flag.
- *
- * Revision 1.4 1993/09/27 17:48:02 geoff
- * Fix some lint complaints and some parenthesization errors.
- *
- * Revision 1.3 1993/09/09 01:11:11 geoff
- * Add a return value to fieldwrite. Add support for backquotes and for
- * unstripped backslashes.
- *
- * Revision 1.2 1993/08/26 00:02:50 geoff
- * Fix a stupid null-pointer bug
- *
- * Revision 1.1 1993/08/25 21:32:05 geoff
- * Initial revision
- *
- */
-
-#include <stdio.h>
-#include "config.h"
-#include "fields.h"
-
-field_t * fieldread P ((FILE * file, char * delims,
- int flags, int maxf));
- /* Read a line with fields from a file */
-field_t * fieldmake P ((char * line, int allocated, char * delims,
- int flags, int maxf));
- /* Make a field structure from a line */
-static field_t * fieldparse P ((field_t * fieldp, char * line, char * delims,
- int flags, int maxf));
- /* Parse the fields in a line */
-static int fieldbackch P ((char * str, char ** out, int strip));
- /* Process backslash sequences */
-int fieldwrite P ((FILE * file, field_t * fieldp, int delim));
- /* Write a line with fields to a file */
-void fieldfree P ((field_t * fieldp));
- /* Free a field returned by fieldread */
-
-unsigned int field_field_inc = 20; /* Increment to increase # fields by */
-unsigned int field_line_inc = 512; /* Incr to increase line length by */
-
-#ifndef USG
-#define strchr index
-#endif /* USG */
-
-extern void free ();
-extern char * malloc ();
-extern char * realloc ();
-extern char * strchr ();
-extern int strlen ();
-
-/*
- * Read one line of the given file into a buffer, break it up into
- * fields, and return them to the caller. The field_t structure
- * returned must eventually be freed with fieldfree.
- */
-field_t * fieldread (file, delims, flags, maxf)
- FILE * file; /* File to read lines from */
- char * delims; /* Characters to use for field delimiters */
- int flags; /* Option flags; see fields.h */
- int maxf; /* Maximum number of fields to parse */
- {
- register char * linebuf; /* Buffer to hold the line read in */
- int linemax; /* Maximum line buffer size */
- int linesize; /* Current line buffer size */
-
- linebuf = (char *) malloc (field_line_inc);
- if (linebuf == NULL)
- return NULL;
- linemax = field_line_inc;
- linesize = 0;
- /*
- * Read in the line.
- */
- while (fgets (&linebuf[linesize], linemax - linesize, file)
- != NULL)
- {
- linesize += strlen (&linebuf[linesize]);
- if (linebuf[linesize - 1] == '\n')
- break;
- else
- {
- linemax += field_line_inc;
- linebuf = (char *) realloc (linebuf, linemax);
- if (linebuf == NULL)
- return NULL;
- }
- }
- if (linesize == 0)
- {
- free (linebuf);
- return NULL;
- }
- return fieldmake (linebuf, 1, delims, flags, maxf);
- }
-
-field_t * fieldmake (line, allocated, delims, flags, maxf)
- char * line; /* Line to make into a field structure */
- int allocated; /* NZ if line allocated with malloc */
- char * delims; /* Characters to use for field delimiters */
- int flags; /* Option flags; see fields.h */
- int maxf; /* Maximum number of fields to parse */
- {
- register field_t * fieldp; /* Structure describing the fields */
- int linesize; /* Current line buffer size */
-
- fieldp = (field_t *) malloc (sizeof (field_t));
- if (fieldp == NULL)
- return NULL;
- fieldp->nfields = 0;
- fieldp->linebuf = allocated ? line : NULL;
- fieldp->fields = NULL;
- fieldp->hadnl = 0;
- linesize = strlen (line);
- if (line[linesize - 1] == '\n')
- {
- line[--linesize] = '\0';
- fieldp->hadnl = 1;
- }
- /*
- * Shrink the line buffer if necessary.
- */
- if (allocated && (flags & FLD_NOSHRINK) == 0)
- {
- line = fieldp->linebuf =
- (char *) realloc (fieldp->linebuf, linesize + 1);
- if (fieldp->linebuf == NULL)
- {
- fieldfree (fieldp);
- return NULL;
- }
- }
- return fieldparse (fieldp, line, delims, flags, maxf);
- }
-
-static field_t * fieldparse (fieldp, line, delims, flags, maxf)
- register field_t * fieldp; /* Field structure to parse into */
- register char * line; /* Line to be parsed */
- char * delims; /* Characters to use for field delimiters */
- int flags; /* Option flags; see fields.h */
- int maxf; /* Maximum number of fields to parse */
- {
- int fieldmax; /* Max size of fields array */
- char * lineout; /* Where to store xlated char in line */
- char quote; /* Quote character in use */
-
- fieldp->nfields = 0;
- fieldmax =
- (maxf != 0 && maxf < field_field_inc) ? maxf + 2 : field_field_inc;
- fieldp->fields = (char **) malloc (fieldmax * sizeof (char *));
- if (fieldp->fields == NULL)
- {
- fieldfree (fieldp);
- return NULL;
- }
- if ((flags
- & (FLD_SHQUOTES | FLD_SNGLQUOTES | FLD_BACKQUOTES | FLD_DBLQUOTES))
- == FLD_SHQUOTES)
- flags |= FLD_SNGLQUOTES | FLD_BACKQUOTES | FLD_DBLQUOTES;
- while (1)
- {
- if (flags & FLD_RUNS)
- {
- while (*line != '\0' && strchr (delims, *line) != NULL)
- line++; /* Skip runs of delimiters */
- if (*line == '\0')
- break;
- }
- fieldp->fields[fieldp->nfields] = lineout = line;
- /*
- * Skip to the next delimiter. At the end of skipping, "line" will
- * point to either a delimiter or a null byte.
- */
- if (flags
- & (FLD_SHQUOTES | FLD_SNGLQUOTES | FLD_BACKQUOTES
- | FLD_DBLQUOTES | FLD_BACKSLASH))
- {
- while (*line != '\0')
- {
- if (strchr (delims, *line) != NULL)
- break;
- else if (((flags & FLD_SNGLQUOTES) && *line == '\'')
- || ((flags & FLD_BACKQUOTES) && *line == '`')
- || ((flags & FLD_DBLQUOTES) && *line == '"'))
- {
- if ((flags & FLD_SHQUOTES) == 0
- && line != fieldp->fields[fieldp->nfields])
- quote = '\0';
- else
- quote = *line;
- }
- else
- quote = '\0';
- if (quote == '\0')
- {
- if (*line == '\\' && (flags & FLD_BACKSLASH))
- {
- line++;
- if (*line == '\0')
- break;
- line += fieldbackch (line, &lineout,
- flags & FLD_STRIPQUOTES);
- }
- else
- *lineout++ = *line++;
- }
- else
- {
- /* Process quoted string */
- if ((flags & FLD_STRIPQUOTES) == 0)
- *lineout++ = quote;
- ++line;
- while (*line != '\0')
- {
- if (*line == quote)
- {
- if ((flags & FLD_STRIPQUOTES) == 0)
- *lineout++ = quote;
- line++; /* Go on past quote */
- if ((flags & FLD_SHQUOTES) == 0)
- {
- while (*line != '\0'
- && strchr (delims, *line) == NULL)
- line++; /* Skip to delimiter */
- }
- break;
- }
- else if (*line == '\\')
- {
- if (flags & FLD_BACKSLASH)
- {
- line++;
- if (*line == '\0')
- break;
- else
- line += fieldbackch (line, &lineout,
- flags & FLD_STRIPQUOTES);
- }
- else
- {
- *lineout++ = '\\';
- if (*++line == '\0')
- break;
- *lineout++ = *line;
- }
- }
- else
- *lineout++ = *line++;
- }
- }
- }
- }
- else
- {
- while (*line != '\0' && strchr (delims, *line) == NULL)
- line++; /* Skip to delimiter */
- lineout = line;
- }
- fieldp->nfields++;
- if (*line++ == '\0')
- break;
- if (maxf != 0 && fieldp->nfields > maxf)
- break;
- *lineout = '\0';
- if (fieldp->nfields >= fieldmax)
- {
- fieldmax += field_field_inc;
- fieldp->fields =
- (char **) realloc (fieldp->fields, fieldmax * sizeof (char *));
- if (fieldp->fields == NULL)
- {
- fieldfree (fieldp);
- return NULL;
- }
- }
- }
- /*
- * Shrink the field pointers and return the field structure.
- */
- if ((flags & FLD_NOSHRINK) == 0 && fieldp->nfields >= fieldmax)
- {
- fieldp->fields = (char **) realloc (fieldp->fields,
- (fieldp->nfields + 1) * sizeof (char *));
- if (fieldp->fields == NULL)
- {
- fieldfree (fieldp);
- return NULL;
- }
- }
- fieldp->fields[fieldp->nfields] = NULL;
- return fieldp;
- }
-
-static int fieldbackch (str, out, strip)
- register char * str; /* First char of backslash sequence */
- register char ** out; /* Where to store result */
- int strip; /* NZ to convert the sequence */
- {
- register int ch; /* Character being developed */
- char * origstr; /* Original value of str */
-
- if (!strip)
- {
- *(*out)++ = '\\';
- if (*str != 'x' && *str != 'X' && (*str < '0' || *str > '7'))
- {
- *(*out)++ = *str;
- return *str != '\0';
- }
- }
- switch (*str)
- {
- case '\0':
- *(*out)++ = '\0';
- return 0;
- case 'a':
- *(*out)++ = '\007';
- return 1;
- case 'b':
- *(*out)++ = '\b';
- return 1;
- case 'f':
- *(*out)++ = '\f';
- return 1;
- case 'n':
- *(*out)++ = '\n';
- return 1;
- case 'r':
- *(*out)++ = '\r';
- return 1;
- case 'v':
- *(*out)++ = '\v';
- return 1;
- case 'X':
- case 'x':
- /* Hexadecimal sequence */
- origstr = str++;
- ch = 0;
- if (*str >= '0' && *str <= '9')
- ch = *str++ - '0';
- else if (*str >= 'a' && *str <= 'f')
- ch = *str++ - 'a' + 0xa;
- else if (*str >= 'A' && *str <= 'F')
- ch = *str++ - 'A' + 0xa;
- if (*str >= '0' && *str <= '9')
- ch = (ch << 4) | (*str++ - '0');
- else if (*str >= 'a' && *str <= 'f')
- ch = (ch << 4) | (*str++ - 'a' + 0xa);
- else if (*str >= 'A' && *str <= 'F')
- ch = (ch << 4) | (*str++ - 'A' + 0xa);
- break;
- case '0':
- case '1':
- case '2':
- case '3':
- case '4':
- case '5':
- case '6':
- case '7':
- /* Octal sequence */
- origstr = str;
- ch = *str++ - '0';
- if (*str >= '0' && *str <= '7')
- ch = (ch << 3) | (*str++ - '0');
- if (*str >= '0' && *str <= '7')
- ch = (ch << 3) | (*str++ - '0');
- break;
- default:
- *(*out)++ = *str;
- return 1;
- }
- if (strip)
- {
- *(*out)++ = ch;
- return str - origstr;
- }
- else
- {
- for (ch = 0; origstr < str; ch++)
- *(*out)++ = *origstr++;
- return ch;
- }
- }
-
-int fieldwrite (file, fieldp, delim)
- FILE * file; /* File to write to */
- register field_t * fieldp; /* Field structure to write */
- int delim; /* Delimiter to place between fields */
- {
- int error; /* NZ if an error occurs */
- register int fieldno; /* Number of field being written */
-
- error = 0;
- for (fieldno = 0; fieldno < fieldp->nfields; fieldno++)
- {
- if (fieldno != 0)
- error |= putc (delim, file) == EOF;
- error |= fputs (fieldp->fields[fieldno], file) == EOF;
- }
- if (fieldp->hadnl)
- error |= putc ('\n', file) == EOF;
- return error;
- }
-
-void fieldfree (fieldp)
- register field_t * fieldp; /* Field structure to free */
- {
-
- if (fieldp == NULL)
- return;
- if (fieldp->linebuf != NULL)
- free ((char *) fieldp->linebuf);
- if (fieldp->fields != NULL)
- free ((char *) fieldp->fields);
- free ((char *) fieldp);
- }
BIN  testdata/fireworks.jpeg
View
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
94 testdata/grammar.lsp
View
@@ -1,94 +0,0 @@
-;;; -*- Mode: Lisp; Syntax: Common-Lisp; -*-
-
-(define-language
- :grammar
- '(((S $any) -> (S1 $any))
- ((S (Compound $s1 $s2)) -> (S1 $s1) (Conjunction) (S1 $s2))
-
- ((S1 (Statement $v)) -> (NP $subj) (VP $subj $tense $v))
- ((S1 (Acknowledge $a)) -> (Acknowledge $a))
- ((S1 (Command $v)) -> (VP Self present $v))
- ((S1 (Question $v)) -> (Aux $tense) (NP $subj) (VP $subj $tense $v))
- ((S1 (Question $v)) -> (Be $tense) (NP $subj) (Be-Arg $subj $tense $v))
-
- ((Be-Arg $subj $tense (Occur $tense (loc $subj $loc))) ->
- (Loc-Adjunct $tense (loc $subj $loc)))
-
- ((VP $subj $tense (Occur $tense $v)) -> (VP1 $subj $tense $v))
- ((VP $subj $tense (Occur $tense $v)) -> (Aux $tense)(VP1 $subj present $v))
-
- ((VP1 $subj $tense $v) -> (VP2 $subj $tense $v) (Adjunct? $v))
-
- ((VP2 $subj $tense ($rel $subj $loc)) ->
- (Verb/in $rel $tense))
- ((VP2 $subj $tense ($rel $subj $loc $obj)) ->
- (Verb/tr $rel $tense) (NP $obj))
- ((VP2 $subj $tense ($rel $subj $loc $obj $obj2)) ->
- (Verb/di $rel $tense) (NP $obj) (NP $obj2))
- ((VP2 $subj $tense (loc $subj $loc)) ->
- (Be $tense) (Loc-Adjunct $tense (loc $subj $loc)))
-
- ((NP $n) -> (Pronoun $n))
- ((NP $n) -> (Article) (Noun $n))
- ((NP $n) -> (Noun $n))
- ((NP ($x $y)) -> (Number $x) (Number $y))
-
- ((PP ($prep $n)) -> (Prep $prep) (NP $n))
- ((Adjunct? $v) ->)
- ((Adjunct? $v) -> (Loc-Adjunct $tense $v))
- #+Allegro ((Loc-Adjunct $tense ($rel $subj $loc @rest)) -> (PP $loc))
- #+Allegro ((Loc-Adjunct $tense ($rel $subj $loc @rest)) -> (Adjunct $loc))
- #+Lucid ((Loc-Adjunct $tense ($rel $subj $loc . $rest)) -> (PP $loc))
- #+Lucid ((Loc-Adjunct $tense ($rel $subj $loc . $rest)) -> (Adjunct $loc))
-
- )
- :lexicon
- '(
- ((Acknowledge $a) -> (yes true) (no false) (maybe unknown) (huh unparsed))
- ((Adjunct $loc) -> here there (nearby near) near left right up down)
- ((Article) -> a an the)
- ((Aux $tense) -> (will future) (did past) (do $finite))
- ((Be $tense) -> (am present) (are present) (is present) (be $finite)
- (was past) (were past))
- ((Conjunction) -> and --)
- ((Noun $n) -> gold Wumpus pit breeze stench glitter nothing)
- ((Number $n) -> 0 1 2 3 4 5 6 7 8 9)
- ((Prep $prep) -> in at to near)
- ((Pronoun $n) -> (you self) (me master) (I master))
-
- ((Verb/in $rel $tense) -> (go move $finite) (went move past)
- (move move $finite) (move move past) (shoot shoot $finite))
- ((Verb/tr $rel $tense) -> (move carry $finite) (moved carry past)
- (carry carry $finite) (carry carried past)
- (grab grab $finite) (grab grabbed past) (get grab $finite)
- (got grab past) (release release $finite) (release release past)
- (drop release $finite) (dropped release past) (shoot shoot-at $finite)
- (shot shoot-at past) (kill shoot-at $finite) (killed shoot-at past)
- (smell perceive $finite) (feel perceive $finite) (felt perceive past))
- ((Verb/di $rel $tense) -> (bring bring $finite) (brought bring past)
- (get bring $finite) (got bring past))
- ))
-
-(defparameter *sentences*
- '((I will shoot the wumpus at 4 4)
- (yes)
- (You went right -- I will go left)
- (carry the gold)
- (yes and no)
- (did you bring me the gold)
- (a breeze is here -- I am near 5 3)
- (a stench is in 3 5)
- (a pit is nearby)
- (is the wumpus near)
- (Did you go to 3 8)
- (Yes -- Nothing is there)
- (Shoot -- Shoot left)
- (Kill the wumpus -- shoot up)))
-
-(defun ss (&optional (sentences *sentences*))
- "Run some test sentences, and count how many were not parsed."
- (count-if-not
- #'(lambda (s)
- (format t "~2&>>> ~(~{~a ~}~)~%" s)
- (write (second (parse s)) :pretty t))
- *sentences*))
BIN  testdata/house.jpg
View
Deleted file not rendered
BIN  testdata/kennedy.xls
View
Binary file not shown
BIN  testdata/mapreduce-osdi-1.pdf
View
Binary file not shown
598 testdata/paper-100k.pdf
View
598 additions, 0 deletions not shown
BIN  testdata/ptt5
View
Binary file not shown
BIN  testdata/sum
View
Binary file not shown
112 testdata/xargs.1
View
@@ -1,112 +0,0 @@
-.TH XARGS 1L \" -*- nroff -*-
-.SH NAME
-xargs \- build and execute command lines from standard input
-.SH SYNOPSIS
-.B xargs
-[\-0prtx] [\-e[eof-str]] [\-i[replace-str]] [\-l[max-lines]]
-[\-n max-args] [\-s max-chars] [\-P max-procs] [\-\-null] [\-\-eof[=eof-str]]
-[\-\-replace[=replace-str]] [\-\-max-lines[=max-lines]] [\-\-interactive]
-[\-\-max-chars=max-chars] [\-\-verbose] [\-\-exit] [\-\-max-procs=max-procs]
-[\-\-max-args=max-args] [\-\-no-run-if-empty] [\-\-version] [\-\-help]
-[command [initial-arguments]]
-.SH DESCRIPTION
-This manual page
-documents the GNU version of
-.BR xargs .
-.B xargs
-reads arguments from the standard input, delimited by blanks (which can be
-protected with double or single quotes or a backslash) or newlines,
-and executes the
-.I command
-(default is /bin/echo) one or more times with any
-.I initial-arguments
-followed by arguments read from standard input. Blank lines on the
-standard input are ignored.
-.P
-.B xargs
-exits with the following status:
-.nf
-0 if it succeeds
-123 if any invocation of the command exited with status 1-125
-124 if the command exited with status 255
-125 if the command is killed by a signal
-126 if the command cannot be run
-127 if the command is not found
-1 if some other error occurred.
-.fi
-.SS OPTIONS
-.TP
-.I "\-\-null, \-0"
-Input filenames are terminated by a null character instead of by
-whitespace, and the quotes and backslash are not special (every
-character is taken literally). Disables the end of file string, which
-is treated like any other argument. Useful when arguments might
-contain white space, quote marks, or backslashes. The GNU find
-\-print0 option produces input suitable for this mode.
-.TP
-.I "\-\-eof[=eof-str], \-e[eof-str]"
-Set the end of file string to \fIeof-str\fR. If the end of file
-string occurs as a line of input, the rest of the input is ignored.
-If \fIeof-str\fR is omitted, there is no end of file string. If this
-option is not given, the end of file string defaults to "_".
-.TP
-.I "\-\-help"
-Print a summary of the options to
-.B xargs
-and exit.
-.TP
-.I "\-\-replace[=replace-str], \-i[replace-str]"
-Replace occurences of \fIreplace-str\fR in the initial arguments with
-names read from standard input.
-Also, unquoted blanks do not terminate arguments.
-If \fIreplace-str\fR is omitted, it
-defaults to "{}" (like for `find \-exec'). Implies \fI\-x\fP and
-\fI\-l 1\fP.
-.TP
-.I "\-\-max-lines[=max-lines], -l[max-lines]"
-Use at most \fImax-lines\fR nonblank input lines per command line;
-\fImax-lines\fR defaults to 1 if omitted. Trailing blanks cause an
-input line to be logically continued on the next input line. Implies
-\fI\-x\fR.
-.TP
-.I "\-\-max-args=max-args, \-n max-args"
-Use at most \fImax-args\fR arguments per command line. Fewer than
-\fImax-args\fR arguments will be used if the size (see the \-s option)
-is exceeded, unless the \-x option is given, in which case \fBxargs\fR
-will exit.
-.TP
-.I "\-\-interactive, \-p"
-Prompt the user about whether to run each command line and read a line
-from the terminal. Only run the command line if the response starts
-with `y' or `Y'. Implies \fI\-t\fR.
-.TP
-.I "\-\-no-run-if-empty, \-r"
-If the standard input does not contain any nonblanks, do not run the
-command. Normally, the command is run once even if there is no input.
-.TP
-.I "\-\-max-chars=max-chars, \-s max-chars"
-Use at most \fImax-chars\fR characters per command line, including the
-command and initial arguments and the terminating nulls at the ends of
-the argument strings. The default is as large as possible, up to 20k
-characters.
-.TP
-.I "\-\-verbose, \-t"
-Print the command line on the standard error output before executing
-it.
-.TP
-.I "\-\-version"
-Print the version number of
-.B xargs
-and exit.
-.TP
-.I "\-\-exit, \-x"
-Exit if the size (see the \fI\-s\fR option) is exceeded.
-.TP
-.I "\-\-max-procs=max-procs, \-P max-procs"
-Run up to \fImax-procs\fR processes at a time; the default is 1. If
-\fImax-procs\fR is 0, \fBxargs\fR will run as many processes as
-possible at a time. Use the \fI\-n\fR option with \fI\-P\fR;
-otherwise chances are that only one exec will be done.
-.SH "SEE ALSO"
-\fBfind\fP(1L), \fBlocate\fP(1L), \fBlocatedb\fP(5L), \fBupdatedb\fP(1)
-\fBFinding Files\fP (on-line in Info, or printed)
Please sign in to comment.
Something went wrong with that request. Please try again.