Pierre R. Mai's Miscellany

The personal pages of Pierre R. Mai

(Yet Another) Deflate Decompression Implementation in CL

Permalink

I have finally gotten around to separating out and cleaning up the Deflate (RFC 1951) decompression routines we have been using in-house for quite some time, and released them under an X-style free license (see links below), for those who need something with a less restrictive license or better performance than is currently freely available1.

The library supports decompression of pure deflate streams, zlib-style (RFC 1950) and gzip-style (RFC 1952) streams, including optional checksum checking. The code should be fully portable across all conforming ANSI Common Lisp implementations, and has been performance tuned for SBCL and CMU CL, and somewhat for Lispworks (CRC-32 checksum code). While the performance does not reach the level of zlib/gzip (by a factor of around 3 to 3.52 on my most recent tests with SBCL), mostly due to stream I/O overhead and a not very sophisticated huffman decoder, it is eminently usable.

Support for compression and ZIP-file handling are currently not included.

Enjoy.

Links


  1. Since publishing this entry, I’ve been made aware of Chipz from Nathan Froyd, which achieves comparable levels of performance and is as free as Deflate; see my newer blog entry for more information.

  2. It seems that on larger files the factor is actually nearer to 1.25-1.5, see this entry for details.