June 2009 Archives

In this blog post Ingvar Mattsson was wondering about the printed line resulting from evaluating:

(defun frob (x) (format t 'Frob: ~a~%' x))
(frob (defun frob (x) (format t 'New frob: ~a~%' x))

This provides an example where I think the ANSI Common Lisp standard is actually very helpful, since it often tries to go out of its way to point out things that are explicitly left undefined, instead of simply leaving them left undefined by not defining them, as many other standards (out of fear of being ambiguous) do. Quoting from the HyperSpec, Section 3.1.2.1.2.3 Function Forms:

Although the order of evaluation of the argument subforms themselves is strictly left-to-right, it is not specified whether the definition of the operator in a function form is looked up before the evaluation of the argument subforms, after the evaluation of the argument subforms, or between the evaluation of any two argument subforms if there is more than one such argument subform. For example, the following might return 23 or 24.

(defun foo (x) (+ x 3))
(defun bar () (setf (symbol-function 'foo) #'(lambda (x) (+ x 4))))
(foo (progn (bar) 20))

Thus we can see that the effect of evaluating the orginal two forms is indeed undefined (as already suspected by Ingvar Mattsson) as to which function definition is called in the second form, without going to the trouble of trying to take into account possible differences between evaluation and compilation, compile-time effects of defun, etc.

Which is my long-winded way of saying a big thank you to the people involved in creating the ANSI Common Lisp standards document (with special thanks to all involved in creating, releasing and maintaining the HyperSpec online text equivalent thereof, especially of course Kent Pitman), which is one of the nicest language standards I have had the pleasure of working with and against.

Following the “long-standing” tradition of pundits predicting what will be announced by Apple at WWDC (or other events), here are my predictions:

1. iPhone 3GS

   I agree with John Gruber’s prediction that a faster iPhone “3GS” will be announced, and of course it will have at least 16GB and 32GB models, though I sincerely hope, but do not expect, a 64GB model that would aid in replacing my 160GB iPod Classic. Well, maybe next year.

   I also think that we are going to see something for the lower end of the market, and I suspect it is not going to be the current iPhone 3G.

2. Snow Leopard

   Of course WWDC on the Mac side will be all about Snow Leopard, and I fully expect an announced ship date, as well as one or two currently unknown features for Snow Leopard, though probably nothing earth shattering, technically speaking.

   I doubt a new UI will be announced for Snow Leopard, with such a short lead time for developers, and announcing a new UI that is going to ship with 10.7 before 10.6 ships is out of the question.

   I fully expect Snow Leopard to be a full price paid update, i.e. $129.

3. No Tablet

   I’d love to be proven wrong, but I don’t think an “iTablet” is ready for release. If it should be released, it would be a high-end iPhone.

   The reason I’d love to be proven wrong is not that I’d personally want a tablet (I’d rather have either an iPhone or MacBook Air, thanks very much), but it would finally put to rest all the euphoric Apple tablet-excitement. I’ve been around since before (probably) the first time in the late 80s when tablets were all the rage (remember the GRiDPad?), and with each new tablet mania each 5-8 years, (IMHO unrealistic and unclear) expectations were never ever met by the products, and they subsequently vanished in small niche markets, never to be heard of again. An Apple tablet is either going to meet the same fate, not unlikely, or it is going to revolutionize the tablet form factor into something which people can really use in practice, not merely crave in the abstract. In either case we’ll know something more about tablets and their place in the world.

4. Cinema Displays

   I continue to expect that they’ll update the Cinema Display range sometime this year, and it is getting kind of urgent with all current products going to DisplayPort connectors, and only one matching display. However Apple has continually failed to update them, so why now? We’ll see…

So it seems the best way of finding out about lisp packages is releasing a similar package yourself ;): Right on the foot of releasing Deflate, I get to notice Chipz, a similar, but more capable package by Nathan Froyd, the release of which in early 2008 seems to have slipped my notice completely. So anyone who is interested in RFC 1951 Deflate (or even BZIP2 decompression), please have a look at Chipz as well, which is likely to better fit your needs!

Since I can’t let any benchmarking opportunity go to waste, here’s an overview of performance when decompressing a 18.6MB gzip-compressed file into 81.9 MB uncompressed form with Deflate 1.0.0, Chipz 0.7.3 and inflate.cl from Franz, version 2.6, on various CL implementations, and gzip 1.3.10:

Since CRC-32 calculations can overshadow deflate performance itself on some implementations, and since inflate.cl does not perform CRC-32 checksum calculations at all, the columns showing Deflate and Chipz without CRC-32 calculations might be more indicative of actual deflate performance.

Note that this was all done on a MacBook Air 1.8GHz Core2 Duo under Mac OS X 10.5.7, timing was done as an average of 3 runs each.

What can be learned from the results is that 64bit implementations with their bigger fixnum range help quite a bit in performing 32bit algorithms (like CRC-32) well even without extensive/effective declaration fine-tuning or special 32bit arithmetics operators.

I have finally gotten around to separating out and cleaning up the Deflate (RFC 1951) decompression routines we have been using in-house for quite some time, and released them under an X-style free license (see links below), for those who need something with a less restrictive license or better performance than is currently freely available¹.

The library supports decompression of pure deflate streams, zlib-style (RFC 1950) and gzip-style (RFC 1952) streams, including optional checksum checking. The code should be fully portable across all conforming ANSI Common Lisp implementations, and has been performance tuned for SBCL and CMU CL, and somewhat for Lispworks (CRC-32 checksum code). While the performance does not reach the level of zlib/gzip (by a factor of around 3 to 3.5² on my most recent tests with SBCL), mostly due to stream I/O overhead and a not very sophisticated huffman decoder, it is eminently usable.

Support for compression and ZIP-file handling are currently not included.

Enjoy.

Links

• PMSF Resources Page
• GitHub Project Page

1. Since publishing this entry, I’ve been made aware of Chipz from Nathan Froyd, which achieves comparable levels of performance and is as free as Deflate; see my newer blog entry for more information.

2. It seems that on larger files the factor is actually nearer to 1.25-1.5, see this entry for details.