A new patch release 1.0.2 of the SHA-3 library has been released, which fixes an important bug in the generation of message digests, where multiple calls to sha3-update with partially filled buffers could lead to input data being lost and therefore incorrect and colliding message digests being generated.

All users of SHA-3 should upgrade to the new release to avoid this issue. Uses of the library that only called sha3-update once for each message digest, as well as all uses of the high-level entry points were not affected by this issue, but should still upgrade as a precaution.

Thanks to Orivej Desh for reporting this issue.

The release is available from its PMSF page, and its GitHub home.

A new patch release 1.0.1 of the SHA-3 library has been released, which switches the implementation choice for 64bit versions of LispWorks to the 32bit implementation instead of the fixnum default implementation. No other changes are contained in this release.

The release is available from its PMSF page, and its GitHub home.

I have released a new patch release 2.0.1 of the MD5 implementation for Common Lisp, which fixes a bug on 64-bit releases of LispWorks, that would lead to wrong digests being generated at all times. The new release is available from its PMSF page, and its GitHub home.

Note that MD5 has been cryptographically broken, and should therefore not be used in situations where a cryptographically secure hash function is needed (you might want to consider using my implementation of SHA-3 in those situations instead). However due to its wide historical use and its remaining suitability in non-cryptographically secure settings, this implementation is still of some interest.

As atonement for the recent MD5 release, and since I seem to perversely enjoy bumming numeric Common Lisp code, I’ve put together an implementation of the newly announced SHA-3, i.e. the algorithm formerly known as Keccak.

The package should be portable across nearly¹ all ANSI compliant CL implementations with specialized versions tuned for implementations that offer unboxed 64bit arithmetic, unboxed 32bit arithmetic and for implementations with efficient fixnum arithmetic (requiring fixnums that can represent (unsigned-byte 16)). Especially the 64 and 32bit implementations have been mostly optimized for SBCL and CMU CL. For those implementations, digests with a 1024 bit-rate (and 288 bit digest output) can be generated in between 30 (64bit SBCL) to around 100 (32bit CMU CL) cycles/byte on an i7-640M; whereas optimized C/assembler implementations reach around 12 to 50 cycles/byte on 64/32 bit Intel hardware. The reason for the discrepancy probably lies in missing peephole and dependency optimizations in the SBCL/CMU CL compiler backend, which I might look at in a future release.

The release is available from its PMSF page, and its GitHub home.

Many thanks go to the Keccak Team (Guido Bertoni, Joan Daemen, Michaël Peeters and Gilles Van Assche, cf. http://keccak.noekeon.org) for their algorithm and excellent documentation and reference implementations.

Enjoy.

1. Currently there is no version of the code that would work for CL implementations that have the smallest allowable fixnum size, which only covers (signed-byte 16), not (unsigned-byte 16) as required by the 16bit version in the package. I’m not aware of such a CL implementation, and in any case this could be easily fixed, at the cost of some code clarity and/or performance.

The MD5 implementation for Common Lisp I did a decade ago has been floating around in various tweaked versions for some time now. I’ve finally gotten around to producing a new consolidated release (release 2.0.0) of this with proper packaging, all known fixes and tweaks and implementation optimizations. The new release is still available from its PMSF page, and its new GitHub home.

Many thanks go to Christophe Rhodes, Alexey Dejneka, Nathan Froyd, Andreas Fuchs, John Desoi, Dmitriy Ivanov, and Kevin M. Rosenberg who improved, tweaked and maintained versions of MD5 over the years.

Note that MD5 has been cryptographically broken, and should therefore not be used in situations where a cryptographically secure hash function is needed. However due to its wide historical use and its remaining suitability in non-cryptographically secure settings, this implementation is still of some interest.

Recently I’ve been tentatively moving back to using Emacs, BSD, Linux for more of my stuff, for various reasons. One of the changes this brings with it, is that I can finally shutter a long neglected MovableType install and move to Octopress for blogging. The basic migration has been completed, with permalinks and atom feed preserved (the old RSS feeds have been shuttered). Expect a few changes in the coming weeks on the edges, and maybe some new content.

This is just a public service announcement for those wishing to replace the HD/SSD in a first generation MacBook Air with a Mach Xtreme Technology MX-Nano P-ATA ZIF SSD. Since information on P-ATA ZIF connectors and cables is kind of hard to come by, and the MX-Nano uses a different orientation for Pin 1 than the normal Toshiba or Samsung HDs used in the MacBook Air, there can be some confusion as to how to correctly install the MX-Nano.

So for those that plan on using the MX-Nano, this is the correct way to install the SSD (for full instructions just follow the relevant iFixit Guide): Place the SSD with the green top label on the bottom side, and the specs label on the top side into the rubber gasket as shown in this picture:

This still allows you to attach the flex cable without too much bending and the SSD itself works like a charm.

A new minor release of Deflate is available, which only contains two changes from 1.0.0:

• Due to popular demand, the Deflate system is now called "deflate" instead of "Deflate", thus allowing

1

(require :deflate)

or

1

(asdf:oos 'asdf:load-op :deflate)

to do the right thing. Note that this is an incompatible change from version 1.0.0, thus requiring changes to system definition or init files referencing "Deflate" to reference "deflate" or :deflate or #:deflate instead. Sorry for the inconvenience.

• A minor bug workaround for ECL, which also improves performance on ECL by a factor of 2 (although performance on ECL is still very much non-optimized, patches are always welcome). Note that you will need to use ECL from the current git repository, since only this version currently contains other fixes needed to let Deflate work correctly (thanks go to Juan Jose Garcia-Ripoll for the quick fixes)!

Links

• PMSF Resources Page for Deflate
• GitHub Project Page for Deflate

This book gives a very enjoyable look at Georgian and Victorian era British horse-racing and gambling, centered around the scandal of the Derby of 1844. It paints a very illuminating picture of the respective eras and the changes in gambling they brought about. Combined with the mystery surrounding the “stolen” Derby of 1844 this makes for an absorbing read. For those so inclined Amazon also offers a nice Kindle edition. Definitely recommended!

I’ve finally gotten around to slightly improving the layout of this blog, mostly thanks to the fantastic CSS Editor CSSEdit. Now that font-embedding for the web is finally gaining acceptance, I also switched to the very nice font Fontin Sans by Jos Buivenga (exljbris), which is employed through web font-embedding via @font-face, as supported by Safari 3.1 and Firefox 3.5 and later. Everyone else who doesn’t have Fontin Sans installed will still see this site in Trebuchet MS, Helvetica, Arial or whatever sans-serif font your user agent defaults to.

Please expect minor re-edits of old posts in the following days to manage some of the fallout from these design tweaks.