Ghost of DEC Alpha is why Windows is rubbish at file compression

Microsoft's made an interesting confession: Windows file compression is rubbish because the operating system once supported Digital Equipment Corporation's (DEC's) Alpha CPU.

Alpha was a 64-bit RISC architecture that DEC developed as the successor to its VAX platform. DEC spent much of the late 1990s touting Alpha as a step beyond anything else on the market at the time. It was mostly right: x86-64 didn't arrive until the year 2000. But that didn't stop DEC faltering and being acquired by Compaq in 1998.

But because Alpha was pretty impressive for its time, Microsoft did versions of Windows NT 4.0 and Windows Server 2000 to run on the silicon.

And that effort, writes Microsoftie Raymond Chen, is why Windows file compression remains feeble.

Chen says Windows' compression algorithms are chosen to ensure they don't degrade overall system performance no matter what box Windows runs on.

In the best possible world, Chen says you'd design an OS to “use compression algorithm X if the system is slower than K megahertz, but algorithm Y if the system is faster". Which is a fine way to match compression to a machine's capabilities, but a lousy way to make data portable because if a system only has access to algorithm Y, data created on an algorithm-X-using machine won't be readable. Which could make it impossible to move drives between machines.

“In other words, you cannot change the compression algorithm depending on what machine you are running on … every system has to be able to decompress every compression algorithm.”

Chen says one of his “now-retired colleagues worked on real-time compression, and he told me that the Alpha AXP processor was very weak on bit-twiddling instructions. For the algorithm that was ultimately chosen, the smallest unit of encoding in the compressed stream was the nibble; anything smaller would slow things down by too much. This severely hampers your ability to get good compression ratios.”

That problem was coded into Windows' compression algorithms and appears to persist to this day, which is why Windows file compression remains feeble.

So why doesn't Microsoft fix this? Chen also points out that “... many (most?) popular file formats are already compressed, so file system compression wouldn't accomplish anything anyway.”

But if nothing else it's nice to know that DEC Alpha lives on, albeit as an irritant, deep within the bowels of Windows. ®