SLUG 1.x

[Christian]

Hi everyone!

Here's another fun-tool I wrote called SLUG. This time it's about highspeed compression. SLUG is an LZ-based compressor with an huffman entropy-stage. It has only a 128K non-sliding window because I wanted SLUG to be a flyweight on CPU *and* memory.

I used "thor" as a reference because I really like this crazy compressor. SLUG's compression speed is comparable to "thor ef". Luckily, decompression is faster and compression is better than "thor ef" almost always.

Have fun with SLUG,

Christian




Download SLUG 1.1: http://www.mytempdir.com/1309238

[encode]

Crrrazy thing! Very cool!

Mirror:
slug1.1.zip (7 KB)

[Aidanael]

Brilliant work ! Speed is really impressive.

[AiZ]

Hello Christian,

Don't want to bother you but could you generate Linux versions of your wonderful tools ? Please forgive me if you have already been asked for.

Have a nice day,


AiZ

[LovePimple]

Thanks Chris! This is probably the fastest compressor on the planet!!!

Mirror: Download

SLUG

[encode]

SLUG

I recall the Metal Slug game from SNK (Neo-Geo).

[Christian]

Everyone who wondered why the decompression command e doesnt work ... its a stupid typo, sorry. It should be a d.

So, heres a new version which displays some correct usage information: http://www.mytempdir.com/1309828

Hello Christian,

Dont want to bother you but could you generate Linux versions of your wonderful tools ?

Im not planing to release the sources anytime soon. So I dont know if I can release something under linux.

This is probably the fastest compressor on the planet!!!

quick0 is just too fast to catch. On some data "thor ef" is faster, too. I could still gain some more speed by dropping the huffman coder or adding some ASM *shivers* - but I wont.

[LovePimple]

Thanks Chris!

Mirror: Download

[Matt Mahoney]

#1 in compression speed.
http://cs.fit.edu/~mmahoney/compression/text.html# 4042

[encode]

It's awesome!

[Christian]

Thanks Matt! So, this lame slug seems to speed up on enwik.

[Bulat Ziganshin]

Matt just don't tested quicklz in fast modes (his test includes only -3 mode)

[LovePimple]

quick0 is just too fast to catch.

Why?

[Bulat Ziganshin]

because quick1 (not even quick0) uses simplest lz77 scheme known at this moment (lzrw1 + lzss encoding)

[Christian]

Why?

SLUG uses huffman which simply cant be faster than a byte or word-coder with some bit-flags. If Id replace the huffman-coder with a byte-coder, SLUG would become quite a bit faster in most situations, but this would go along with a loss in ratio (as always, depending on the data).
Additionally, I compiled quick0 with GCC and compared its performance to the provided executable by Lasse. His compiler seems to create faster code than GCC (which I use), too.

So, to sum it up in 3 words: huffman and compiler.


Anyway, SLUG was a fun-project and mainly an experiment on writing a fast semi-dynamic huffman-coder. But I dont plan to work on SLUG anymore.

[LovePimple]

because quick1 (not even quick0) uses simplest lz77 scheme known at this moment (lzrw1 + lzss encoding)

I hope that someone will eventually write a compressor thats as fast, or faster, than QuickLZ.

SLUG uses huffman which simply cant be faster than a byte or word-coder with some bit-flags. If Id replace the huffman-coder with a byte-coder, SLUG would become quite a bit faster in most situations, but this would go along with a loss in ratio (as always, depending on the data).
Additionally, I compiled quick0 with GCC and compared its performance to the provided executable by Lasse. His compiler seems to create faster code than GCC (which I use), too.

So, to sum it up in 3 words: huffman and compiler.

Thanks for the explanation!

Anyway, SLUG was a fun-project and mainly an experiment on writing a fast semi-dynamic huffman-coder. But I dont plan to work on SLUG anymore.

[Bulat Ziganshin]

hope that someone will eventually write a compressor thats as fast, or faster, than QuickLZ.

for what? just for curiosity?

[encode]

I think that any RLE-based compressor will be faster...

The catch in compression.

[LovePimple]

for what? just for curiosity?

Yes! I would like to know if such a thing is really possible.

I think that any RLE-based compressor will be faster...

Interesting!