Traffic compression (1500 byte chunks)

[blackhaz]

Dear All,

I am experimenting with real-time IP traffic compression. I wonder if you could advise on what would be the best algorithm to compress typical mixed data traffic coming in via 4-20 kB chunks? Has anybody done such a testing before?

Thank you in advance!

Max

[Matt Mahoney]

Probably something simple and fast, like LZSS or byte level LZ77. A lot of packets won't be compressible anyway, either because it is encrypted or compressed at a higher layer (JPEG, MPEG, PDF, HTTP-deflate, etc). You'll always get better compression higher up the network stack, like at the application layer.

[caveman]

Are you looking for V.42bis BTLZ or V.44 LZJH alike compression?

[blackhaz]

No, it could be just any other compression method. After researching I realized that the task is pretty much identical to anything else, so it doesn't really matter it's Internet traffic. I guess I'm going to try all the libs available and see what performs best.

[m^2]

I suggest that you try this to do the benchmarking. Use the default sector size of 1 byte and block size of whatever you want.