http://nishi.dreamhosters.com/u/cdm_test_1.rar

This is a coder that attempts to make use of actual tiny redundancies

left after some compression algorithms, especially bitcode-based ones.

The idea is rather simple - there're some (combinatorical) coding methods

that always compress any data (not counting the space required to store stats).

For example, a 1024-bit block with 0s count = 412 and 1s count = 612 yields redundancy of 33 bits,

which can be used to encode counts, and still save some space:1024-Log[2, Binomial[1024.,1024/2-100]] = 33.6576 bits

So cdm runs a parsing optimization algorithm that tries to split the data

into a series of small blocks which can be either stored or encoded.

The method is asymmetric, so decoding is much faster than encoding.

Encoding can be also faster with some weaker optimizer settings.

Unfortunately I wasn't able to compress any paq archives with this version,

but results for any (unknown) bitcoded formats seem to be interesting.

Also, it seems that cdm can be used for file analysis/format detection -

here we can clearly see that jpeg uses bit7-0 bit order, while deflate

uses bit0-7:

Please test it, I'd like to know the filetypes which it can compress (better than normal methods).Code:cdm = tuned to A10.jpg cdm1 = tuned to A10.jpg.bwts7-0 cdm2 = tuned to book1__kzip.raw.bwts0-7 [A10.jpg] cdm cdm1* cdm2 original size === 842468 842468 842468 7z -mx=9 compression === 842114 842114 842114 direct test === 841932 842526 842047 bytewise bwts === 838909 838926 838711 bit7-0 bitwise bwts === 820363* 817187* 818645* bit0-7 bitwise bwts === 835902 835128 835247 [book1__kzip.raw] cdm cdm1 cdm2* original size === 299437 299437 299437 7z -mx=9 compression === 299594 299594 299594 direct test === 299387 299551 299381 bytewise bwts === 299460 299687 299476 bit7-0 bitwise bwts === 299450 299657 299460 bit0-7 bitwise bwts === 299271* 299331* 299196* [decmprs8.exe] cdm cdm1 cdm2* original size === 16720 16720 16720 7z -mx=9 compression === 16929 16929 16929 direct test === 16616* 16615* 16609* bytewise bwts === 16647 16651 16643 bit7-0 bitwise bwts === 16632 16641 16630 bit0-7 bitwise bwts === 16622 16630 16621 [zik.ogg] cdm cdm1* cdm2 original size === 2652463 2652463 2652463 7z -mx=9 compression === 2622858 2622858 2622858 direct test === 2630762 2624089 2626150 bytewise bwts === 2620787 2618797 2618391 bit7-0 bitwise bwts === 2606332 2603721 2603797 bit0-7 bitwise bwts === 2584570*2580573*2581467* [cdm_test_0.rar] cdm* cdm1 cdm2 original size === 920156 920156* 920156 7z -mx=9 compression === 920346 920346 920346 direct test === 919991 920653 920100 bytewise bwts === 920062 920777 920190 bit7-0 bitwise bwts === 919907* 920593 920032* bit0-7 bitwise bwts === 919996 920704 920132 [chm.7z] cdm cdm1 cdm2* original size === 1414783 1414783 1414783 7z -mx=9 compression === 1414981 1414981 1414981 direct test === 1414065*1414445*1413792* bytewise bwts === 1414718 1415734 1414773 bit7-0 bitwise bwts === 1414694 1415719 1414757 bit0-7 bitwise bwts === 1414704 1415730 1414775