CCM 1.2x branch

[Christian]

Hi everyone!
This 'branch' includes a new model. Speed is lower, Memory usage is up to 210M and ratio improved a bit on most files. But don't worry, I won't increase memory usage much more.
At the moment three of CCM's models are always on. This costs speed. It's possible that I'll add a dynamic model switcher or even a commandline switch to turn off some of these. But before this there're still a couple of other things I want to look into.

Download 1.20a

If there are any problems please report them.

[encode]

Thanks!

ccm120a.zip (14 KB)
ccm120g.zip (43 KB)

[JohanDeBock]

Tanks, to me this is the most promising new archiver
Testing in progress...

[Nania Francesco]

CCM 1.20

A10.jpg > 831.337
AcroRd32.exe > 1.187.181
english.dic > 498,849
FlashMX.pdf > 3.667.884
FP.LOG > 437.964
MSO97.DLL > 1.585.763
ohs.doc > 764.687
rafale.bmp > 764.728
vcfiu.hlp > 528.831
world95.txt > 475.737
TOTAL=10.742.961
Enwik8=21.350.295
Very Good Work! Need more faster!

[LovePimple]

Awesome! Thanks Chris!

[Christian]

Thank you all for your nice comments! Special thanks to Encode for giving CCM a home!

english.dic > 498,849

Can anyone confirm this result? Maybe its from an older version?

[JohanDeBock]

Testing finished:
http://uclc.info/

[Christian]

Testing finished:
http://uclc.info/

Thanks a lot! Damn, youre fast!

[LovePimple]

Can anyone confirm this result?

I get 477,233 bytes..

[Nania Francesco]

A10.jpg > 831.337
AcroRd32.exe > 1.187.181
english.dic > 477.233
FlashMX.pdf > 3.667.884
FP.LOG > 437.964
MSO97.DLL > 1.585.763
ohs.doc > 764.687
rafale.bmp > 764.728
vcfiu.hlp > 528.831
world95.txt > 475.737
TOTAL=10721345
Enwik8=21.350.295
Enwik9=182.784.655
Excuse Me Christian !

[JohanDeBock]

Thanks a lot! Damn, youre fast!

I use a well oiled system

[Christian]

Excuse Me Christian !

Theres no need to apologize. It couldve been a bug. Therefore, thank you very much for retesting!

I get 477,233 bytes..

Phew, at first I thought that a bug crawled in.

[LovePimple]

My test results...

A10.jpg > 831,337
AcroRd32.exe > 1,187,181
english.dic > 477,233
FlashMX.pdf > 3,667,884
FP.LOG > 437,964
MSO97.DLL > 1,585,763
ohs.doc > 764,687
rafale.bmp > 764,728
vcfiu.hlp > 528,831
world95.txt > 475,737

Total = 10,721,345 bytes

Awesome!

[Christian]

ok, you guys are fooling with me. That's the results of 1.17e.

[LovePimple]

ok, you guys are fooling with me. Thats the results of 1.17e.

Just a genuine mistake! There was a copy of 1.17e in my test folder.

And I ran the test TWICE!!!

[Christian]

Hehe... and I was beginning to think that you and Nania schemed a conspiracy.

[Black_Fox]

Nice to see a new version of my favourite experimental compressor

[Matt Mahoney]

Big improvement in ccm.
http://cs.fit.edu/~mmahoney/compression/text.html

[Christian]

Thank you very much for your time, Matt!

[Christian]

Hi guys!

Here's an easter egg from me. I just had a funny idea how to speed up modeling in general. And what's even more funny, it worked. Sadly, no compression related changes.

Besides, there have been requests to reintroduce a faster profile. Thus, there are 2 versions again. CCMx is comparable to 1.20a. And CCM without x is comparable to 1.16c. The faster profile just has one model less. Of course, both versions use the latest filter framework.

Some results from my machine (on ENWIK8):

CCM 1.20a: -> 21.350.292 at 646.588 B/s
CCMx1.20c: -> 21.310.301 at 790.508 B/s (22% faster than 1.20a)
CCM 1.16c: -> 22.570.847 at 953.789 B/s
CCM 1.20c: -> 22.570.403 at 1.109.940 B/s (16% faster than 1.16c)

-CCMx is still slower than 1.16c (as expected)
-on MFC-efficiency-list CCM (without x) should take place THREE away from WinRAR!

Download 1.20c: <u>I have a bug. Use my bigger brother.</u>

Enjoy,

Christian

[Stephan Busch]

What a cool easter egg

Tests are aleady running.. Thanks heaps Christian!

[LovePimple]

Fantastic! Thanks Chris!

Mirror: Download

[Christian]

Tests are aleady running.. Thanks heaps Christian!

Im sorry to say this, but I just found a bug in the image width detection.
I was testing some real images (not rafale) and the new version was somewhat slow and ratio was lower as expected. I forgot to initialize a sum variable on which a later decision is based. Funny, that this comes up now after such a long time. Btw. this bug didnt affect stability in any way.
Damn, now I know why CCM has been slower on some images.


Anyway, heres the fixed version.

Download 1.20d http://www.mytempdir.com/1286551

[LovePimple]

Quick test...


CCM v1.20c

A10.jpg > 831,353
AcroRd32.exe > 1,193,678
english.dic > 499,565
FlashMX.pdf > 3,672,133
FP.LOG > 451,862
MSO97.DLL > 1,593,068
ohs.doc > 766,924
rafale.bmp > 773,533
vcfiu.hlp > 537,801
world95.txt > 499,359

Total = 10,819,276 bytes


CCMx v1.20c

A10.jpg > 831,334
AcroRd32.exe > 1,187,092
english.dic > 477,132
FlashMX.pdf > 3,667,586
FP.LOG > 438,006
MSO97.DLL > 1,585,445
ohs.doc > 764,650
rafale.bmp > 764,796
vcfiu.hlp > 528,574
world95.txt > 475,574

Total = 10,720,189 bytes

Will test the fixed version!

[Black_Fox]

Thanks Christian! 1.20c has become very fast!
EDIT: Your fix finally answers my concerns why the hell was "normal" version compressing better than the "extra" one

[Stephan Busch]

Yep, that' what I wanted to post now.. Anyway, CCM 1.20cx
has a marvelous speed improvement. Very well done
The results are online now.

[LovePimple]

Quick test...

CCM v1.20d

A10.jpg > 831,353
AcroRd32.exe > 1,193,678
english.dic > 499,565
FlashMX.pdf > 3,672,133
FP.LOG > 451,862
MSO97.DLL > 1,593,068
ohs.doc > 766,924
rafale.bmp > 773,533
vcfiu.hlp > 537,801
world95.txt > 499,359

Total = 10,819,276 bytes


CCMx v1.20d

A10.jpg > 831,334
AcroRd32.exe > 1,187,092
english.dic > 477,132
FlashMX.pdf > 3,667,586
FP.LOG > 438,006
MSO97.DLL > 1,585,445
ohs.doc > 764,650
rafale.bmp > 764,796
vcfiu.hlp > 528,574
world95.txt > 475,574

Total = 10,720,189 bytes

Quick compression times!

[Christian]

Thank you very much for testing!

Your fix finally answers my concerns why the hell was "normal" version compressing better than the "extra" one

Hehe... the same happend on my side. I was compressing a bitmap and thought... Whats that!?
This feature can be seen on Stephans bitmap, too.

But I dont think theres much room for further speed improvements. CCM still is a context mixing algorithm and its extremly bound by memory bandwidth.
LZ speed is out of reach. Luckily compression ratio of LZ is out of reach, too.

Im looking forward to some more results. I hope, that Werner finds time to test both versions in his next tests.

[LovePimple]

Quick test with ENWIK8...

Test Machine: AMD Sempron 2400+

CCM v1.17e - a 'multimedia' file-to-file compressor - Mar 14 2007 **
Compressed Size = 22,426,686 bytes
Compression Time = 00:05:52.453


CCM v1.20a - a 'multimedia' file-to-file compressor - Mar 21 2007 ***
Compressed Size = 21,350,295 bytes
Compression Time = 00:09:03.203


CCM v1.20d - 'multimedia' file compressor - Apr 7 2007 (easter edition)
Compressed Size = 22,570,405 bytes
Compression Time = 00:04:28.484


CCMx v1.20d - 'multimedia' file compressor - Apr 7 2007 (easter edition)
Compressed Size = 21,310,303 bytes
Compression Time = 00:07:00.156

[Christian]

Compression Time = 00:04:28.484...
00:07:00.156

Wow, the difference is much bigger than on my system (57% vs. 40%). Luckily, ENWIK8 is some kind of a worst case because it makes heavy use of the highest order model (which is missing in fast profile). Btw. how fast is 1.20a on your machine?

The last one switches the lights off... Good night!