CCMx Benchmark, for MMB Executables stored in TAR/7z/RAR/QFC

[Raymond_NGhM]

CCMX 1.30a

Memory mode:4 (278 MB)

List of files:

06/04/2003 07:03 PM 2,027,520 Optimizer.exe
03/09/2005 04:07 AM 2,162,688 CoolType.dll
01/30/2007 08:26 AM 2,322,432 Converter.exe
08/04/2006 01:16 PM 2,655,272 Babylon.exe
10/20/2004 02:36 PM 3,162,185 neroAPI.dll
03/08/2005 07:23 AM 3,170,304 VersionCueUI.dll
03/22/2005 05:34 AM 3,567,616 Filter Gallery.dll
03/10/2005 08:31 PM 3,715,072 MPS.dll
08/30/2003 01:16 PM 4,026,368 GhostScript.dll
03/13/2005 01:10 PM 4,096,000 PDFL70.dll
03/08/2005 07:23 AM 4,153,344 VersionCue.dll
04/25/2003 04:16 PM 4,247,552 PMagicNT.exe
09/23/2005 11:56 PM 6,369,280 IconWorkshop.exe
05/16/2006 10:44 PM 9,609,216 AcroRd32.dll
11/02/2003 11:44 PM 12,251,136 Flash.exe
04/14/2006 08:40 AM 19,546,112 Photoshop.exe
03/22/2005 03:41 AM 19,980,288 ImageReady.exe
07/28/2006 06:54 PM 35,979,264 nero.exe
12/08/2006 04:56 AM 44,261,376 Photoshop-CS3.exe
--------------------------------------------------------
19 File(s) 178MB (187,303,025) bytes

TAR 178 MB (187,314,688 bytes)--> TAR.CCMX 38.7 MB (40,646,459 bytes)
RAR 178 MB (187,303,901 bytes)--> RAR.CCMX 38.4 MB (40,291,398 bytes)
.7Z 178 MB (187,303,622 bytes)--> .7Z.CCMX 38.4 MB (40,274,811 bytes)
QFC 178 MB (187,303,539 bytes)--> QFC.CCMX 38.7 MB (40,641,936 bytes)

I think ccmx work with combined files as solid work.

Real test with 7Zip:
Parameters:
7z/LZMA/Ultra/Dict:32MB+1Core,2 threads (Mem. usage:405MB)/WS:128/Solid

Result: 40.0 MB (42,029,906 bytes)

For 100x, I tell,
Only Max/Ultra, special for EXEcutable files & NO on any format,
& Fastest~Normal affect on other files & nothing.

Maximum=(Normal+BCJ) & Ultra=(Maximum+BCJ2)
BCJ=CALL+JMP (E8/E9)
BCJ2=Call+JMP+JCC (E8/E9/Jxx) that, JCC same as (Jump Condition Codes)
Anyway, i know this technic work only in UPX & one radical paker that
programmers call it "kkrunchy".

Look, What's new in lzma SDK

4.57 2007-12-12
-------------------------
- Speed optimizations in -++ LZMA Decoder.
- Small changes for more compatibility with some C/C++ compilers.
- .7z ANSI-C Decoder:
- now it supports BCJ and BCJ2 filters
- now it supports files larger than 4 GB.
- now it supports "Last Write Time" field for files.
- C++ code for .7z archives compressing/decompressing from 7-zip
was included to LZMA SDK.

Bulat, Maybe BCJ2 add in FreeARC ?

[toffer]

One suggestion: try ccm on a single executable. Then replace the first 2 bytes with two binary zeros and try again. If the compression is worse, than ccm can't apply its filter, since it you replaced the magic number with 00.

My filter contained an Jxx transform too, but there wasn't any compression improvement on average, so i dropped it...

[Christian]

CCM detects everything by content, therefore it doesn't need the headers (EXE, RGB24, DELTA, even image width and stuff). Just try it out. Still, if available, EXE headers are used for repositioning the address-transform - IIRC.

[Bulat Ziganshin]

1. can you stop creating new threads for each your post?
2. fa can't use bcj2 ATM because multi-streaming compression algorithms are not yet supported. it's planned, though, together with embedded MM recognition
3. bcj2 has better compression due to its multistreaming nature. there is even better approach - http://www.compression.ru/ds/disasm32.rar plus splitting executable into segments investigated once by Shelwien

[toffer]

CCM detects everything by content, therefore it doesnt need the headers (EXE, RGB24, DELTA, even image width and stuff). Just try it out. Still, if available, EXE headers are used for repositioning the address-transform - IIRC.

Ok, this is really nice. But how do you do that? Splitting everything into blocks, apply filter and estimate entropy?

[Raymond_NGhM]

To toffer,
Change "MZ" with 0,0

Original: 2,027,520 Optimizer.exe
Normal : 561,837 Optimizer.exe.ccmx
Modified: 561,839 Optimizer-2zero.ccmx It's only 2 bytes more

You are EXCELLENT Christian M.
Double Thanks

[Intrinsic]

kkcrunchy and the awesome crinkler URL aren't really aimed at general compression(exe) and have many tricks they perform which means they aren't useful for the vast majority of stuff on this forum beyond study.

[Christian]

Thanks Raymond.

Ok, this is really nice. But how do you do that? Splitting everything into blocks, apply filter and estimate entropy?

Actually, the short answer is yes. But e.g. RGB filtering is more complicated - at least my implementation.

[nimdamsk]

Sorry for offtopic, but i am really bored trying to manage how crincler is working. Can anyone help? Yes, i've read manual. No, i haven't Visaul Studio 200x. I have Visual C++ Express 2005, WinGw, DMC. How can i make crinkler output compressed exe's from my obj's? I get only "fatal error: LNK 0: could not find symbol '__imp__MessageBoxA@16'" errors always. Need help.

[Intrinsic]

Check the link below, this provides a few solutions to that error, and if not, just post it and see what someone says. For a more direct route email the authors( authors @ crinkler.net ) and hopefully they get back to you.

URL