Dictionary to Archive Size Question

[GipFace]

Hello! I was referred to here for this question because you people are very intelligent. ^_^

I am preparing an archive which contains about 3GB of BMP files, many of which are slight modifications of a single BMP (there are several of these files). However, since this archive is going to be part of a public build, I cannot risk more than an 800MB dictionary size because some of my users will only have 1GB RAM. This dictionary size is less than the archive size. Would I get better compression if I were to split the archive into six 500MB parts and forcing an 800MB dictionary? I would split them so that all the key BMP files would be grouped together with the modified BMP files, allowing for the best possible compression.

I'm looking at using CCMx (setting 6 for 786MB) as the compressor because it's 12-15% better than 7zip (ultra) and decompresses in about 20 minutes which is fine by me. FreeARC (-mx) performs consistently worse (about 3-5%) than 7zip, while PAQ is out of the question because it's not practical.

Thank you for your time!

[nanoflooder]

[offtopic]
Did you try UHARC? It should do seriously well on such data. Better than any other LZ archiver, that's for sure ^^
[/offtopic]

[schnaader]

I think you should consider some preprocessing, for example only storing the key BMP files as they are and delta-encoding the others. If they're only slightly modified, you could save a lot of compression time and get a better ratio.

[encode]

You may try PIM archiver as well. Although it has no solid mode...

[Intrinsic]

RAR does well on BMPs and at compressing lots of files which are similar as it finds the differences between them and only compresses those.
SBC is also cracking at BMP and another good choice.
Both of those will use little memory.

[m^2]

RAR does well on BMPs and at compressing lots of files which are similar as it finds the differences between them and only compresses those.
SBC is also cracking at BMP and another good choice.
Both of those will use little memory.

Probably much too small dictionary size.

[GipFace]

I didn't try UHARC because of its very slow decompression time (twice as slow as CCMx) in all the benchmark charts, but I tried it today because nanoflooder suggested it. It simply ran away with the crown. What's funny is that -m3 is actually better than -mx (by ~1%) and decompresses in a fraction of the time! Sample data was 1200 BMP files (varying 24-bit and 32-bit), 2.1gb on a Core 2 Quad 2.7ghz.

7-zip -md=27 -mfb=273 -mmc=10000 -mlc=8
300 155 160, decode time 82s

uharc -m3 -md32768 -b32768 -mm+
239 137 085, decode time 218s [2 files to get past the 2gb limit]

Thanks, nanoflooder! Under 4 minutes is definitely acceptable for a public build.