When compressing images and especially game data, I often made the observation that some images compress worse using image compression algorithms (PNG and FLIF). Further analysing revealed that one of the cases when that happens is when the images are repetitive. Attached are two images, a real world example (source) and a generated image (random noise pattern 64*64, total size 512*512). Both PNG and FLIF perform bad here while general compression algorithms like LZMA or mixed ones (PAQ) can handle the situation.
With the second test file (64*21 pattern) we can see that the 32K deflate window size limits PNG here, because in the 64*64 case, it takes 64 lines for a repetition and 512*64*3 = 98304 is out of the window, while in the 64*21 case, the pattern can be matched.
786.486 gen.bmp (generated, 64*64 pattern)
300.366 gen.flif (-e)
253.216 gen.flif (-e -N)
230.139 gen.flif (-e -N -R5)
13.647 gen.pcf (Precomp 0.4.7, uses LZMA)
12.775 gen.paq8p (-3)
12.288 theoretical optimum (64*64*3)
786.486 gen2.bmp (generated, 64*21 pattern)
142.530 gen2.flif (-e)
50.982 gen2.flif (-e -N)
4.428 gen2.paq8p (-3)
4.032 theoretical optimum (64*12*3)
226.338 glitch.flif (-e)
203.995 glitch.flif (-e -N)
So I just wanted to share these observations. Does somebody know a (lossless) image format that doesn't have this problem? Didn't try all the others (BPG, Gralic etc.) yet.
Since I would prefer to use FLIF by default for image data in Precomp, this is a problem I'd like to solve, so a preprocessor that detects and replaces repetitive patterns in images would be the next step, I'll post further progress with this.