Page 1 of 2 12 LastLast
Results 1 to 30 of 44

Thread: BIM (a new lossless image compressor) is here!

  1. #1
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts

    Cool BIM (a new lossless image compressor) is here!

    http://encode.narod.ru/

    Please welcome the very first version of my new lossless image compressor!

    Also note that this version is kind of a "core" or "bare-bone" of my new image codec - BIM can compress faster with a slightly worse compression, also BIM can compress notable better at the cost of processing speed.

    Anyway, please enjoy new release!




    UPDATE:
    BIM v0.02 supports 24-bit BMP, TGA and PNM/PPM images and has a higher compression

  2. #2
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,134
    Thanks
    179
    Thanked 921 Times in 469 Posts
    Code:
    CRW_3473                 CRW_3475                 CRW_3480
    9843646  1.578s  1.734s  9259860  1.531s  1.687s  10806540  1.672s  1.828s // bim 01
    9633421  0.297s  0.328s  8942837  0.281s  0.312s  10629911  0.297s  0.328s // qlic v1.demo
    9606709  5.453s  0.938s  8810824  5.187s  0.937s  10642841  5.313s  0.969s // bcif 1.0 beta
    9262732  2.344s  1.532s  8514152  2.328s  1.500s  10127448  2.437s  1.594s // bmf 2.01
    9258959  1.844s  1.953s  8450343  1.813s  1.937s   9908162  1.938s  2.047s // flic v1.4.demo
    9198483  1.438s  1.609s  8586752  1.406s  1.594s  10072920  1.437s  1.640s // flic v2.1.demo
    8682920 37.281s 32.688s  7899272 33.234s 29.203s   9490224 36.172s 32.437s // bmf 2.01 -S 
    8681236 91.593s 33.781s  7898968 84.766s 29.219s   9490224 96.031s 31.547s // bmf 2.01 -S -Q9
    images are 2840x2128x24bit files converted from canon raws.

  3. #3
    Member Alexander Rhatushnyak's Avatar
    Join Date
    Oct 2007
    Location
    Canada
    Posts
    232
    Thanks
    38
    Thanked 80 Times in 43 Posts
    1 204 847 739 bytes on LPCB images, all decompressed files were bit-exact.

    Why flic v1.4.demo instead of 2.1.demo? There is no doubt the latter must be used.

    This newsgroup is dedicated to image compression:
    http://linkedin.com/groups/Image-Compression-3363256

  4. #4
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Thanks for testing! I hope you'll add it to the LPCB page too.

    2Shelwien
    Thanks for good examples of "bad" images for BIM. I will keep these for testing!

  5. #5
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,134
    Thanks
    179
    Thanked 921 Times in 469 Posts
    added flic 2.1 - 1.4 was the first link in http://www.imagecompression.info/gralic/
    Looks like it was better though.

  6. #6
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Currently working on BIM v0.02. This new version will feature BMP image support, support for PPM images with comments, it will be slightly faster and 100% compatible with current version.
    As an example, please download this packed BMP file and unpack it with the current BIM v0.01! BIM image format is extremely flexible!
    Attached Files Attached Files

  7. #7
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Well, new version will have a higher compression at higher speed - i.e. with same complexity + some optimizations. And it will be not compatible with the current one!

  8. #8
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Some testing results:
    Code:
    cow.ppm
    bim001: 18361655->7207617
    bim002: 18361655->6956958
    
    canon24.ppm
    bim001: 53747729->22567517
    bim002: 53747729->21844975
    
    DSC_8341.ppm
    bim001: 36636689->15934249
    bim002: 36636689->14886643
    
    CRW_3480.ppm
    bim001: 18130577->10806497
    bim002: 18130577->10426843
    
    But, in some cases:
    sony24.ppm
    bim001: 48098321->19324061
    bim002: 48098321->19689776
    The most important thing to note is that the new BIM v0.02 IS FASTER than BIM v0.01! Overall it has notable better compression (Except sony24.ppm and some weird artificial files)

  9. #9
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Added TGA image support. Please note that it's better to test BIM with PPM/PNM or TGA files, instead of BMPs. BMP files contain extra bytes for row alignment, latter may contain noise. Since BIM is TRULY lossless (all header info and comments are kept untouched) it will compress that noise/redundant bytes as well. With a fake PPM header, as example, you may compress and decompress any file with no loss, even a text file!

    Code:
    C:\bim\Release>bim c cow.ppm cow.bim
    Compressing cow.ppm: 18361655->6956962 in 0.624s
    
    C:\bim\Release>bim c cow.tga cow.bim
    Compressing cow.tga: 18361656->6956572 in 0.624s
    
    C:\bim\Release>bim c cow.bmp cow.bim
    Compressing cow.bmp: 18367770->7236460 in 0.624s

  10. #10
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    That doesn't look right. A normal BMP header is only 54 bytes, and end of row padding should be predictable.

  11. #11
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    The file shown is the worst case scenario. Looks like these padding bytes add noise.

  12. #12
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    An opposite example:
    Code:
    C:\bim\Release>bim c PIA12203.ppm PIA12203.bim
    Compressing PIA12203.ppm: 353296335->95418720 in 10.873s
    
    C:\bim\Release>bim c PIA12203.bmp PIA12203.bim
    Compressing PIA12203.bmp: 353311650->95378096 in 10.966s

  13. #13
    Member Alexander Rhatushnyak's Avatar
    Join Date
    Oct 2007
    Location
    Canada
    Posts
    232
    Thanks
    38
    Thanked 80 Times in 43 Posts
    Code:
    C:\bim\Release>bim c cow.ppm cow.bim
    Compressing cow.ppm: 18361655->6956962 in 0.624s
    ...
    C:\bim\Release>bim c cow.bmp cow.bim
    Compressing cow.bmp: 18367770->7236460 in 0.624s
    in PPM files the lines are in top-to-bottom order,
    in BMP they are bottom-to-top,
    this may be the reason.

    This newsgroup is dedicated to image compression:
    http://linkedin.com/groups/Image-Compression-3363256

  14. #14
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    BIM v0.02 has been released!

    http://encode.narod.ru/

    Please enjoy new release!



    Just a few notes. As I said, new version is slightly faster, supports 24-bit BMP, TGA and PNM/PPM images. It also has a slightly more complex internals than BIM v0.01, but I'm aiming on speed, so I'll continue to keep it as simple as possible. As example, changing CM Counters to more advanced ones will notable improve compression at the cost of 1.5-2X slowdown - it's not the case for a fast image compressor.

  15. #15
    Member Bloax's Avatar
    Join Date
    Feb 2013
    Location
    Dreamland
    Posts
    52
    Thanks
    11
    Thanked 2 Times in 2 Posts
    After long years of lurking, I couldn't really miss out to post that BIM performs pretty badly on some images.

    Which could of course be explained by the primary focus being photographic images, but that's far from the only use for a lossless image format.
    So yeah, uh, hi?

  16. #16
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    BIM is designed for large (12+ MP), true-color, photographic images. It is just not intended for such artificial images.

    However, about such "bad" performance. Check out how some other image compressors perform on one of your test files:

    ProgramCompressed Size
    Original "Test.ppm"3,307,588 bytes
    QLIC v1.demo2,441,867 bytes
    BCIF v1.0 beta1,699,125 bytes
    FLIC v2.1.demo1,668,990 bytes
    BIM v0.02847,367 bytes
    BIM v0.01687,272 bytes

    Yes, for artificial images BIM v0.01 is just better. (Better pattern recognition blah blah blah)


  17. #17
    Member Bloax's Avatar
    Join Date
    Feb 2013
    Location
    Dreamland
    Posts
    52
    Thanks
    11
    Thanked 2 Times in 2 Posts
    Oh noes!
    Code:
    *** LZ4 v1.3b, by Yann Collet (Sep  3 2012) ***
    Compressed Filename is : H:\Test.ppm.lz4
    Detected : 2 cores
    Compressing H:\Test.ppm using 1 threads (compression level = 0)
    Compression completed : 3230KB --> 317KB  (9.82%) (324960 Bytes)
    Total Time : 0.02s ==> 194.6MB/s
    (CPU : 0.02s = 92%)


    Though this is definitely as artificial as images get, it's literally one huge scene composed from a set of little images. (Which is why it's so compressible.)

    Also oh dear, BCIF sure did bad on it. :s
    Last edited by Bloax; 6th February 2013 at 02:13.

  18. #18
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts

  19. #19
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Code:
    C:\bcm\x64\Release>bcm test.ppm
    BCM 0.12 Experimental BWT compressor
    Copyright (C) 2010 Ilia Muraviev
    
    Compressing...
    3307588 -> 54477 in 0 sec

  20. #20
    Member Bloax's Avatar
    Join Date
    Feb 2013
    Location
    Dreamland
    Posts
    52
    Thanks
    11
    Thanked 2 Times in 2 Posts
    The point really is that it's kind of silly that lz4 in maximum Speed Racer mode is that much better.

  21. #21
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    Code:
       29,493 test1.paq8px v69 -7
       54,461 test1-3.zpaq
       54,884 test1-8.zpaq
       54,886 test1-7.zpaq
       54,894 test1-6.zpaq
       54,903 test1-5.zpaq
       64,877 test1.dmc 1000000000
       67,690 test1-4.zpaq
       73,067 test1.bz2 -9
       75,602 test1.7z -mx
       80,261 test1.pmd -o8 -m256 -r1
      108,830 test1-2.zpaq
      125,027 test1.gz -9
      152,642 test1.ctw -d6 -n16M -f16M
      158,299 test1-1.zpaq
      419,393 test1.Z
    2,407,842 test1.fpaq0
    3,307,588 test.ppm
    zpaq -method 3 is BWT. 4..8 are CM. 1..2 are LZ77.

  22. #22
    Member Alexander Rhatushnyak's Avatar
    Join Date
    Oct 2007
    Location
    Canada
    Posts
    232
    Thanks
    38
    Thanked 80 Times in 43 Posts
    1 161 296 397 bytes on LPCB images, all decompressed files were bit-exact.

    This newsgroup is dedicated to image compression:
    http://linkedin.com/groups/Image-Compression-3363256

  23. #23
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Thanks for testing!

  24. #24
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Results with different counters - BIM v0.02 - Fast counters, BIM v0.02s - Slow counters (1.5X slower)

    ImageBIM v0.02BIM v0.02s
    lena.ppm464,619447,432
    cow.ppm6,956,9626,890,665
    butterfly4.ppm4,884,0264,825,383
    Test.ppm847,367644,043
    CRW_3473.ppm9,519,8489,448,486
    CRW_3475.ppm8,924,2378,854,691
    CRW_3480.ppm10,426,84710,356,928
    PIA13912.ppm30,959,30030,592,489
    PIA12203.ppm95,418,72094,794,028
    DSC_8341.ppm14,886,64714,711,769
    canon24.ppm21,844,97921,662,606
    fuji24.ppm16,407,17616,287,323
    hubble.ppm22,827,69022,624,009
    oly24.ppm13,992,74313,860,374
    radiograph.ppm4,436,3414,373,007
    sigma24.ppm4,293,1004,253,541
    sony24.ppm19,689,78019,482,372

    This means only one thing - current counters are crap. Will search for a faster version of these "slow" counters.

  25. #25
    Member Alexander Rhatushnyak's Avatar
    Join Date
    Oct 2007
    Location
    Canada
    Posts
    232
    Thanks
    38
    Thanked 80 Times in 43 Posts
    Quote Originally Posted by encode View Post
    This means only one thing - current counters are crap.
    Not necessarily. Look at them from the other side: when you switch from slow counters to fast, compression quality is ~1% worse, but speed is 1.5 times higher!
    Actually, even less than 1% on average most likely,
    e.g. PIA12203.ppm: 95418720/94794028 = 1.00659

    This newsgroup is dedicated to image compression:
    http://linkedin.com/groups/Image-Compression-3363256

  26. #26
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Yep, in terms of efficiency - with no doubt. But my current goal with BIM - a higher compression! BIM must beat RKIM and a couple of others. At the same time, I have no room for an additional complexity that will slow it down, easily. FLIC defined a bargain.
    Like I said, I have a bunch of tricks that will significantly improve BIM's compression performance - at the cost of 2X-10X speed penalty - no reason to do that since we have PAQ8(im) and such. So currently I'm trying to squeeze-in some of the ideas, in simplified and optimized forms.
    Like these slow counters might be simulated with much faster FSM-based ones. Just currently, I had no luck - FSM building and optimization is not the easiest task.

  27. #27
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    GARTEST - re-rendered with a newer ViewNX software + NEF(RAW)
    http://compressme.net/gartest.rar

    Currently, I'm working on BIM v0.03

    Some results on DSC_8341.ppm:
    BIM v0.02 -> 14,871,253 bytes
    BIM v0.03 -> 14,363,389 bytes

  28. #28
    Member kampaster's Avatar
    Join Date
    Apr 2010
    Location
    ->
    Posts
    55
    Thanks
    4
    Thanked 6 Times in 6 Posts
    Hi Ilya.
    AIMP_free.tga 16 437 887 bytes
    BIM v0.02 2 049 953 bytes
    BMF v.1.1 496 300 bytes
    BMF v.2.01 381 512 bytes
    PPMd 367 930 bytes
    Lzma 324 165 bytes
    UHARC 303 796 bytes
    FP8231 778 bytes
    Attached Files Attached Files

  29. #29
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    I used ImageMagick convert to convert .tga to .bmp and .pmm. Tests with zpaq were on the .tga file.

    Code:
       544,707 AIMP_free-m1.zpaq
       494,449 AIMP_free-m2.zpaq
       518,582 AIMP_free-m3.zpaq
       355,052 AIMP_free-m4.zpaq
       308,943 AIMP_free-m5.zpaq
       279,088 AIMP_free-m6.zpaq
    16,439,334 AIMP_free.bmp
     2,050,086 AIMP_free.bmp.bim
    16,437,860 AIMP_free.ppm
     2,064,893 AIMP_free.ppm.bim
    16,437,887 AIMP_free.tga
     2,049,953 AIMP_free.tga.bim
       731,038 AIMP_free.zip

  30. #30
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Once again - BIM was not designed for such images. BIM is for true color pictures from your camera...

Page 1 of 2 12 LastLast

Similar Threads

  1. FLIC - a new fast lossless image compressor
    By Alexander Rhatushnyak in forum Data Compression
    Replies: 25
    Last Post: 10th January 2013, 19:46
  2. New lossless image compressor
    By encode in forum Data Compression
    Replies: 105
    Last Post: 10th January 2013, 10:36
  3. Lossless image coders
    By Madgeniy in forum Data Compression
    Replies: 26
    Last Post: 11th July 2011, 09:06
  4. GraLIC - new lossless image compressor
    By Alexander Rhatushnyak in forum Data Compression
    Replies: 17
    Last Post: 29th November 2010, 21:27
  5. New lossless compressor for 24-bit images (3 channels, 8 bits per channel)
    By Alexander Rhatushnyak in forum Data Compression
    Replies: 28
    Last Post: 23rd September 2010, 01:43

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •