Results 1 to 23 of 23

Thread: BCM v0.09 - The ultimate BWT-based file compressor!

  1. #1
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts

    Exclamation BCM v0.09 - The ultimate BWT-based file compressor!

    Let me introduce the best piece of my software! Please welcome, the ultimate BWT-based file compressor - BCM v0.09! This is kind of a final release, since I have no spare time completely...



    Anyway, BCM v0.09 on some test files:

    book1 -> 208,726 bytes
    calgary.tar -> 775,271 bytes
    ENWIK5 -> 1,195,166 bytes
    ENWIK8 -> 20,625,697 bytes
    world95.txt -> 463,725 bytes
    fp.log -> 551,378 bytes
    bible.txt -> 721,591 bytes
    3200.txt-> 3,641,305 bytes

    Attached Files Attached Files

  2. #2
    Member
    Join Date
    Aug 2009
    Location
    Bari
    Posts
    74
    Thanks
    1
    Thanked 1 Time in 1 Post
    yessssss!!! thanks encode!!! you're great!! improve compression ratio than v0.08!!
    Last edited by PiPPoNe92; 20th August 2009 at 01:41.

  3. #3
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts

    Thumbs up

    Thanks Ilia!

  4. #4
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    681
    Thanks
    37
    Thanked 168 Times in 84 Posts
    My congrats with new BCM, Ilia
    Some tests:
    Code:
                  | encode time |     size    | decode time |
    --------------------------------------------------------|
                  |         PAK0.PAK (302 895 669)          |
    --------------------------------------------------------|
    0.07 -b65536  |  181.813    | 117 053 060 |  135.813    |
    0.08 e64      |  227.422    | 116 587 092 |  185.234    |
    0.09 -b64     |  383.133    | 115 564 822 |  343.401    |
    --------------------------------------------------------|
                  |          ENWIK8 (you know :) )          |
    --------------------------------------------------------|
    0.07 -b102400 |   61.265    |  20 770 673 |   45.109    |
    0.08 e100     |   77.828    |  20 744 613 |   61.172    |
    0.09 -b100    |  110.964    |  20 625 697 |   95.135    |
    --------------------------------------------------------|
                  |         NKOEF.DBF (228 819 632)         |
    --------------------------------------------------------|
    0.07 -b32768  |  108.078    |   4 432 347 |   89.859    |
    0.08 e32      |  154.953    |   4 393 106 |  131.453    |
    0.09 -b32     |  214.163    |   4 245 588 |  192.951    |
    --------------------------------------------------------'
    By the way, on ENWIK8 BWT_reorder_v2 + BCM 0.09 -b100 = 20 555 272

  5. #5
    Member
    Join Date
    Aug 2009
    Location
    Bari
    Posts
    74
    Thanks
    1
    Thanked 1 Time in 1 Post

    Talking

    encode, bcm is more powerful than 7z, but it takes a few ram and cpu, if you do bcm 0.1, can you improve compression increase cpu and ram?? because now general pc have 4 gb ram and 2 cpu thanks for answers
    Last edited by PiPPoNe92; 20th August 2009 at 16:10.

  6. #6
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Use '-b' option! As example:
    bcm -b320 enwik9
    bcm -b200 pak0.pak
    bcm -b471 textures.tar

  7. #7
    Member
    Join Date
    Aug 2009
    Location
    Bari
    Posts
    74
    Thanks
    1
    Thanked 1 Time in 1 Post
    don't change nothing! the compression ratio and final size are equals
    Last edited by PiPPoNe92; 20th August 2009 at 22:45.

  8. #8
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Well, I might be if the block size (-b) is equal or bigger than the file size...

    As example, if you about to compress 4 MB file

    bcm file
    bcm -b5 file
    bcm -b500 file

    will produce the same results.

    But if the file size is 800 MB, the -b option will take a serious effect.

    But note, in some cases like with EXE-files, TAR-archive with mixed stuff, etc. a smaller block might be better:

    bcm -b5 reaktor.exe


  9. #9
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Looks like the faster BCM engine (v0.07,v0.0 was more successful and popular. I'm just thinking about to release a new BCM v0.10 with much faster/lighter compression engine. Incorporating some ideas and optimizations from the latest v0.09 and simplicity of v0.07. I think it will be even more simpler than v0.07...

  10. #10
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    The brand new BCM v0.10 should be the fastest BCM ever. I've done many optimizations and time measurements using RDTSC to find out the lightest and yet efficient CM back end. 209xxx on book1 it has less complex CM part, new CM is even simpler than that BCM v0.07 had. On some files it may provide a slightly worse compression compared to the v0.07, on some may not - providing a higher compression. Overall, it's something like BCM v0.07, but notable faster - due to the simplest CM back-end and some new stuff that I've invented working on BCM v0.09. I think new BCM is very interesting piece of software, firstly, at the practical point of view...


  11. #11
    Member
    Join Date
    Aug 2009
    Location
    Bari
    Posts
    74
    Thanks
    1
    Thanked 1 Time in 1 Post
    thanks encode for the info!!

  12. #12
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts

    Cool

    Here are some of the testing results of a test version of the new BCM v0.10. As a note new BCM is faster than v0.07 and much faster than any newer...

    book1 -> 209,781 bytes
    bible.txt -> 726,154 bytes
    3200.txt -> 3,663,617 bytes
    fp.log -> 561,549 bytes
    ...

  13. #13
    Member zody's Avatar
    Join Date
    Aug 2009
    Location
    Germany
    Posts
    90
    Thanks
    0
    Thanked 1 Time in 1 Post
    Sounds VERY great; perfect efficiency

  14. #14
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts

    Exclamation

    Additionally, new BCM v0.10 more likely will be like a full-blown command-line compressor. With an extra error checking, more stable&efficient code, own file format with a small header, etc. I did many optimizations not only that refers to a main compression engine, as example a faster unbwt (decompression), but CLI-related stuff as well. Faster percentage calculation, more stable command-line parsing&file name generation, etc. In other words, it's the entire program rewrite! I really checked and rechecked each line of code. The release is soon...


  15. #15
    Tester

    Join Date
    May 2008
    Location
    St-Petersburg, Russia
    Posts
    182
    Thanks
    3
    Thanked 0 Times in 0 Posts
    Encode, what's about idea of adding new BCM into your Pim Archiver? I think, it will be very interesting

  16. #16
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts

    PIM Logo

    Quote Originally Posted by squxe View Post
    Encode, what's about idea of adding new BCM into your Pim Archiver? I think, it will be very interesting
    BCM is still an experimental file compressor - since it's under heavy development and some compression related changes may occur, breaking any compatibility. I do plan to add BCM to PIM archiver as one of the PIM's algorithms and/or as .BCM file extraction. Anyway, the main goal of the BCM is a freeware file compression tool and a commercial data compression library; i.e. it will never be an open source as my other projects like BALZ or QUAD. Although, I do plan to release binaries for Linux/Unix.


  17. #17
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts

    Cool BCM goes 64-bit!

    Just made a 64-bit version of the BCM. And it runs faster than its 32-bit version. So, the main version will be for 64-bit platforms. Alternatively, the BCM package will contain the X86 compile.


  18. #18
    Member Vacon's Avatar
    Join Date
    May 2008
    Location
    Germany
    Posts
    523
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hello everyone,

    Quote Originally Posted by encode View Post
    Just made a 64-bit version of the BCM. And it runs faster than its 32-bit version. So, the main version will be for 64-bit platforms. Alternatively, the BCM package will contain the X86 compile.
    So the children will get something to play with soon?

    Best regards!

  19. #19
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Actually, the new BCM already written some time ago. I'm just playing with some optimizations and thinking about more correct error messages.

  20. #20
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    New BCM was developed and optimized under at least three followed processors:
    Pentium 4
    Core 2 Duo
    Core 2 Quad

    I think I'll look more to the 64-bit side. I optimized some parts of the BCM specially for the 64-bit machines. At some point, I'm about to release kind of a final, practical BCM, to keep compatibility between versions. Some ways of the optimizations are incompatible with each other. That's why I need to choose a right way. Testing under all these processors helped me a lot.

  21. #21
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts

    Cool

    New BCM v0.10 will be released very soon. It's completely written for now, I just need a more testing with the max block sizes. Anyway, the key differences:
    • New BCM displays progress for the entire file, not just a current block
    • For the first time, of all of my command line compressors, new BCM has it's own file format, with its own file header, file ID, etc.
    • Better error handling and CL user interface
    • And of course, the brand new and much more improved BCM's compression engine
    • Concluding, it's a complete data compression tool with its non-experimental look and feel
    After releasing this piece of software, I'll continue working on my byte-oriented PPMX...

  22. #22
    Member Fu Siyuan's Avatar
    Join Date
    Apr 2009
    Location
    Mountain View, CA, US
    Posts
    176
    Thanks
    10
    Thanked 17 Times in 2 Posts
    Highly expecting!

  23. #23
    Member
    Join Date
    Oct 2015
    Location
    Portugal
    Posts
    12
    Thanks
    1
    Thanked 1 Time in 1 Post
    any changes open sourcing this project?

    EDIT: Oh, I just find it. Thanks. https://sourceforge.net/projects/bcm/

  24. The Following User Says Thank You to Senoble For This Useful Post:

    encode (6th March 2016)

Similar Threads

  1. PPMX v0.05 - new PPM-based compressor
    By encode in forum Data Compression
    Replies: 49
    Last Post: 28th July 2010, 02:47
  2. BCM v0.08 - The ultimate BWT-based file compressor!
    By encode in forum Data Compression
    Replies: 78
    Last Post: 12th August 2009, 10:14
  3. BCM v0.01 - New BWT+CM-based compressor
    By encode in forum Data Compression
    Replies: 81
    Last Post: 9th February 2009, 16:47
  4. Blizzard - Fast BWT file compressor!!!
    By LovePimple in forum Data Compression
    Replies: 40
    Last Post: 6th July 2008, 14:48
  5. DARK - a new BWT-based command-line archiver
    By encode in forum Forum Archive
    Replies: 138
    Last Post: 23rd September 2006, 21:42

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •