Results 1 to 24 of 24

Thread: quad 1.03a has been released!

  1. #1
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    What's new:
    + Added CRC32 checking
    + Improved compression (Improved PPM part: Added Full Exclusion, plus now for LZ output coding, Quad uses my own invention - a Fast PPM encoder)
    + Some code optimizations

    Enjoy!

    Link:
    Download Quad 1.03a (26 KB)


  2. #2
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Awesome! Thanks Ilia!

  3. #3
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    First test results...

    A10.jpg > 861,165
    AcroRd32.exe > 1,489,459
    english.dic > 822,920
    FlashMX.pdf > 3,835,098
    FP.log > 632,475
    MSO97.dll > 1,905,686
    ohs.doc > 845,232
    rafale.bmp > 1,013,579
    vcfiu.hlp > 681,426
    world95.txt > 601,009

    Total = 12,688,049 bytes


    ENWIK8: > 29,410,508

    PHT > 1,135,544

    table4_2007-01-14.png > 172,341

    Compression/decompression time was VERY quick for all files.

  4. #4
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    Niiiice Thanks!

  5. #5
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    quad 1.02a - 13,695,765
    quad 1.03a - 13,484,815
    @ my test...
    Details tomorrow

  6. #6
    Tester

    Join Date
    May 2008
    Location
    St-Petersburg, Russia
    Posts
    182
    Thanks
    3
    Thanked 0 Times in 0 Posts
    Thanks!

  7. #7
    Tester

    Join Date
    May 2008
    Location
    St-Petersburg, Russia
    Posts
    182
    Thanks
    3
    Thanked 0 Times in 0 Posts
    What do you think about increase memory usage for better compression ratio in future version of Quad?

  8. #8
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Quote Originally Posted by Squxe
    What do you think about increase memory usage for better compression ratio in future version of Quad?
    Well, Im not planning this. I have an idea about add a CM variant to the Quad, but even in this case, memory usage will be the same.


  9. #9
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by encode
    I have an idea about add a CM variant to the Quad
    Does this mean that we will be seeing an increased compression ratio?

  10. #10
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Quote Originally Posted by LovePimple
    Does this mean that we will be seeing an increased compression ratio?
    Yes, any future improvements will be compression related. I will try to increase compression without significant speed loss. Ive already done some experiments with CM (PAQ-like encoder). Some info:

    Fast CM - a la PAQ1 with some serious speed improvements:
    Provides high binary compression, moderate text compression and poor random data "compression" (A10.jpg increased in size). The speed of this variant is fast enough - about the same as current PPM with Full Exclusion.

    The downside - poor random data handling and we need a higher text compression.

    Fast CM with SSE (or APM):
    Provides highest compression overall (A10.jpg is also compressed), keeping relatively good speed. However, speed is about 2...3 times slower compared to the pure Fast CM.

    The downside – low speed. I think, decompression becomes too slow.

    Fast CM with Adaptive Weighing:
    Im not tried this. Probably this is the best one. Also I guess the Malcolms ROLZ3 uses very similar CM modification.


  11. #11
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by encode
    Yes, any future improvements will be compression related. I will try to increase compression without significant speed loss.


    Quote Originally Posted by encode
    Fast CM - a la PAQ1 with some serious speed improvements:
    Provides high binary compression, moderate text compression and poor random data "compression" (A10.jpg increased in size). The speed of this variant is fast enough - about the same as current PPM with Full Exclusion.

    The downside - poor random data handling and we need a higher text compression.

    Fast CM with SSE (or APM):
    Provides highest compression overall (A10.jpg is also compressed), keeping relatively good speed. However, speed is about 2...3 times slower compared to the pure Fast CM.

    The downside – low speed. I think, decompression becomes too slow.

    Fast CM with Adaptive Weighing:
    Im not tried this. Probably this is the best one. Also I guess the Malcolms ROLZ3 uses very similar CM modification.
    As Im a fan of highest compression ratios, I would prefer the "Fast CM with SSE (or APM)" option because it gives best compression. The "Fast CM with Adaptive Weighing" option also looks very interesting.

    I understand that Quad is more about speed than high compression, but if compression is not improved the project becomes less interesting for many people.

  12. #12
    Member Fallon's Avatar
    Join Date
    May 2008
    Location
    Europe - The Netherlands
    Posts
    154
    Thanks
    14
    Thanked 10 Times in 5 Posts
    Quote Originally Posted by LovePimple
    I understand that Quad is more about speed than high compression, but if compression is not improved the project becomes less interesting for many people.
    You are right that compression is the name of the game and any
    compression improvement on existing similar compressors is fine.
    But I also like encodes stated goal to keep decompression speed high. Its of interest if somewhere down the road you want to end up with an archiver that can actually be used.

    At some point new technology could make the slower compression of today, like paq, usable in practice, even if now fast compression will then get even more speed.

    Just one thing to come that could impact on compression:

    January 27, Intel announced that new transistors will be implemented in the 45 nanometer Processortypes in the second half of 2007. The transistor change will enable 25x the number of transistors at 10 times the speed they operate today (also reducing heat and power consumption).

    For the extended story, search on intel.com for transistor or see:
    http://www.nytimes.com/2007/01/27/technology/27chi p.html
    http://www.mercurynews.com/mld/mercurynews/news/lo cal/16556996.htm
    http://news.findlaw.com/ap/ht/58/01-27-2007/615000 2712a1c587.html

  13. #13
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by Fallon
    You are right that compression is the name of the game and any
    compression improvement on existing similar compressors is fine.
    But I also like encodes stated goal to keep decompression speed high. Its of interest if somewhere down the road you want to end up with an archiver that can actually be used.
    I agree. If a file compressor such as Quad does not have superior compression or speed, why would anyone want to use it?

    As you said; "compression is the name of the game".

    Quote Originally Posted by Fallon
    At some point new technology could make the slower compression of today, like paq, usable in practice, even if now fast compression will then get even more speed.
    IMO this is true! Compressors like PIMPLE have already become much more practical with current technology.

  14. #14
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Quote Originally Posted by LovePimple
    I agree. If a file compressor such as Quad does not have superior compression or speed, why would anyone want to use it?
    Supercompression == unusable speed
    Superspeed == dummy compression

    The goal is to find something in the middle, with fast enough speed and high compression.

    Quote Originally Posted by LovePimple
    IMO this is true! Compressors like PIMPLE have already become much more practical with current technology.
    To be honest, currently Im seriously thinking about PIMPLE engine. Like I said, during this time, I collect a huge number of improvements - in both terms of compression speed and compression ratio. I hope, within a couple of weeks I start writing something (Quad 2.0, or TC 5.2)


  15. #15
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by encode
    Supercompression == unusable speed
    Superspeed == dummy compression

    The goal is to find something in the middle, with fast enough speed and high compression.
    I would prefer to have a user selectable choice of higher compression with lower speed or lower compression with higher speed.

  16. #16
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Quote Originally Posted by LovePimple
    I would prefer to have a user selectable choice of higher compression with lower speed or lower compression with higher speed.
    To cover such high range of speed and compression, compressor must use many algorithms. My compression programs (CL versions) uses just one compression algorithm, and it is right, since the goal of command-line compressors is introduce a new compression method.


  17. #17
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    I understand!

  18. #18
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    By the way, the work on Quad 1.04 in progress!
    Features:
    + High compression (currently, 11,912,598 bytes at SFC)
    + High speed (faster than LZPXJ)
    + CRC32 checking
    + Flexible Parsing

    In other words, this is crazy!


  19. #19
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Quote Originally Posted by encode
    In other words, this is crazy!


    Im looking forward to this new breed of super-high compression Quad!

  20. #20
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    The Flexible Parsing is very efficient with LZP compression. First, unlike with LZ77/ROLZ methods speed is not too much affected (2x in worst case). Also compression is also not so much improved as with LZ77. However, in some cases compression gain is very impressive. For example:

    fp.log
    Quad 1.04 with Flexible Parsing: 579,928 bytes
    Quad 1.04 with Greedy Parsing: 659,340 bytes
    Original: 20,617,071 bytes

    gimp-2.0.0.tar
    Quad 1.04 with Flexible Parsing: 9,110,534 bytes
    Quad 1.04 with Greedy Parsing: 9,450,509 bytes
    Original: 78,745,600 bytes

    world95.txt
    Quad 1.04 with Flexible Parsing: 577,788 bytes
    Quad 1.04 with Greedy Parsing: 594,338 bytes
    Original: 2,988,578 bytes

    In addition, this is my first LZP-based program with Flexible Parsing. Note that LZPX/LZPXJ, PIMPLE, PIM and new TC is also LZP-based, but these programs use Greedy Parsing only.


  21. #21
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Posted results show it to be quite impressive!

  22. #22
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Another results:

    Photoshop.exe
    Quad 1.04a: 5,727,429 bytes
    LZPXJ 1.2g: 6,032,957 bytes
    WinRAR 3.51, Best: 6,337,395 bytes
    DARK 0.51: 7,242,124 bytes
    PKZIP 2.50, -max: 8,632,418 bytes
    Original: 19,533,824 bytes

    Currently new Quad stays at testing stage. Probably, I'll change something, probably not, however release is soon!


  23. #23
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    This test shows how efficiently new LZP-engine (Quad 1.04a) can cover a large distances within dictionary.

    Test file: 1.tar, 66,241,536 bytes
    This file contains two copies of the same mpeg video file. Each file is 33,119,324 bytes long.

    <u>Results:</u>
    7-Zip 4.42, Ultra: 31,498,847 bytes
    Quad 1.04a: 37,624,776 bytes
    TC 5.1dev7: 61,193,908 bytes
    7-Zip 4.42, Maximum: 62,969,126 bytes
    WinRAR 3.51, Best: 63,401,402 bytes
    Original: 66,241,536 bytes

    <u>Comments:</u>
    RAR failed. Not surprisingly, max dictionary size = 4 MB

    7-Zip with Maximum level also failed. And again dictionary size of 7-Zip with Max mode is just 16 MB - not quite enough.

    TC 5.1dev7 is just my ROLZ variant. Again it screwed-up. Looks like it covers large dictionary not such efficiently.

    New Quad 1.04a is able to find and compress these two files.

    7-Zip with Ultra mode is best. But it needs 6X times larger free memory amount compared to the Quad.


  24. #24
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    I'm impressed with Quad 1.04a! I'm quite impressed by 7-Zip's performance as well!

Similar Threads

  1. QUAD 1.10 has been released!
    By encode in forum Forum Archive
    Replies: 10
    Last Post: 22nd March 2007, 11:40
  2. QUAD 1.08 has been released!
    By encode in forum Forum Archive
    Replies: 28
    Last Post: 16th March 2007, 11:17
  3. Quad 1.04a has been released! [!]
    By encode in forum Forum Archive
    Replies: 25
    Last Post: 15th February 2007, 19:01
  4. quad 1.02a has been released!
    By encode in forum Forum Archive
    Replies: 56
    Last Post: 20th January 2007, 15:03
  5. quad 1.01a has been released!
    By encode in forum Forum Archive
    Replies: 25
    Last Post: 1st January 2007, 22:50

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •