Results 1 to 22 of 22

Thread: New lpaq1 version

  1. #1
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    Alexander Rhatushnyak has done some optimization to lpaq1 while preserving archive compatibility. It is about 6% faster than version 1 when compiled with g++ or 10% faster with VC++. However, g++ is still faster, at least on my PC, so I included a g++ compile.

    http://cs.fit.edu/~mmahoney/compression/#lpaq1

  2. #2
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Thanks Alexander, and Matt!

    Mirror: Download

  3. #3
    Member
    Join Date
    Sep 2007
    Location
    Italy
    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Very good, thank you

  4. #4
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    LPAQ2 has been released!

  5. #5
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    lpaq5 and lpaq5e

    are by Alexander Rhatushnyak, Oct. 16, 2007. Option 9 selects 1542 MB memory. lpaq5e is tuned for large text files. It includes separate programs for compression only (lpaq5e-c.exe) and decompression only (lpaq5e-d.exe). Tests were done with these programs, rather than the version that does both (lpaq5e.exe).
    lpaq5e OPTION 9
    ENWIK8: 19,078,767
    ENWIK8: 156,194,860

  6. #6
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Thanks!

    Mirror: Download

  7. #7
    Member
    Join Date
    May 2009
    Location
    France
    Posts
    95
    Thanks
    13
    Thanked 72 Times in 42 Posts
    Hi,

    Viruscan tells me that it has detected "New Malware.aj" in LPAQ5.EXE. Please check your archives...


    AiZ

  8. #8
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    That's common anti-virus reaction on files packed with UPack.

    (unfortunately, many malware files are packed with UPack just because it is one of the best exe-compressing programs, thus a definition was added that EVERY file packed with it is automatically a malware)

  9. #9
    Member
    Join Date
    Oct 2007
    Posts
    51
    Thanks
    0
    Thanked 0 Times in 0 Posts
    I'ver tried lpaq5 and it is a little better in compression than lpaq4, same speed.

    I love LPAQ5, it offers the best size/speed tradeoff and, at the moment, it is the best among open source compressors (GPL).

    CCM and Freearc are interesting but are not at the moment are not GPL and in my opinion that a very bad choiche.

    I've asked to Giorgio Tani (the creator of the fine Peazip) to support the last LPAQ. It already support LPAQ1 (with automatic tar creation) and he told me he is very interested and he will support the other LPAQ releases and the only worry is the backward compatibility among them.
    He will probably try to support of all the LPAQ release using a selector.

    I suggest to have a try to Peazip (GPL); you can even replace the lpaq1.exe with lpaq5.exe (after renaming it lpaq1.exe) to use efficiently lpaq through a GUI and having automatic TAR creation...

  10. #10
    Member
    Join Date
    Oct 2007
    Posts
    51
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Someone knows how to contact the author of LPAQ5?

    I have the following questions:

    Will the next release LPAQx be backward compatible?

    Is LPAQ5 a "stable" release, or a beta ?

    Thanks in advance,
    Gish

  11. #11
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    LPAQ5 is not stable. I have already received LPAQ6 today and will post after running tests overnight. I would not be surprised to see LPAQ7 next week

  12. #12
    Member
    Join Date
    Oct 2007
    Posts
    51
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Now, let's see to facts and numbers, from the test of Calgary at your web site:

    LPAQ6 is 0.15% better in compression, and 4.9% slower of the previous release (LPAQ5)
    Instead LPAQ5 was 0.157% better in compression and 1.87% faster than LPAQ4.

    Making some rough esteem, I foresee that LPAQ89 will have the same compression ratio of the best actual PAQ release.

    With an approximate exteem on the same data, LPAQ89 will be at least 13 times slower than now (1300%) ...

    However, since also PAQ is constantly updated, at that point the race will not be over... it's hard to foresee, but I suppose the matching point could not be reached before the release of LPAQ320 and PAQ90...

    Obviously, using different test results, (e.g. enwik, the esteem change. Apparently not for better figures...

    Patience is the virtue of the strong one.
    However, that should happen not before than ten years and a couple of things maybe will change about compression in the meantime ...

    I hope you will tell me I'm completely wrong. really.

  13. #13
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Thanks Alexander, and Matt!

    Mirror: Download

  14. #14
    Member
    Join Date
    Oct 2007
    Posts
    51
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by LovePimple
    Thanks Alexander, and Matt!
    Yes, I thank them too: M att Mahoney and Alexander Rhatushnyak are two giants...

    They have won the Hutter Prize for compression, that means they have developoed the absolutely *best* compression algorithms.
    Matt has invented PAQ, LPAQ but its innovation was not only focused on compression rate, because Ive just discovered that SR2 (another compressor of Matt) is probably one of the fastest.

    Alexander has done so many optimizations on anything, and he won the last 2 Hutter Prize (The first was won by Matt).

    The hobby of Matt and Alexander are races, and both are winners.
    However, I guess that if Alexander will go on searching the upper limit of compression for LPAQ, he will find that limit is ... PAQx, in fact (if Im not wrong) LPAQ is the simplified version of PAQ... (and will it be still optimized for speed???)

    Oh, and thanks LovePimple for the mirroring and the links...

  15. #15
    Member
    Join Date
    Jun 2009
    Location
    Kraków, Poland
    Posts
    1,471
    Thanks
    26
    Thanked 120 Times in 94 Posts
    lpaq doesn't contain models for specific files (eg. bmp, jpg, sparse models). those are reserved for paq. lpaq is focused mainly on text compression, ie. ppmonstr competitor.

  16. #16
    Member
    Join Date
    Oct 2007
    Posts
    51
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by donkey7
    paq doesnt contain models for specific files (eg. bmp, jpg, sparse models). those are reserved for paq. lpaq is focused mainly on text compression, ie. ppmonstr competitor.
    LPAQ is optimized for text files, but Im doing test on binary files and it works nicely. I would say it is general purpose compressor, .i.e. it is not specialized for multimedia compression.

    Moreover LPAQ6, just released, has "E8-E9 filter for better compression of x86 executables", using the words of Matt, who is surely more expert than me about LPAQ...

  17. #17
    Member Alexander Rhatushnyak's Avatar
    Join Date
    Oct 2007
    Location
    Canada
    Posts
    232
    Thanks
    38
    Thanked 80 Times in 43 Posts
    Quote Originally Posted by Gish
    Will the next release LPAQx be backward compatible?
    Is LPAQ5 a "stable" release, or a beta ?
    Compatibility: NO,
    but I think Ill release last version of LPAQ in November or December.
    LPAQ is more an algorithm than an archiver: is stores almost nothing except the compressed data in the archive:

    fprintf(out, "pQ%c%c%c%c%c%c", 6, argv[1][0], size>>24, size>>16, size>>8, size);

    where argv[1][0] is the memory option (0...9), and 6 is the version number.

    BUGS: probably YES for executables (should be tested, especially on *.exe smaller than 32 kb),
    NO (I think ) for other files.

    This newsgroup is dedicated to image compression:
    http://linkedin.com/groups/Image-Compression-3363256

  18. #18
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    Alexander, is it possible to improve speed about 1.5 times? say, decrease amount of cache misses in some way?

  19. #19
    Member
    Join Date
    Oct 2007
    Posts
    51
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Alexander Rhatushnyak
    BUGS: probably YES for executables (should be tested, especially on *.exe smaller than 32 kb),
    NO (I think ) for other files.
    Is test needed only for EXE smaller than 32K or also DLL?

    If lot of EXE and DLL files are put in a TAR uncompressed archive, and LPAQ is used to compress that archive, is the E8-E9 filter still applied?

    The test about small EXE must be done on single files or can it be done on a TAR archive containing lot of EXE files?

    Ive done a simple test with LPAQ6 using a huge TAR uncompressed archive (MediaCoder-0.060.3930, a Sourceforge project) containing 35Mbytes of .EXE and .DLL files. In that archive there are not EXE smaller than 32K (only few DLL are).
    The binary comparison between original TAR file and LPAQ compressed+decompressed file is OK: no difference.

    The tests has been done using the best compressors I know (LPAQ, CCM, FreeArc, Durlica, UHARC, PPMonstrJ) and the resutls shows differences are very little between compressors (I guess some of those DLL, EXE are already compressed with UPX...)
    I repeat here the results (for commodity).
    Ive used MediaCoder.0.6.0.-3930 (OpenSource, you can download it from SourceForge) as a single tar file uncompressed: 39,381,504 bytes, mostly binary filefor Windows.

    Program________Compr%____Speed_______Params___
    ===============================================
    CCM 1.25_______45.77%____1108.18 KiB/s_"c 4"
    CCMX 1.25______45.74%____982.14 KiB/s__"c 4"
    FreeArc 0.40_____49.80%____287.65 kb/s__"a -mx"
    FreeArc 0.40_____50.10%____634.63 KB/s__"a -m5"
    LPAQ6__________45.24%____313.03 KB/s__"6"
    LPAQ1v2________45.69%____319.25 KB/s__"6"
    PPmonstrJ_______45.70%____112 KB/s_____"e -m256 -o16"
    PPMdj___________54,20%___911 Kb/s______"e -m256 -o16"
    DurlicaLightt 0.5___45.56%____418.03 KB/s__"-t1"
    7-zip 4.56_______45.76%____1005Kb/s____"ultra lzma 16M 128b"

    Working on folders/files (not on TAR file) gives better results. Unfortunately that cannot be done by most of the above compressors. Below are the results with mediacoder3930, 167 Files 31 folder, 39249112 bytes (directly extracted from the Mediacoder-0.6.0.3930.7z, available at SourceForge):

    Program________Compr%____Speed_______Params___
    ===============================================
    FreeArc 0.40pr1__43.04%____563.66 KiB/s_"a -mx"
    7-zip 4.56_______43.28%____1127Kb/s____"ultra lzma 16M 128b"
    7-zip 4.56_______43.05%____1035Kb/s____"ultra lzma 24M 128b"

  20. #20
    Member
    Join Date
    Jan 2007
    Location
    Moscow
    Posts
    239
    Thanks
    0
    Thanked 3 Times in 1 Post
    Hmm... Strange numbers...
    I have 0.6.0 build 3520. Program folder is 40,413,332 bytes long.
    WinRar max,default,solid - 16,159,034
    7-Zip lzma,max,24 m,128 - 14,002,459
    FreeArc m5 - 13,901,535 = 34.3%

    After [for %x in (*.*) do upx -d %x] in some folders size is 48,625,812.
    WinRar max,default,solid - 14,002,459
    7-Zip lzma,max,24 m,128 - 12,206,526
    FreeArc m5 - 12,023,171 = 24.7%

  21. #21
    Member
    Join Date
    May 2007
    Location
    Poland
    Posts
    85
    Thanks
    8
    Thanked 3 Times in 3 Posts
    Since LPAQ is so nice and small, would it be possible to make SFX stub from it? Even dos console SFX would be great.
    Anyone?

  22. #22
    Member
    Join Date
    May 2008
    Location
    Kuwait
    Posts
    301
    Thanks
    26
    Thanked 22 Times in 15 Posts
    i think a UPX preprocessor is required as UPX will compress single file but archivers use solid method to look for shared streams between files thats why when you unpack them all it improved compression easily

Similar Threads

  1. lpaq1 is here
    By Matt Mahoney in forum Forum Archive
    Replies: 10
    Last Post: 31st July 2007, 18:06

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •