Results 1 to 29 of 29

Thread: LZTURBO

  1. #1
    Member
    Join Date
    Mar 2013
    Location
    Worldwide
    Posts
    456
    Thanks
    46
    Thanked 164 Times in 118 Posts

    LZTURBO

    i have uploaded a new version and new benchmark results under: http://sites.google.com/site/powturbo

  2. #2
    Tester
    Black_Fox's Avatar
    Join Date
    May 2008
    Location
    [CZE] Czechia
    Posts
    471
    Thanks
    26
    Thanked 9 Times in 8 Posts
    Quote Originally Posted by Bulat Ziganshin View Post
    i'm pleased to remind readers that dnd lives in Iran, not Germany, and he copied my open-source GPL Tornado library, made some modifications and started to sell it as "lzturbo" library a few years ago. This is violation of GPL rules and theft of my intellectual property
    I am... Black_Fox... my discontinued benchmark
    "No one involved in computers would ever say that a certain amount of memory is enough for all time? I keep bumping into that silly quotation attributed to me that says 640K of memory is enough. There's never a citation; the quotation just floats like a rumor, repeated again and again." -- Bill Gates

  3. The Following User Says Thank You to Black_Fox For This Useful Post:

    Bulat Ziganshin (30th April 2013)

  4. #3
    Member
    Join Date
    Mar 2013
    Location
    Worldwide
    Posts
    456
    Thanks
    46
    Thanked 164 Times in 118 Posts
    Now, stop this nonsense! You and others want to follow me with your lies to the eternity? This is simply jealousy. Look at benchmarks and see where Tornado is ranked. There are tons of free LZ library on the net, even lzma is public domain,
    and your are talking about a simple lz77 program, which is borrowing most of the ideas from zlib.

    So, shame on you!
    Last edited by dnd; 30th April 2013 at 12:53.

  5. #4
    Tester
    Black_Fox's Avatar
    Join Date
    May 2008
    Location
    [CZE] Czechia
    Posts
    471
    Thanks
    26
    Thanked 9 Times in 8 Posts
    If you took GPL sources, made some improvements and then closed the source, then you are violating GPL license rules no matter how good your program is or how many similar programs are out there.
    I am... Black_Fox... my discontinued benchmark
    "No one involved in computers would ever say that a certain amount of memory is enough for all time? I keep bumping into that silly quotation attributed to me that says 640K of memory is enough. There's never a citation; the quotation just floats like a rumor, repeated again and again." -- Bill Gates

  6. The Following User Says Thank You to Black_Fox For This Useful Post:

    Cyan (30th April 2013)

  7. #5
    Member
    Join Date
    Mar 2013
    Location
    Worldwide
    Posts
    456
    Thanks
    46
    Thanked 164 Times in 118 Posts
    First, why are you polluting my thread and me, even if it is not your problem, second reread your quotation "... the quotation just floats like a rumor, repeated again and again".

  8. #6
    Member
    Join Date
    Sep 2008
    Location
    France
    Posts
    856
    Thanks
    447
    Thanked 254 Times in 103 Posts
    You seem to be surprised that seasoned O.S. developers judge GPL license violation a big enough issue to look at infringers with a cautionary eye.
    Yes, it is important you comply with the terms of GPL License, you can't expect to be accepted as "peer" with this attitude.
    "It is not your problem" is not a good enough defense.

  9. The Following User Says Thank You to Cyan For This Useful Post:

    Black_Fox (8th May 2013)

  10. #7
    Member
    Join Date
    Sep 2008
    Location
    France
    Posts
    856
    Thanks
    447
    Thanked 254 Times in 103 Posts
    (*double post*)
    Last edited by Cyan; 1st May 2013 at 21:18.

  11. #8
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    Input ramdisk:
    299,627,772 bytes, HTML

    Output ramdisk:
    299,627,996 bytes, 0.221 sec. - 0.290 sec., FreeArc - 0
    299,627,842 bytes, 0.279 sec. - 0.263 sec., WinRAR - 0
    298,901,696 bytes, 2.837 sec. - 0.522 sec., WinZpaq - 0
    95,696,597 bytes, 0.135 sec. - 0.324 sec., Qpress - 1
    94,662,987 bytes, 0.110 sec. - 0.134 sec., LZ4 - 0
    94,087,726 bytes, 1.005 sec. - x.xxx sec., eXdupe - 1
    94,084,823 bytes, 0.115 sec. - 0.086 sec., lzturbo - 10
    88,146,671 bytes, 0.178 sec. - 0.325 sec., Qpress - 2
    85,991,359 bytes, 0.112 sec. - 0.088 sec., lzturbo - 20
    85,613,166 bytes, 0.267 sec. - 0.116 sec., lzturbo - 11
    84,171,853 bytes, 0.464 sec. - 0.244 sec., Qpress - 3
    81,684,926 bytes, 0.677 sec. - 0.116 sec., lzturbo - 12
    80,132,858 bytes, 1.305 sec. - x.xxx sec., eXdupe - 2
    79,078,051 bytes, 0.505 sec. - 0.124 sec., LZ4 - 1
    76,335,360 bytes, 0.684 sec. - 0.118 sec., LZ4 - 2
    75,842,122 bytes, 30.942 sec. - 0.117 sec., lzturbo - 19
    73,254,405 bytes, 0.255 sec. - 0.128 sec., lzturbo - 21
    68,558,832 bytes, 0.151 sec. - 0.101 sec., lzturbo - 30
    67,384,123 bytes, 1.294 sec. - 1.178 sec., WinRAR - 1
    67,361,069 bytes, 0.656 sec. - 0.132 sec., lzturbo - 22
    66,212,618 bytes, 0.441 sec. - 0.375 sec., FreeArc - 1
    62,968,564 bytes, 3.422 sec. - x.xxx sec., eXdupe - 3
    62,613,450 bytes, 3.152 sec. - 0.673 sec., WinZpaq - 1
    61,336,971 bytes, 0.286 sec. - 0.168 sec., lzturbo - 31
    60,709,511 bytes, 1.365 sec. - 2.908 sec., 7-Zip - 1
    57,875,366 bytes, 1.578 sec. - 2.671 sec., 7-Zip - 2
    55,470,065 bytes, 4.582 sec. - 2.461 sec., WinZpaq - 2
    55,413,718 bytes, 2.080 sec. - 2.486 sec., 7-Zip - 3
    54,277,773 bytes, 34.914 sec. - 0.140 sec., lzturbo - 29
    53,911,603 bytes, 0.674 sec. - 0.170 sec., lzturbo - 32
    53,741,619 bytes, 2.767 sec. - 2.365 sec., 7-Zip - 4
    51,409,259 bytes, 2.668 sec. - 0.945 sec., WinRAR - 2
    50,885,382 bytes, 1.313 sec. - 1.822 sec., Bsc - 3
    50,247,173 bytes, 1.473 sec. - 1.944 sec., FreeArc - 2
    49,957,046 bytes, 3.641 sec. - 0.928 sec., WinRAR - 3
    49,518,664 bytes, 4.500 sec. - 0.922 sec., WinRAR - 4
    49,372,114 bytes, 5.081 sec. - 0.922 sec., WinRAR - 5
    49,241,323 bytes, 2.527 sec. - 1.877 sec., FreeArc - 3
    47,857,086 bytes, 3.609 sec. - 2.795 sec., FreeArc - 4
    45,368,587 bytes, 40.598 sec. - 0.168 sec., lzturbo - 39
    45,349,234 bytes, 1.387 sec. - 2.008 sec., Bsc - 4
    44,208,725 bytes, 10.970 sec. - 10.954 sec., WinZpaq - 3
    44,089,851 bytes, 14.216 sec. - 2.043 sec., 7-Zip - 5
    43,291,614 bytes, 1.660 sec. - 2.183 sec., Bsc - 5
    42,770,168 bytes, 56.835 sec. - 0.351 sec., lzturbo - 49
    42,570,449 bytes, 28.559 sec. - 1.990 sec., 7-Zip - 6
    42,315,996 bytes, 2.016 sec. - 2.291 sec., Bsc - 6
    41,105,392 bytes, 10.219 sec. - 10.239 sec., zcm -0
    40,927,232 bytes, 3.155 sec. - 1.617 sec., Bsc - 0
    39,541,515 bytes, 70.740 sec. - 1.902 sec., 7-Zip - 7
    39,541,515 bytes, 70.669 sec. - 1.900 sec., 7-Zip - 8
    39,541,515 bytes, 70.614 sec. - 1.901 sec., 7-Zip - 9
    39,413,468 bytes, 9.693 sec. - 4.889 sec., FreeArc - 5
    39,027,759 bytes, 10.180 sec. - 10.088 sec., zcm -1
    37,306,256 bytes, 9.739 sec. - 10.031 sec., zcm -2
    36,974,548 bytes, 34.327 sec. - 34.676 sec., WinZpaq - 4
    36,720,971 bytes, 27.852 sec. - 23.379 sec., FreeArc - 6
    36,720,971 bytes, 27.925 sec. - 23.466 sec., FreeArc - 7
    36,720,971 bytes, 27.849 sec. - 23.471 sec., FreeArc - 8
    36,720,971 bytes, 27.874 sec. - 23.386 sec., FreeArc - 9
    36,053,711 bytes, 10.136 sec. - 10.054 sec., zcm -3
    35,028,838 bytes, 9.972 sec. - 9.982 sec., zcm -4
    34,318,775 bytes, 10.457 sec. - 10.630 sec., zcm -5
    33,914,376 bytes, 10.779 sec. - 10.826 sec., zcm -6
    33,738,319 bytes, 11.194 sec. - 11.288 sec., zcm -7
    32,641,962 bytes, 136.236 sec. - 132.410 sec., WinZpaq - 5

    LZ4 1.3.3 (Feb 8, 2013)
    qpress 1.1
    eXdupe 0.4.0 beta
    lzturbo 1.1 (Apr 29, 2013)
    bsc 3.1.0 (8 July, 2012)
    FreeArc 0.67 (December 12, 2012)
    RAR 5.00 beta 2 (30 Apr, 2013)
    7-Zip 9.30 alpa (2012-10-26)
    zpaq 6.24 (Apr 14, 2013)

    12 threads, 64-bit.
    Last edited by Sportman; 4th May 2013 at 13:13.

  12. The Following User Says Thank You to Sportman For This Useful Post:

    Matt Mahoney (1st May 2013)

  13. #9
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    lzturbo 40-42 where not working, depending testing that levels I maybe miss the 33 option what can be round my sweet spot.

    Somebody is guilty when judge decided, so let them start a law suit if they lost money and/or like paying lawyers hour rates.
    Cheaper is accepting released code always ends in closed source one day, don't release code or don't whine.
    After cheating or whining created karma, universe take care for this life and six next life's, there is no random, we are all responsible for our own future and indirect each other...
    Last edited by Sportman; 1st May 2013 at 03:58.

  14. #10
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    Sportman, look at this. he have used my sources, even coder numbering wasn't changed:

    Tornado compressor v0.1 (c) Bulat.Ziganshin@gmail.com 16.04.2007
    -c# -- coder (1-bytes,2-bits,4-arith), default 4
    LZTURBO 0.0.1
    1:byte-output (fastest decompression)
    2:bit-output
    3:not implemented
    4:best compression
    he have added huffman coder just a few weeks after tornado 0.3 sources with the new huffman coder was released. are you believe that all these just a coincidence? and he was developed the same things like me, just a few months later?

  15. #11
    Member
    Join Date
    Mar 2013
    Location
    Worldwide
    Posts
    456
    Thanks
    46
    Thanked 164 Times in 118 Posts
    - When the interface and the methods are similar, this doesn't mean that the code is based on.
    Freearc is also trying to mimic some functions and options from the rar archiver.
    - I admit, i have used some infos from the forum posts.
    - The extensions mentioned in your link are from the forum posts were used in conjunction with
    the libmagic to permit automatic selection of some parameters, like context bits
    for the literal coder (ex. 8 for text, 3 for jpg,...). The parameters was optimized for the maximumcompression testset.
    - The huffman coder in lzturbo was an optimized version from the paper "In-place calculation of minimum-redundancy codes"
    see : http://ww2.cs.mu.oz.au/~alistair.
    Building huffman trees is only one part, you have also the sorting part and other optimizations.


    - The big and most complicated difference is: lzturbo had and have optimal parsing with bitwise arithmetic coding.
    Until today Tornado doesn't have neither optimal parsing nor bitwise arithmetic coding, but
    you are propaging this lies on the net. see: https://code.google.com/p/lzham/wiki/statistics.


    - To my knowledge, Lzturbo was the first compressor to use suffix arrays as basis for a match
    finder. The first paper i have found was "Suffix arrays: a competitive choice for fast Lempel-Ziv compression"
    from 2008, although the implementation in lzturbo is different (lz-factorization with lcp).
    This was however dropped in the next lzturbo versions in favor of compact tries (Not binary trees).


    - Now, if you or other are convinced from the contrary, then there is another place for that, and please
    do not pollute my threads.


    P.S. sorry, i hope my english is understandable.
    Last edited by dnd; 1st May 2013 at 17:06.

  16. #12
    Member
    Join Date
    Mar 2013
    Location
    Worldwide
    Posts
    456
    Thanks
    46
    Thanked 164 Times in 118 Posts
    Sportman
    thanks very much for testing and your wise comment. Sorry, but the options -40,-41, -42 are dropped because
    arithmetic coding is inefficient in the fast modes.
    Please, if you have time, add the options for maximum compression -19,-29,-39 and -49.
    You can set the dictionary size with the option -b (Ex. lzturbo -19 -b25 file .)
    Last edited by dnd; 1st May 2013 at 16:52.

  17. #13
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    I read:

    "LZTURBO is designed from crash. There is nothing revolutionary. This is only a carrefull implementation of
    existing ideas and many many testing and optimizing hours."
    "LZTURBO is not using any idea from lzma-algorithm."

    He write "existing ideas" so order 1-bytes, 2-bits, 4-arith and/or huffman coder can be picked up by him. Do you expect people do not use improvements ideas in their own code, maybe they even copy paste the source, play with it and rewrite it in their own format what makes it with source to source matching not similar anymore.

    This same discussion you can find in Chess engine development http://www.open-chess.org/download/file.php?id=304 (PDF) with as result "The International Computer Games Association had disqualified star programmer Vasik Rajlich for plagiarism, retroactively stripped him of all titles, and banned him for life" story http://en.chessbase.com/home/TabId/211/PostId/4007791

    Where is the border is the question, reading somebody's published source code, see a good idea and program something similar is that copying? Copy paste a good idea from somebody's published source code play with it and change it for 95% is that copying?

    I wrote once in the alt.comp.compression newsgroup that I was not a fan of open source because it can cost income/job of closed source developers. Only when we switch all at the same time to open source I saw a fair path. It's a complex thing, if tomorrow people start selling bread for free, bakers lose income/job. But I agree if every baker shared his/her receipt and production method, bread could be tasting better and cost could be cut down. Now we are in a situation where both sides take each other code, open source reverse engineer closed source and make it open source and closed source take ideas from open source to use it in closed source.

  18. #14
    Member
    Join Date
    Sep 2008
    Location
    France
    Posts
    856
    Thanks
    447
    Thanked 254 Times in 103 Posts
    There is a line between learning from someone else program or source code, then implementing one's own version based on the learning,
    and copying verbatim another source code, modifying a few lines here and there overtime.

    The first one is free, has always been free, and must remain free for science to be able to progress. It basically means : you have a right to learn.
    The second one is forbidden under the most basic copyright laws.

    Btw, it's not necessary to talk about programming/code for these matters,
    similar issues can happen for books and music too, rulings are similar.

  19. #15
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    Quote Originally Posted by dnd View Post
    Please, if you have time, add the options for maximum compression -19,-29,-39 and -49.
    Added.

  20. #16
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    Same test 1 thread:

    Input ramdisk:
    299,627,772 bytes, HTML

    Output ramdisk:
    299,627,996 bytes, 0.222 sec. - 0.290 sec., FreeArc - 0
    299,627,842 bytes, 0.279 sec. - 0.262 sec., WinRAR - 0
    298,901,696 bytes, 3.492 sec. - 2.944 sec., WinZpaq - 0
    95,696,597 bytes, 0.777 sec. - 0.616 sec., Qpress - 1
    94,662,987 bytes, 0.614 sec. - 0.135 sec., LZ4 - 0
    94,084,823 bytes, 0.681 sec. - 0.165 sec., lzturbo -10
    94,077,563 bytes, 3.284 sec. - x.xxx sec., eXdupe - 1
    88,146,671 bytes, 1.281 sec. - 0.625 sec., Qpress - 2
    85,991,359 bytes, 0.648 sec. - 0.254 sec., lzturbo -20
    85,613,166 bytes, 1.140 sec. - 0.195 sec., lzturbo -11
    84,171,853 bytes, 3.527 sec. - 0.467 sec., Qpress - 3
    81,684,926 bytes, 2.712 sec. - 0.188 sec., lzturbo -12
    80,129,265 bytes, 5.474 sec. - x.xxx sec., eXdupe - 2
    79,078,051 bytes, 3.708 sec. - 0.125 sec., LZ4 - 1
    76,335,360 bytes, 4.966 sec. - 0.119 sec., LZ4 - 2
    75,842,122 bytes, 186.159 sec. - 0.185 sec., lzturbo -19
    73,254,405 bytes, 1.017 sec. - 0.265 sec., lzturbo -21
    68,558,832 bytes, 0.941 sec. - 0.611 sec., lzturbo -30
    67,383,703 bytes, 3.449 sec. - 1.180 sec., WinRAR - 1
    67,361,069 bytes, 2.630 sec. - 0.256 sec., lzturbo -22
    64,950,320 bytes, 1.849 sec. - 1.339 sec., FreeArc - 1
    62,964,629 bytes, 21.226 sec. - x.xxx sec., eXdupe - 3
    62,613,450 bytes, 8.565 sec. - 3.708 sec., WinZpaq - 1
    61,336,971 bytes, 1.238 sec. - 0.592 sec., lzturbo -31
    60,249,776 bytes, 9.637 sec. - 2.876 sec., 7-Zip - 1
    56,984,134 bytes, 10.150 sec. - 2.613 sec., 7-Zip - 2
    55,470,065 bytes, 18.529 sec. - 16.081 sec., WinZpaq - 2
    54,896,726 bytes, 11.146 sec. - 2.451 sec., 7-Zip - 3
    54,277,773 bytes, 215.211 sec. - 0.251 sec., lzturbo -29
    53,911,603 bytes, 3.026 sec. - 0.539 sec., lzturbo -32
    53,421,713 bytes, 13.348 sec. - 2.343 sec., 7-Zip - 4
    51,615,275 bytes, 6.998 sec. - 1.103 sec., tor-small -7
    51,408,601 bytes, 11.463 sec. - 0.943 sec., WinRAR - 2
    50,885,382 bytes, 7.829 sec. - 11.435 sec., Bsc - 3
    50,247,173 bytes, 6.538 sec. - 9.001 sec., FreeArc - 2
    49,956,335 bytes, 18.010 sec. - 0.928 sec., WinRAR - 3
    49,518,109 bytes, 23.677 sec. - 0.924 sec., WinRAR - 4
    49,371,577 bytes, 27.565 sec. - 0.921 sec., WinRAR - 5
    45,368,587 bytes, 250.089 sec. - 0.490 sec., lzturbo -39
    45,349,234 bytes, 7.518 sec. - 12.178 sec., Bsc - 4
    44,232,287 bytes, 18.446 sec. - 15.688 sec., FreeArc - 3
    44,208,725 bytes, 62.645 sec. - 71.047 sec., WinZpaq - 3
    43,543,934 bytes, 76.795 sec. - 2.020 sec., 7-Zip - 5
    43,291,614 bytes, 9.826 sec. - 13.258 sec., Bsc - 5
    42,770,168 bytes, 353.678 sec. - 2.041 sec., lzturbo -49
    42,315,996 bytes, 11.038 sec. - 14.326 sec., Bsc - 6
    42,008,054 bytes, 84.239 sec. - 1.971 sec., 7-Zip - 6
    40,927,232 bytes, 18.267 sec. - 10.778 sec., Bsc - 0
    40,644,114 bytes, 42,941 sec. - 43.835 sec., ZCM - 0
    39,172,351 bytes, 24.182 sec. - 20.800 sec., FreeArc - 4
    39,069,402 bytes, 113.081 sec. - 1.888 sec., 7-Zip - 9
    39,069,402 bytes, 113.010 sec. - 1.884 sec., 7-Zip - 8
    39,069,402 bytes, 112.940 sec. - 1.885 sec., 7-Zip - 7
    38,394,520 bytes, 44.508 sec. - 44.766 sec., ZCM - 1
    38,254,924 bytes, 26.343 sec. - 22.679 sec., FreeArc - 5
    36,974,548 bytes, 210.396 sec. - 221.750 sec., WinZpaq - 4
    36,720,971 bytes, 27.936 sec. - 23.480 sec., FreeArc - 9
    36,720,971 bytes, 27.897 sec. - 23.472 sec., FreeArc - 6
    36,720,971 bytes, 27.811 sec. - 23.461 sec., FreeArc - 8
    36,720,971 bytes, 27.798 sec. - 23.417 sec., FreeArc - 7
    36,240,654 bytes, 47.009 sec. - 47.267 sec., ZCM - 2
    34,267,842 bytes, 47.879 sec. - 48.238 sec., ZCM - 3
    32,641,962 bytes, 705.996 sec. - 714.149 sec., WinZpaq - 5
    32,523,430 bytes, 47.672 sec. - 47.867 sec., ZCM - 4
    31,266,152 bytes, 47.331 sec. - 47.424 sec., ZCM - 5
    30,514,076 bytes, 46.786 sec. - 47.075 sec., ZCM - 6
    30,037,304 bytes, 46.785 sec. - 46.814 sec., ZCM - 7

    LZ4 1.3.3 (Feb 8, 2013)
    qpress 1.1
    eXdupe 0.4.0 beta
    lzturbo 1.1 (Apr 29, 2013)
    bsc 3.1.0 (8 July, 2012)
    FreeArc 0.67 (December 12, 2012)
    RAR 5.00 beta 2 (30 Apr, 2013)
    7-Zip 9.30 alpa (2012-10-26)
    zpaq 6.24 (Apr 14, 2013)
    tor-small 0.4a (2008-06-02)

    1 thread, 64-bit.
    Last edited by Sportman; 4th May 2013 at 13:14.

  21. #17
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    Quote Originally Posted by dnd View Post
    Sorry, but the options -40,-41, -42 are dropped because arithmetic coding is inefficient in the fast modes
    Singe thread the old tor-small is still the best option for me, is it possible to add -33, -34 to see if one of them can beat it?
    Update: I tested with -b option and now lzturbo beat the old tor-small:

    53,911,603 bytes, 3.026 sec. - 0.539 sec., lzturbo -32
    52,659,065 bytes, 3.054 sec. - 0.564 sec., lzturbo -32 -b50
    51,615,275 bytes, 6.998 sec. - 1.103 sec., tor-small -7
    51,453,436 bytes, 3.041 sec. - 0.588 sec., lzturbo -32 -b100
    51,408,601 bytes, 11.463 sec. - 0.943 sec., WinRAR -2
    50,800,486 bytes, 3.074 sec. - 0.601 sec., lzturbo -32 -b150
    50,695,399 bytes, 3.079 sec. - 0.614 sec., lzturbo -32 -b200
    50,418,342 bytes, 3.088 sec. - 0.624 sec., lzturbo -32 -b250
    49,892,171 bytes, 3.108 sec. - 0.637 sec., lzturbo -32 -b300
    Last edited by Sportman; 4th May 2013 at 14:02.

  22. #18
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    I did some head to head compare at an other system:

    Input ramdisk:
    847,799,142 bytes HTML

    Output ramdisk:
    241,230,573 bytes 2.864 sec 2.446 sec qpress64 -L1
    237,320,976 bytes 2.473 sec 0.863 sec lzturbo -10

    221,841,837 bytes 4.579 sec 2.522 sec qpress64 -L2
    217,022,158 bytes 4.095 sec 0.820 sec lzturbo -11

    213,850,905 bytes 12.829 sec 1.959 sec qpress64 -L3
    206,937,096 bytes 10.448 sec 0.793 sec lzturbo -12

    129,680,650 bytes 25.275 sec 4.181 sec tor -7
    117,473,684 bytes 10.856 sec 2.256 sec lzturbo -32 -b1024

    1 thread

    Output ramdisk:
    241,230,573 bytes 0.542 sec 0.784 sec qpress64 -L1
    237,320,976 bytes 0.421 sec 0.601 sec lzturbo -10

    221,841,837 bytes 0.742 sec 0.788 sec qpress64 -L2
    217,022,158 bytes 0.871 sec 0.527 sec lzturbo -11

    213,850,905 bytes 1.888 sec 0.774 sec qpress64 -L3
    206,937,096 bytes 2.163 sec 0.529 sec lzturbo -12

    16 threads

    I noticed with switch from 12 to 16 threads lzturbo started to become slower in this modes.
    I did the same lzturbo -32 test also at SSD and HD RAID and speeds where almost similar.

  23. #19
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    Input ramdisk:
    8,302,331,253 bytes, IIS log file

    Output ramdisk:
    8,305,359,198 bytes, 68.77 sec. - 12.212 sec., WinZpaq - 0
    8,302,331,478 bytes, 4.615 sec. - 5.615 sec., FreeArc - 0
    8,302,331,332 bytes, 6.909 sec. - 7.541 sec., WinRAR - 0
    976,892,016 bytes, 3.461 sec. - 4.320 sec., Qpress - 1
    939,996,684 bytes, 9.349 sec. - 4.320 sec., Qpress - 3
    914,308,015 bytes, 3.523 sec. - 4.448 sec., Qpress - 2
    904,755,839 bytes, 9.081 sec. - x.xxx sec., eXdupe - 1
    902,846,089 bytes, 12.101 sec. - x.xxx sec., eXdupe - 2
    891,998,408 bytes, 1.917 sec. - 2.672 sec., LZ4 - 0
    879,153,045 bytes, 2.444 sec. - 2.879 sec., lzturbo -10
    815,525,077 bytes, 2.937 sec. - 3.137 sec., lzturbo -11
    795,755,990 bytes, 5.389 sec. - 3.143 sec., lzturbo -12
    758,629,531 bytes, 7.559 sec. - 2.536 sec., LZ4 - 1
    711,579,634 bytes, 12.566 sec. - 2.586 sec., LZ4 - 2
    699,772,701 bytes, 2.285 sec. - 2.883 sec., lzturbo -20
    595,355,652 bytes, 2.717 sec. - 3.131 sec., lzturbo -21
    582,896,735 bytes, 110.199 sec. - x.xxx sec., eXdupe - 3
    582,349,867 bytes, 16.996 sec. - 34.067 sec., 7-Zip - 1
    560,120,134 bytes, 9.337 sec. - 6.369 sec., FreeArc - 1
    542,921,797 bytes, 4.637 sec. - 3.059 sec., lzturbo -22
    536,041,383 bytes, 2.347 sec. - 2.831 sec., lzturbo -30
    527,137,467 bytes, 27.12 sec. - 13.683 sec., WinRAR - 1
    521,460,294 bytes, 71.278 sec. - 28.14 sec., WinZpaq - 2
    512,426,931 bytes, 68.521 sec. - 11.277 sec., WinZpaq - 1
    495,602,566 bytes, 17.627 sec. - 28.711 sec., 7-Zip - 2
    467,830,743 bytes, 2.837 sec. - 3.121 sec., lzturbo -31
    440,928,835 bytes, 20.903 sec. - 25.548 sec., 7-Zip - 3
    428,913,720 bytes, 4.940 sec. - 3.072 sec., lzturbo -32
    412,924,714 bytes, 24.958 sec. - 24.018 sec., 7-Zip - 4
    408,063,762 bytes, 6.938 sec. - 5.597 sec., lzturbo -32 -b512
    401,900,605 bytes, 44.661 sec. - 12.087 sec., WinRAR - 2
    388,264,017 bytes, 56.341 sec. - 12.042 sec., WinRAR - 3
    385,509,640 bytes, 65.751 sec. - 12.471 sec., WinRAR - 4
    384,389,286 bytes, 72.767 sec. - 12.102 sec., WinRAR - 5
    348,942,038 bytes, 133.852 sec. - 23.428 sec., 7-Zip - 5
    316,240,722 bytes, 11.229 sec. - 17.014 sec., Bsc - 3
    254,041,833 bytes, 9.527 sec. - 9.389 sec., FreeArc - 2
    247,163,904 bytes, 11.839 sec. - 9.829 sec., FreeArc - 3
    245,065,956 bytes, 12.185 sec. - 17.918 sec., Bsc - 4
    241,324,238 bytes, 15.135 sec. - 12.853 sec., FreeArc - 4
    223,264,850 bytes, 15.283 sec. - 19.291 sec., Bsc - 5
    222,911,356 bytes, 40.225 sec. - 16.757 sec., Bsc - 0
    215,548,686 bytes, 19.511 sec. - 20.379 sec., Bsc - 6
    205,807,524 bytes, 97.885 sec. - 29.858 sec., FreeArc - 5
    164,306,471 bytes, 112.897 sec. - 112.169 sec., ZCM - 0

    LZ4 1.3.3 (Feb 8, 2013)
    qpress 1.1
    eXdupe 0.4.0 beta
    lzturbo 1.1 (Apr 29, 2013)
    bsc 3.1.0 (8 July, 2012)
    FreeArc 0.67 (December 12, 2012)
    RAR 5.00 beta 2 (30 Apr, 2013)
    7-Zip 9.30 alpa (2012-10-26)
    zpaq 6.24 (Apr 14, 2013)

    12 threads, 64-bit.
    Last edited by Sportman; 12th May 2013 at 13:44.

  24. #20
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,612
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by Cyan View Post
    The first one is free, has always been free, and must remain free for science to be able to progress.
    Unfortunately, this is not true.
    Keyword: patents.

  25. #21
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    Input ramdisk:
    574,219,767 bytes, Genefile generate file (http://www.metacompressor.com/download/genefile.zip)

    Output ramdisk:
    576,510,967 bytes, 1.420 sec. - x.xxx sec., eXdupe - 1
    574,523,787 bytes, 4.565 sec. - 0.905 sec., WinZpaq - 0
    574,523,787 bytes, 4.605 sec. - 0.918 sec., WinZpaq - 1
    574,219,997 bytes, 0.428 sec. - 0.479 sec., FreeArc - 0
    574,219,844 bytes, 0.514 sec. - 0.492 sec., WinRAR - 0
    569,253,069 bytes, 0.326 sec. - 0.272 sec., Qpress - 1
    566,719,380 bytes, 0.625 sec. - 0.272 sec., Qpress - 2
    564,820,734 bytes, 1.116 sec. - 0.261 sec., Qpress - 3
    555,949,495 bytes, 0.317 sec. - 0.266 sec., lz4 - 0
    555,659,872 bytes, 0.263 sec. - 0.180 sec., lzturbo - 10
    555,117,967 bytes, 0.281 sec. - 0.186 sec., lzturbo - 20
    552,725,941 bytes, 0.883 sec. - 0.232 sec., lzturbo - 11
    552,406,948 bytes, 47.583 sec. - 0.233 sec., lzturbo - 39
    551,242,438 bytes, 0.935 sec. - 0.230 sec., lzturbo - 21
    550,417,101 bytes, 3.407 sec. - 0.226 sec., lzturbo - 12
    550,244,614 bytes, 40.689 sec. - 0.226 sec., lzturbo - 19
    549,945,321 bytes, 4.353 sec. - 0.235 sec., lzturbo - 22
    549,387,572 bytes, 0.979 sec. - 0.265 sec., lz4 - 1
    548,658,007 bytes, 1.103 sec. - 0.265 sec., lz4 - 2
    546,197,782 bytes, 43.361 sec. - 0.232 sec., lzturbo - 29
    526,091,198 bytes, 0.334 sec. - 0.203 sec., lzturbo - 30
    514,195,990 bytes, 1.063 sec. - 0.304 sec., lzturbo - 31
    495,020,486 bytes, 4.559 sec. - 0.369 sec., lzturbo - 32
    494,995,260 bytes, 23,196 sec. - 2.286 sec., lzturbo - 32 -b1024
    492,180,647 bytes, 3.284 sec. - x.xxx sec., eXdupe - 2
    482,991,289 bytes, 3.276 sec. - 1.362 sec., FreeArc - 2
    481,990,904 bytes, 15.412 sec. - 20.211 sec., 7-Zip - 4
    481,881,478 bytes, 7.771 sec. - x.xxx sec., eXdupe - 3
    481,222,066 bytes, 10.381 sec. - 4.961 sec., WinRAR - 2
    481,165,585 bytes, 13.735 sec. - 20.399 sec., 7-Zip - 3
    480,753,350 bytes, 11.940 sec. - 4.975 sec., WinRAR - 3
    480,676,947 bytes, 12.053 sec. - 4.964 sec., WinRAR - 4
    480,645,825 bytes, 12.121 sec. - 4.977 sec., WinRAR - 5
    480,638,352 bytes, 1.191 sec. - 1.040 sec., FreeArc - 1
    480,512,360 bytes, 4.401 sec. - 4.861 sec., WinRAR - 1
    480,027,854 bytes, 10.107 sec. - 20.588 sec., 7-Zip - 2
    475,893,088 bytes, 7.009 sec. - 20.853 sec., 7-Zip - 1
    474,299,226 bytes, 9.010 sec. - 12.428 sec., Bsc - 3
    474,111,478 bytes, 9.905 sec. - 11.033 sec., Bsc - 4
    474,025,178 bytes, 13.919 sec. - 11.187 sec., Bsc - 5
    473,975,100 bytes, 17.061 sec. - 11.551 sec., Bsc - 6
    473,924,354 bytes, 13.656 sec. - 9.934 sec., Bsc - 0
    471,286,151 bytes, 18.563 sec. - 14.456 sec., WinZpaq - 2
    465,773,883 bytes, 8.788 sec. - 3.176 sec., FreeArc - 3
    464,377,531 bytes, 57.342 sec. - 2.631 sec., lzturbo - 49
    458,673,080 bytes, 20.756 sec. - 20.868 sec., 7-Zip - 5
    458,596,015 bytes, 69.758 sec. - 24.956 sec., FreeArc - 5
    456,073,977 bytes, 12.772 sec. - 3.243 sec., FreeArc - 4
    450,442,417 bytes, 45.681 sec. - 48.531 sec., ZCM - 0

    LZ4 1.3.3 (Feb 8, 2013)
    qpress 1.1
    eXdupe 0.4.0 beta
    lzturbo 1.1 (Apr 29, 2013)
    bsc 3.1.0 (8 July, 2012)
    FreeArc 0.67 (December 12, 2012)
    RAR 5.00 beta 2 (30 Apr, 2013)
    7-Zip 9.30 alpa (2012-10-26)
    zpaq 6.24 (Apr 14, 2013)

    12 threads, 64-bit.
    Last edited by Sportman; 12th May 2013 at 13:34.

  26. #22
    Member
    Join Date
    Sep 2008
    Location
    France
    Posts
    856
    Thanks
    447
    Thanked 254 Times in 103 Posts
    Quote Originally Posted by m^2 View Post
    Unfortunately, this is not true.
    Keyword: patents.
    Oh right, indeed,
    patent is about getting a monopoly on an implementation,
    so a different implementation (of the same concept) is supposed to circumvent a patent,

    but these last decades, it's not so clear,
    patent-friendly courts in US seem to have enlarged patent to a monopoly on abstract ideas.

    But this is a whole different story.
    Maybe we should not pollute this thread with this. It's named Lzturbo for a reason.

  27. #23
    Member
    Join Date
    Mar 2013
    Location
    Worldwide
    Posts
    456
    Thanks
    46
    Thanked 164 Times in 118 Posts
    From the previous thread.
    I updated the benchmark for lzturbo to include this version. Since there is some controversy over the source code, I included a link to this discussion.
    http://mattmahoney.net/dc/text.html#2174


    ----------------------------
    Hey matt,
    thank you for testing. First there is no controversy about the source code, because there are no proofs.
    The only explanation you can find, is that the interface is similar.
    Think! lzturbo has optimal parsing and bitwise arithmetic coding.
    I think some people are only interested in this GPL discussion or because
    lzturbo is closed source. May be bad news are good news.


    Unfortunately you have not published my results, i have mailed to you.
    Lzturbo beats lzma and is pareto frontier on decompression in LTCB.


    Name of program and version: lzturbo v1.1
    Homepage: http://sites.google.com/site/powturbo/
    Compressed size of enwik8 and enwik9:
    enwik8 : 24416777
    enwik9 : 194681713 ctime:1920s dtime: 9 seconds
    Approximate memory used:14.7Gb


    Options to obtain best compression of enwik9: -49 -b1000 -p0
    Size of the decompressor as a zip file (smaller of source or executable): 110,670 bytes


    Description of the test machine (processor, memory, operating system):
    CPU: I7-2600K at 4,6GHz.
    RAM: 16GB Ram.
    OS: Ubutuntu 13.04.
    Last edited by dnd; 15th May 2013 at 10:39.

  28. #24
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    I updated the benchmark with your results. I couldn't test -49 -b1000 -p0 with 4 GB memory. It is on the Pareto frontier for decompression speed now.

    Just IMHO, I think the controversy could be resolved by making it open source, regardless of who is right. When you google lzturbo, the discussions on this thread come up as the third link, which reflects badly on the program whether or not it is true.

  29. #25
    Member
    Join Date
    Mar 2013
    Location
    Worldwide
    Posts
    456
    Thanks
    46
    Thanked 164 Times in 118 Posts
    Thank you for spending your valuable time and testing lzturbo.
    Thank's for your recommendation, i'm thinking about that.

  30. #26
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    page https://sites.google.com/site/powturbo/downloads says that it contains v3 of lzturbo 1.1. so, i ask people downloaded lzturbo earlier to share it here - i need to compare these subversions of program

    the same applies to the older lzturbo versions - since Hamid removes all the old programs, we need to collect them here at forum
    Last edited by Bulat Ziganshin; 16th May 2013 at 13:13.

  31. #27
    Member
    Join Date
    Mar 2013
    Location
    Worldwide
    Posts
    456
    Thanks
    46
    Thanked 164 Times in 118 Posts
    Are you the GOD of the forum?
    Someone like you, will be banned on other forums.
    You are only propagating lies without any proofs. You are repeating the same phrases:
    "do you think this..." or "do you think that..." since years.

    If you are a respectable person then you will not start threads against lzturbo without thinking.


    You have mentionend the match finder and the tornado huffcoder and now i have posted the source
    code used in lzturbo since the first version. You can easly find the similarities with your posted
    (illegally) disassembled code.


    You are using the same tricks from the patent trolls. You are describing every thing from your
    software. You are also suggesting to use the ideas from your software.
    And then if someone implement similar things, you can claim later he is copying your code.


    If you think your tornado is good, why nobody is using it, even if it is free?

    You are abusing the forum for advertising freearc since years. Why are you not using your
    own homepage for that? Only because freearc is GPL or free is not a raison.


    What are your contributions on the forum? Discussing only about simple things and your software?
    Why are you not discussing with Shelwien and Matt about complex problems like context mixing and similar things.


    Look for example at your DICT program (> 1500 lines!). This is one of the most stupid piece of software i have ever seen.
    The program is trying splitting a text using several steps for collecting alphanumeric words. This is for me
    a mistery why the program is not simply searching for the next non-alpha character like XWRT and other programs does?


    This will be my last post about this issue. It is ridiculous to discuss about a simple lz77 and repeating
    the same things.


    I have nothing to prove.

  32. #28
    Tester
    Black_Fox's Avatar
    Join Date
    May 2008
    Location
    [CZE] Czechia
    Posts
    471
    Thanks
    26
    Thanked 9 Times in 8 Posts
    I see no problem in having a FreeArc topic on the forum. By now there is about the same count of FA topics as there are of those about LZTurbo. It's better to coordinate translators' effort than some mailing group.

    The beauty of opensource is that even if you find something to be "the most stupid piece of software you have ever seen", you send the author a patch and he can improve it thanks to your advice. But still, people tend to keep the top tricks to themselves

    Thank you for showing some of the sources and saying more about lzturbo, this is the right step, because it may help some third person - no matter who is guilty. Insults never helped achieve anything positive.
    I am... Black_Fox... my discontinued benchmark
    "No one involved in computers would ever say that a certain amount of memory is enough for all time? I keep bumping into that silly quotation attributed to me that says 640K of memory is enough. There's never a citation; the quotation just floats like a rumor, repeated again and again." -- Bill Gates

  33. The Following User Says Thank You to Black_Fox For This Useful Post:

    load (16th May 2013)

  34. #29
    Member
    Join Date
    May 2013
    Location
    ARGENTINA
    Posts
    54
    Thanks
    62
    Thanked 13 Times in 10 Posts
    .................................................. .................................................. ......
    Last edited by GOZARCK; 5th November 2013 at 20:29.

Similar Threads

  1. LZTURBO
    By dnd in forum Data Compression
    Replies: 9
    Last Post: 15th May 2013, 05:21
  2. The worst case for LZTurbo?
    By m^2 in forum Data Compression
    Replies: 8
    Last Post: 16th April 2009, 00:24
  3. LZTURBO 0.9 parallel compressor
    By donotdisturb in forum Forum Archive
    Replies: 18
    Last Post: 6th March 2008, 01:23
  4. LZTURBO 0.1 parallel compressor
    By donotdisturb in forum Forum Archive
    Replies: 5
    Last Post: 7th October 2007, 22:44
  5. LZTURBO 0.0.1 - Experimental version
    By donotdisturb in forum Forum Archive
    Replies: 50
    Last Post: 29th August 2007, 20:57

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •