Results 1 to 17 of 17

Thread: libzling updates!

  1. #1
    Member RichSelian's Avatar
    Join Date
    Aug 2011
    Location
    Shenzhen, China
    Posts
    156
    Thanks
    18
    Thanked 50 Times in 26 Posts

    Red face libzling updates!

    https://github.com/richox/libzling/tree/20160105

    libzling hasn't been updated for more than a year because of my busy work this version brings with a simplified MTF model and better compression ratio (without hurting too much speed).
    waiting for benchmark

  2. The Following User Says Thank You to RichSelian For This Useful Post:

    Stephan Busch (7th January 2016)

  3. #2
    Member jibz's Avatar
    Join Date
    Jan 2015
    Location
    Denmark
    Posts
    114
    Thanks
    91
    Thanked 69 Times in 49 Posts
    Nice. I rebased my pull request, in case you want to consider it.

  4. The Following User Says Thank You to jibz For This Useful Post:

    RichSelian (5th January 2016)

  5. #3
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    872
    Thanks
    457
    Thanked 175 Times in 85 Posts
    could you please provide a compiled win64 version?

  6. #4
    Member RichSelian's Avatar
    Join Date
    Aug 2011
    Location
    Shenzhen, China
    Posts
    156
    Thanks
    18
    Thanked 50 Times in 26 Posts
    Quote Originally Posted by Stephan Busch View Post
    could you please provide a compiled win64 version?
    here are binaries for win32/win64 (compiled from source: https://github.com/richox/libzling/tree/20160107), compiled with mingw-w64 under archlinux.
    Attached Files Attached Files
    Last edited by RichSelian; 7th January 2016 at 18:23.

  7. The Following User Says Thank You to RichSelian For This Useful Post:

    Stephan Busch (7th January 2016)

  8. #5
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    872
    Thanks
    457
    Thanked 175 Times in 85 Posts
    the program (both builds) crashes when trying to compress..
    the commandline was:

    zling_demo_20160107_win64 e app.tar app.zling
    zling_demo_20160107_win64 e4 app.tar app.zling

  9. #6
    Member RichSelian's Avatar
    Join Date
    Aug 2011
    Location
    Shenzhen, China
    Posts
    156
    Thanks
    18
    Thanked 50 Times in 26 Posts
    Quote Originally Posted by Stephan Busch View Post
    the program (both builds) crashes when trying to compress..
    the commandline was:

    zling_demo_20160107_win64 e app.tar app.zling
    zling_demo_20160107_win64 e4 app.tar app.zling
    can you provide app.tar?

  10. #7
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    872
    Thanks
    457
    Thanked 175 Times in 85 Posts
    The crash also happens on the other Testsets I simply put in a Tar with 7-Zip
    The App testset p.ex. can be downloaded here:

    https://drive.google.com/file/d/0ByL...ew?usp=sharing

  11. #8
    Member RichSelian's Avatar
    Join Date
    Aug 2011
    Location
    Shenzhen, China
    Posts
    156
    Thanks
    18
    Thanked 50 Times in 26 Posts
    Quote Originally Posted by Stephan Busch View Post
    The crash also happens on the other Testsets I simply put in a Tar with 7-Zip
    The App testset p.ex. can be downloaded here:

    https://drive.google.com/file/d/0ByL...ew?usp=sharing
    could you send it to my email: richselian at qq.com? google is banned in china

  12. #9
    Member jibz's Avatar
    Join Date
    Jan 2015
    Location
    Denmark
    Posts
    114
    Thanks
    91
    Thanked 69 Times in 49 Posts
    It's crashing here on binary input files in libzling_lz.cpp line 266, because node is 65535.

  13. #10
    Member
    Join Date
    Jul 2013
    Location
    United States
    Posts
    194
    Thanks
    44
    Thanked 140 Times in 69 Posts
    I already mentioned this in a bug report, but FWIW it fails with testimage.ppm from compfuzz. It's likely they are all the same issue…

  14. #11
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    872
    Thanks
    457
    Thanked 175 Times in 85 Posts
    Here is an alternate download link:
    http://www.compressionratings.com/fi...zechart_app.7z

  15. #12
    Member RichSelian's Avatar
    Join Date
    Aug 2011
    Location
    Shenzhen, China
    Posts
    156
    Thanks
    18
    Thanked 50 Times in 26 Posts
    Quote Originally Posted by nemequ View Post
    I already mentioned this in a bug report, but FWIW it fails with testimage.ppm from compfuzz. It's likely they are all the same issue…
    fixed.
    Attached Files Attached Files

  16. The Following 2 Users Say Thank You to RichSelian For This Useful Post:

    Dimitri (8th January 2016),xezz (8th January 2016)

  17. #13
    Member
    Join Date
    Mar 2013
    Location
    Worldwide
    Posts
    456
    Thanks
    46
    Thanked 164 Times in 118 Posts

    TurboBench - Efficient: zlib,6 class compressors (binary file)

    TurboBench update: - Fri Jan 8 2016 - All latest versions
    CPU: Sandy bridge i7-2600k at 4.2GHz, gcc 5.2, ubuntu 15.10, single thread.
    turbobench -eEFFICIENT app3.tar

    Code:
          C Size  ratio%     C MB/s     D MB/s   Name            (bold = pareto) MB=1.000.0000
        36944243    36.9      69.99    1411.77   lzturbo 32      
        46492542    46.4     191.69    1186.68   lzturbo 31
        46859284    46.8      52.13     867.66   zstd 9          
        47483927    47.4      32.17     151.06   zling 4         
        47758321    47.7      37.08     347.02   brotli 5        
        48827927    48.8     124.02     811.12   zstd 5          
        49918739    49.9     279.78    1067.58   lzturbo 30a     
        49962678    49.9      35.51     293.56   zlib 6          
        50014904    50.0      56.38    1921.87   lzturbo 22      
        50152281    50.1      43.58     291.07   zlib 5          
        50310907    50.3     301.56    1073.47   lzturbo 30      
        52231176    52.2      58.01    1413.25   lz5 6           
        52597358    52.5     262.97    2057.56   lzturbo 21      
        53324558    53.3     152.19    1309.95   lz5 5
    lzturbo compress better, more than 6 times faster, decompress more than 9 times faster than zling.
    Much better compression, faster compression, decompression speed on par with the byte coding lz5

  18. #14
    Member RichSelian's Avatar
    Join Date
    Aug 2011
    Location
    Shenzhen, China
    Posts
    156
    Thanks
    18
    Thanked 50 Times in 26 Posts
    Quote Originally Posted by dnd View Post
    TurboBench update: - Fri Jan 8 2016 - All latest versions
    CPU: Sandy bridge i7-2600k at 4.2GHz, gcc 5.2, ubuntu 15.10, single thread.



    Code:
          C Size  ratio%     C MB/s     D MB/s   Name            (bold = pareto) MB=1.000.0000
        36944243    36.9      69.99    1411.77   lzturbo 32      
        46492542    46.4     191.69    1186.68   lzturbo 31
        46859284    46.8      52.13     867.66   zstd 9          
        47483927    47.4      32.17     151.06   zling 4         
        47758321    47.7      37.08     347.02   brotli 5        
        48827927    48.8     124.02     811.12   zstd 5          
        49918739    49.9     279.78    1067.58   lzturbo 30a     
        49962678    49.9      35.51     293.56   zlib 6          
        50014904    50.0      56.38    1921.87   lzturbo 22      
        50152281    50.1      43.58     291.07   zlib 5          
        50310907    50.3     301.56    1073.47   lzturbo 30      
        52231176    52.2      58.01    1413.25   lz5 6           
        52597358    52.5     262.97    2057.56   lzturbo 21      
        53324558    53.3     152.19    1309.95   lz5 5
    lzturbo compress better, more than 6 times faster, decompress more than 9 times faster than zling.
    Much better compression, faster compression, decompression speed on par with the byte coding lz5
    libzling is lacking some dynamic detecting features. it performs not that good on low redundant binary data, which i believe tornado can do better.

  19. #15
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    Finally got around to testing. Nice improvement.
    http://mattmahoney.net/dc/text.html#2595
    http://mattmahoney.net/dc/silesia.html

  20. #16
    Programmer
    Join Date
    May 2008
    Location
    PL
    Posts
    307
    Thanks
    68
    Thanked 166 Times in 63 Posts
    I've included the newest commit of libzling (ee058fad67d83550f268bfc50ee7824edc74fda2) to lzbench and now I have crash during encoding at level 2 with my "win81" file (https://docs.google.com/uc?id=0BwX7d...xport=download).

    If you are not able to generate the bug you can use "lzbench.exe -v9 -ezling win81" compiled from:
    https://github.com/inikep/lzbench/commits/dev

  21. #17
    Member jibz's Avatar
    Join Date
    Jan 2015
    Location
    Denmark
    Posts
    114
    Thanks
    91
    Thanked 69 Times in 49 Posts
    I took a quick look at the code and posted a patch that seems to stop it crashing to your GitHub issue, hope it helps.

Similar Threads

  1. libzling updates: 20140430
    By RichSelian in forum Data Compression
    Replies: 3
    Last Post: 5th May 2014, 18:03
  2. libzling updates!
    By RichSelian in forum Data Compression
    Replies: 3
    Last Post: 23rd April 2014, 10:23
  3. libzling updates! (20140324)
    By RichSelian in forum Data Compression
    Replies: 3
    Last Post: 24th March 2014, 21:21
  4. libzling updates!
    By RichSelian in forum Data Compression
    Replies: 15
    Last Post: 3rd March 2014, 05:28
  5. libzling is coming soon!
    By RichSelian in forum Data Compression
    Replies: 5
    Last Post: 4th February 2014, 02:06

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •