Page 1 of 3 123 LastLast
Results 1 to 30 of 61

Thread: NEW MOC BENCHMARK

  1. #1
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts

    NEW MOC BENCHMARK

    MONSTER OF COMPRESSION
    BENCHMARK

    By Nania Francesco Antonio.

    This benchmark ranks lossless data compression programs by the compressed size for 2.046.886.013 bytes:

    RULES:
    1) No external dictionaries;
    2) No Memory limit;
    3) No errors!
    4) Solid Mode only for archivers!
    http://heartofcomp.altervista.org/MOC/MOC.htm

  2. #2
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    Also no slow programs like paq I guess

  3. #3
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts

    Cool Hi Matt!

    I have decided to make a will all the programs understood paq, zpaq etc!

    Hi Matt!

  4. #4
    Member Alexander Rhatushnyak's Avatar
    Join Date
    Oct 2007
    Location
    Canada
    Posts
    232
    Thanks
    38
    Thanked 80 Times in 43 Posts
    Last edited by Alexander Rhatushnyak; 23rd April 2010 at 23:00.

  5. #5
    Member
    Join Date
    Jun 2009
    Location
    Kraków, Poland
    Posts
    1,471
    Thanks
    26
    Thanked 120 Times in 94 Posts
    Some postprocessing to those BMP's should be applied. I mean something like that: http://www.cs.tut.fi/~foi/SA-DCT/

  6. #6
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,612
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Quote Originally Posted by Alexander Rhatushnyak View Post
    All bmp images in the test set are ex-JPEGs, there is no exception.
    It's very hard to find uncompressed big images on the web, isn't it?
    And there is still no special model for decompressed JPEGs in paq8*, is there?
    Yeah, I think this part is really flawed.
    IMO it would be OK if one of many files would be a JPEG, but all of them?
    Pointless.

  7. #7
    Member
    Join Date
    Jun 2009
    Location
    Kraków, Poland
    Posts
    1,471
    Thanks
    26
    Thanked 120 Times in 94 Posts
    Here are some PNG samples:
    http://r0k.us/graphics/kodak/

  8. #8
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts

    New MOC update is online

    New MOC update is online
    added:
    - CSC32
    - CSC32a2
    - CSC32a3
    - RZM 0.07H
    - YZX v.0.03
    - BSC 1.03 (2.x in the next release)

    http://heartofcomp.altervista.org/MOC/MOCA.htm

  9. #9
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts

    New MOC update is online

    Corrected more error's in compression/decompression efficiency ratings!

  10. #10
    Member Fu Siyuan's Avatar
    Join Date
    Apr 2009
    Location
    Mountain View, CA, US
    Posts
    176
    Thanks
    10
    Thanked 17 Times in 2 Posts
    Hi Nania, Can you write down the weighted size of each kinds of files in corpus like in Comp.Ratings. I can't download all of the test files but I want to know how the size of each part.

  11. #11
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts

    HI!

    the question in my language is strange however if have understood well this is the MOC composition!

    176 Files

    AUDIO ->168.426.586
    VIDEO-> 212.669.132
    BIN-> 211.709.376
    TXT-> 223.596.523
    XML-> 61.590.668
    BMP-> 214.285.240
    DAT->65.006.080
    DLL-> 23.313.880
    EXE-> 21.501.494
    ISO-> 573.026.304
    JPG-> 16.736.333
    PDF-> 15.087.979
    GAME-> 61.742.498
    TOOLS->178.045.440

  12. #12
    Member Raymond_NGhM's Avatar
    Join Date
    Oct 2008
    Location
    UK
    Posts
    51
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Thumbs up

    Hi Nania Francesco...
    try to add mem.model 4 for CCM & CCMX

  13. #13
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts

    New MOC update is online!

    New MOC update is online
    added:
    - LZTurbo 0.95
    - CSC32a4
    - Thor 0.96
    - YZX v.0.04
    - BSC 2.1
    http://heartofcomp.altervista.org/MOC/MOCA.htm

  14. #14
    Member
    Join Date
    Jun 2010
    Location
    Granada, Spain
    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Nania Francesco can you update Nanozip 0.08 alpha? Thanks a lot.

  15. #15
    Programmer
    Join Date
    May 2008
    Location
    denmark
    Posts
    94
    Thanks
    0
    Thanked 2 Times in 2 Posts
    Where is qpress?

  16. #16
    Member
    Join Date
    May 2008
    Location
    Germany
    Posts
    410
    Thanks
    37
    Thanked 60 Times in 37 Posts
    can you please test the new bsc 2.2.0 with the -m2 switch (ST5/Schindler Transformation order 5) too ?

  17. #17
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts

    Lightbulb New MOC update is online

    New MOC update is online
    added:

    - NANOZIP 0.8A
    - CSC32A6
    - BSC 2.2.0

  18. #18
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    New MOC update is online
    added:
    - lzham
    - csc32 final
    - crush
    - 7zip 9.20

  19. #19
    Member
    Join Date
    May 2008
    Location
    Germany
    Posts
    410
    Thanks
    37
    Thanked 60 Times in 37 Posts
    @nania: thank you for the new release of the moc - benchmark

    if you can, please test in the next release too

    a) the actual version of bsc from http://libbsc.com/Documents/bsc-2.6.1-x86.zip

    with

    method st5: bsc261 e inputfile outputfile -m5 -cp -b328 -t -M128
    or bsc261 e inputfile outputfile -m5 -cp -b328 -t -M192

    method st6: bsc261 e inputfile outputfile -m6 -cp -b328 -t -M128
    or bsc261 e inputfile outputfile -m6 -cp -b328 -t -M192

    and
    b) the actual version of zp from http://mattmahoney.net/dc/zp.103.zip

    with

    zp103 -m4 -t2 -b128 inputfile
    or
    zp103 -m4 -t2 -b512 inputfile

    Do you plan to test the new winrar 4 too ?

    Do you plan further development of your wonderful "rings 1.3" / "rings 1.5c" ?

    In my tests they are fast with a good compression.

    Does you even test these versions within your actual benchmark?

    best regards

  20. #20
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    @joerg

    After a year that I have taken as pause sure I try to test many programs and between those which you have name! Rings will become a archiver, is working more like formulation of the program that like realization!
    between least a new release of a my program Flashzip!
    Hi!

  21. #21
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    New Moc Update is online

    Added:
    - ZP v.1.03
    - LPAQ9
    - FP8 (New best ... ops ... beast of MOC)
    Last edited by Nania Francesco; 3rd June 2011 at 23:08.

  22. #22
    Member
    Join Date
    May 2008
    Location
    Germany
    Posts
    410
    Thanks
    37
    Thanked 60 Times in 37 Posts
    @nania: thank you for the test

    but - i think zp can compress better, if we give a little bit more memory
    for example:

    zp103 -m4 -t2 -b128 inputfile
    or
    zp103 -m4 -t1 -b128 inputfile

    can you please test this ?

    best regards

  23. #23
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    Actually zp103 -m4 -b2000 because memory does not depend on block size for -m3 and -m4. zp does not have a solid mode so it might compress better as a tar file.

  24. #24
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts

    Ok ! help me !

    Best settings for ZP (max 2GB of memory) ?
    Tar mode .. in the future !

  25. #25
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    Memory usage for zp:
    -m1 and -m2 use 5x block size per thread after rounding up block size to a power of 2. So -m1 -b32 -t2 would use 320 MB.
    -m3 uses 111 MB per thread regardless of block size.
    -m4 uses 246 MB per thread regardless of block size.
    Number of threads defaults to number of cores. Change with -t.

    Block size defaults to -b32 for all compression modes. Maximum is -b2047. Smaller blocks always make compression worse but save memory for -m1 and -m2. The only reason for having blocks for -m3 and -m4 is to allow multithreading. To use all cores use a block size equal to total size divided by number of cores.

    Also, I messed up on version numbers. zp 1.03 is newer than zp 2.00. zp 1.03 is derived from pzpaq. Old zp 2.00 is not multithreaded.

    So for best compression: zp -m4 -b2047

  26. #26
    Member
    Join Date
    May 2008
    Location
    Germany
    Posts
    410
    Thanks
    37
    Thanked 60 Times in 37 Posts
    @Matt Mahoney:
    ***
    For best speed,
    the block size should be the input size divided by the number of cores.
    For best compression, use one block.
    ***

    what about this?
    - better compression with:

    zp -m4 -t1 -b2047 inputfile

  27. #27
    Member Alexander Rhatushnyak's Avatar
    Join Date
    Oct 2007
    Location
    Canada
    Posts
    232
    Thanks
    38
    Thanked 80 Times in 43 Posts
    Why is LPAQ9m not tested with options 8 and 9 ?
    The algorithm is CM. And there are two authors, BTW.

    It's likely that LPAQ8 will compress better than LPAQ9m,
    as the latter is tuned for enwik9 and should be used with DRT preprocessor.

    All uncompressed images in the test set are ex-JPEGs, aren't they?
    Do you plan to add some undistorted images?
    Last edited by Alexander Rhatushnyak; 4th June 2011 at 02:46.

    This newsgroup is dedicated to image compression:
    http://linkedin.com/groups/Image-Compression-3363256

  28. #28
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts

    thanks

    Thank Matt
    - I correct the errors in the next release.
    - I single text l' last version of the program
    - DRT is an external preprocessor that creates an external dictionary!
    - bmp or pgmthey come from JPEG, does not seem to me that it changes the result very! - Alexander I admire you a lot, because does not create yours archiver, even LZ77 based?



    Best regards!

  29. #29
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    Quote Originally Posted by joerg View Post
    @Matt Mahoney:
    ***
    For best speed,
    the block size should be the input size divided by the number of cores.
    For best compression, use one block.
    ***

    what about this?
    - better compression with:

    zp -m4 -t1 -b2047 inputfile
    -t1 would only compress slower without any effect on compression. The reason for having it would be to reduce memory when using BWT modes -m1 or -m2 with large blocks. -m4 uses 246 MB per thread so probably the default -t would work. Also, -t would only have an effect if you compress more than one file at a time or if the file is larger than 2 GB because otherwise you only get 1 thread anyway.

  30. #30
    Member Alexander Rhatushnyak's Avatar
    Join Date
    Oct 2007
    Location
    Canada
    Posts
    232
    Thanks
    38
    Thanked 80 Times in 43 Posts
    Quote Originally Posted by Nania Francesco
    Alexander I admire you a lot, because does not create yours archiver, even LZ77 based?
    sorry, I didn't understand this question.
    Quote Originally Posted by Nania Francesco
    - bmp or pgmthey come from JPEG, does not seem to me that it changes the result very!
    Can you see squares in the attached image?
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	exp18home_nasa_big__800pcnt.png 
Views:	326 
Size:	30.6 KB 
ID:	1585  

    This newsgroup is dedicated to image compression:
    http://linkedin.com/groups/Image-Compression-3363256

Page 1 of 3 123 LastLast

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •