Results 1 to 18 of 18

Thread: ISO images compression

  1. #1
    Member Surfer's Avatar
    Join Date
    Mar 2009
    Location
    oren
    Posts
    203
    Thanks
    18
    Thanked 7 Times in 1 Post

    Post ISO images compression

    I've done some tests with fast ISO images compression.
    My notebook so slow, that I can't do more tests.

    Original 196 438 016 bytes.
    Compressed ISZ image 177 899 968 bytes.
    Optimized with UltraISO 131 461 120 bytes.
    Original + rep 41 837 688 bytes.
    Original + srep295a 41 834 807 bytes.
    Original + srep295a -a1 41 817 682 bytes.
    Optimized + rep -b256 41 843 851 bytes.
    Optimized + srep295a 41 836 829 bytes.
    Optimized + srep295a -a1 41 836 829 bytes.

    ECM results are really weak and doesn't included.

    If anybody can help, links to images - original and optimized.
    It would be nice for compression options and other suggestions.

  2. #2
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,612
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Could you briefly describe the ISO contents?

  3. #3
    Member Surfer's Avatar
    Join Date
    Mar 2009
    Location
    oren
    Posts
    203
    Thanks
    18
    Thanked 7 Times in 1 Post
    Some equal files, most of them executables, dlls and msi packages .

  4. #4
    Member
    Join Date
    May 2008
    Location
    Kuwait
    Posts
    301
    Thanks
    26
    Thanked 22 Times in 15 Posts
    ECM does work well on plain iso its main work is on BIn of dumped image.

    Your testing result shows that UltraIso (linking duplicates) is practical for daily use but REP & SREP can overcome it for compression. can please add timing if possible.

  5. #5
    Member
    Join Date
    Apr 2009
    Location
    here
    Posts
    202
    Thanks
    165
    Thanked 109 Times in 65 Posts
    Optimized with UltraISO AND compressed ISZ 117'736'366
    7-z (max lzma2) 38'849'546
    RAR (normal) 173'797'423

    must have done something wrong with winrar, seems far too bad.

  6. #6
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    539
    Thanks
    192
    Thanked 174 Times in 81 Posts
    Quote Originally Posted by load View Post
    must have done something wrong with winrar, seems far too bad.
    I don't think so. This is because WinRAR doesn't have a large dictionary and so isn't very good for deduplication. The ISO contains several copies of one 19 MB file (FERInstall.EXE) and it also appears uncompressed in one of the MSI files. IIRC, WinRAR won't see those matches if they are several MB apart.

    EDIT: You can also see this effect when using different modes for 7-Zip (used 7-Zip GUI and optimized ISO). Also note the quite large difference even for Maximum mode when SREP is used before. This most likely means even Maximum didn't catch all matches. Didn't try Ultra so far, might give results closer to SREP+Max.

    Code:
    Fastest  117.966.588
    Fast     117.306.595
    Normal   115.604.781
    Max       39.320.358
    SREP+Max  37.810.870
    Last edited by schnaader; 21st March 2011 at 20:49.
    http://schnaader.info
    Damn kids. They're all alike.

  7. #7
    Member Surfer's Avatar
    Join Date
    Mar 2009
    Location
    oren
    Posts
    203
    Thanks
    18
    Thanked 7 Times in 1 Post
    Quote Originally Posted by maadjordan View Post
    can please add timing if possible.
    All tests takes less 2 minutes, and I'm interested only in fast compression.
    And this is reason why I'm not tested 7-zip, Rar etc

  8. #8
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    539
    Thanks
    192
    Thanked 174 Times in 81 Posts
    Quote Originally Posted by Surfer View Post
    All tests takes less 2 minutes, and I'm interested only in fast compression.
    And this is reason why I'm not tested 7-zip, Rar etc
    In that case, I don't think you can do much better than SREP combined with a fast compressor (> 10 MB/s). See these results with timings (AMD Duron 800 MHz, results on CD_ROM_opt.iso):

    Code:
    SREP 2.95:           14 s,       41.836.829
    SREP + THOR 0.95 e1: 14 +  3 s,  41.551.372
    SREP + THOR e2:      14 +  4 s,  38.888.840
    SREP + THOR e3:      14 + 10 s,  38.642.908
    SREP + THOR e4:      14 + 36 s,  38.430.476
    SREP + THOR e5:      14 + 30 s,  38.453.808
    SREP + zhuff 0.7:    14 +  4 s,  38.624.966
    THOR 0.95 e1:        9 s,       127.544.464
    THOR 0.95 e5:        98 s,      118.111.108
    zhuff 0.7:           13 s,      118.552.491
    Even when switching to much better and slower compressors, you won't get something like 35 MB here. SREP does the main work with the deduplication part very good and very fast. For the compressing after SREP, zhuff seems to be very good as it is very fast and offers multithreading.

    Sizewise, you could perhaps get it down to around 25 MB with recompression of the .MSI archives inside, but LZX recompression still has to be done and it surely won't be possible to get to that size while still having >10 MB/s.

    EDIT: OK, it seems there's one last thing you can consider - we're lucky and SREP leaves most Precomp streams intact:

    Code:
    SREP + Precomp 0.4.1 "-c-" + zhuff 0.7:      14 + 21 + 4 s,  38.609.067
    SREP + Precomp 0.4.1 "-c- -t+j" + zhuff 0.7: 14 + 9 + 4 s,   38.617.343 // jpeg only
    Last edited by schnaader; 22nd March 2011 at 04:19.
    http://schnaader.info
    Damn kids. They're all alike.

  9. #9
    Member Surfer's Avatar
    Join Date
    Mar 2009
    Location
    oren
    Posts
    203
    Thanks
    18
    Thanked 7 Times in 1 Post
    Thanks schnaader. I'll try something like LZOP, LZTurbo, QuickLZ later.

  10. #10
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,612
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Try also 4x4 with tornado.

  11. #11
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    srep is opimized for memory usage, rep should be faster

  12. #12
    Member Surfer's Avatar
    Join Date
    Mar 2009
    Location
    oren
    Posts
    203
    Thanks
    18
    Thanked 7 Times in 1 Post
    Some results for Optimized+SREP, I'm really lazy to calculate timings
    4x4 t1 - 38 696 431
    4x4 t4 - 38 659 588
    4x4 8 - 38 043 788
    4x4 7 - 38 296 280
    4x4 6 - 38 405 686
    all 4x4 results is slow, may be because I've only HyperThreading.
    tornado - 38 421 129
    tornado -12 - 38 156 032 (too slow)
    lzop - 38 954 969
    lzop -9 - 38 542 156
    csc32 - 37 972 813 (very fast and best result in my quick test)
    lzturbo -55 - 38 461 131 (I can't select best "fast" method)
    Also can't find binaries for QuickLZ 1.5
    Quote Originally Posted by Bulat Ziganshin View Post
    srep is optimized for memory usage, rep should be faster
    I'm using default parameters, so they practically equal.

  13. #13
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    539
    Thanks
    192
    Thanked 174 Times in 81 Posts
    Quote Originally Posted by Surfer View Post
    csc32 - 37 972 813 (very fast and best result in my quick test)
    Nice, tested this one and added timings you can compare to my timings above.

    Code:
    SREP + csc32:                                14 + 30 s,      37.972.810
    SREP + Precomp 0.4.1 "-c- -t+j" + csc32:     14 + 9 + 30 s,  37.973.784
    SREP + Precomp 0.4.1 "-c-" + csc32:          14 + 21 + 30 s, 37.960.513
    Note the first result is 3 bytes smaller than yours, as the input filename is stored inside and I seem to have chosen a shorter one.
    http://schnaader.info
    Damn kids. They're all alike.

  14. #14
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    >I'm using default parameters, so they practically equal.

    no, it's because file is small. OTOH rep is by no means worse than srep on files that completely fits in RAM



    schnaader, how about pure csc32 results (i.e. time)?

  15. #15
    Member Surfer's Avatar
    Join Date
    Mar 2009
    Location
    oren
    Posts
    203
    Thanks
    18
    Thanked 7 Times in 1 Post
    Thanks!
    I've tested Nanozip 0.08 LZHDS 400Mb - 37 850 726 (12 seconds).

  16. #16
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    539
    Thanks
    192
    Thanked 174 Times in 81 Posts
    Quote Originally Posted by Bulat Ziganshin View Post
    schnaader, how about pure csc32 results (i.e. time)?
    Code:
    csc32:                                 79.5 s,        40.140.101
    nz "-cD":                              36 s,          38.303.338
    srep + nz "-cD":                       4 + 25 s,      37.850.726
    srep + Precomp 0.4.1 "-c-" + nz "-CD": 4 + 21 + 26 s, 37.837.629
    Wow, that's the first one I tested that takes less than 2 minutes on my PC and handles the deduplication correctly. Also tested nanozip which is even better. Thanks for the hints.
    Last edited by schnaader; 23rd March 2011 at 18:50.
    http://schnaader.info
    Damn kids. They're all alike.

  17. #17
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    freearc -m2

  18. #18
    Member Surfer's Avatar
    Join Date
    Mar 2009
    Location
    oren
    Posts
    203
    Thanks
    18
    Thanked 7 Times in 1 Post

    arc -m2 - 38 391 049
    lzpm - 38 518 168
    ulz - 38 882 279
    lzss - 43 275 684

Similar Threads

  1. Images PreProcessor - PrePNG
    By PAQer in forum Data Compression
    Replies: 3
    Last Post: 21st May 2010, 12:21
  2. Idea for raising compression efficiency on disk images
    By Mexxi in forum Data Compression
    Replies: 10
    Last Post: 18th February 2010, 05:56
  3. Compression used in .wim (Vista) images
    By jaclaz in forum Data Compression
    Replies: 9
    Last Post: 29th August 2008, 18:05
  4. "decompilling" iso
    By SvenBent in forum Forum Archive
    Replies: 5
    Last Post: 1st April 2008, 00:18
  5. Replies: 12
    Last Post: 30th June 2007, 16:49

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •