Results 1 to 30 of 30

Thread: MM compression library

  1. #1
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    http://www.haskell.org/bz/mm.zip

    i glad to present first open-source multimedia compression library
    which allows to add MM compresion features to any open-source compressor
    or archiver. the library provided under GPL license and includes 3 modules:

    mmdet - multimedia detection. this module allows to examine raw chunk
    of data and determine number of channels and width of data samples.
    returned result may be, for example, "skip 2 bytes at start of file and
    encode rest of data as 6 channels of 24-bit values". floating-point values
    and values with non-intel byte order are not recognized. you've also
    provided with results of simple order-0 and lz+order0 compression which
    allows you to decide whether it has meaning to use MM compression at all.
    MMDET module relies on comparison of applying order-0 model to original
    data and data after MM preprocessing (substraction of successive values).
    additionally, recognition by file header is available, although at this
    moment only .WAV file headers are supported

    tta - this module based on TrueAudio lossless audio codec developed by
    Alexander Djourik and Pavel Zhilin (http://www.true-audio.com/) version 2.0
    mmdet module (both WAV header method and entropy tests) used to determine
    structure of wave file. Like an original TTA codec, 8, 16 and 24 bits integer
    and 32-bit floating-point samples are supported. Unfortunately, in some
    rare cases 24-bit compression leads to program crash and 32-bit FP compression
    decodes "00 00 80 3F" values as "00 00 00 00" so probably high-precision
    samples support will be disabled in FreeArc release

    mm - simple multimedia data preprocessor which doesn't reduce data size
    but only substracts successive values. running grzip or ppmd on its output
    allows to provide compression ratio comparable with that of rar, uharc and sbc.
    it uses mmdet module to determine structure of data being compressed



    All modules includes main() driver which allows you to compile them as
    standalone apps. Use .bat files for this purpose, while provided makefile shows
    how to compile the library as part of larger application. You should also know
    that mmdet standalone program display its suggestion about selection of optimal
    MM model based on simple critery - which model gives the smaller output, while
    tta/mm modules uses mmdet library with smarter criteria (which is slightly
    different in mm and tta) which allows to pretty reliably recognize
    high-precision samples (mmdet in this case may suggest, for example, using
    4*8bit model instead of 1*32). You can test mm/tta autodetection features even
    with WAV files by using -s switch

    I thank Christian Maeder and Uwe Herklotz for their help in development of
    mmdet module


    ps: can you please help me fix language mistakes in this readme text?

  2. #2
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    just fixed problem with storing non-MM files in MM preprocessor. please reload

    also i've tested compression of mod/stm file with and without MM preprocessor:

    Arc.exe a a -m4xb
    Compressed 15 files, 2.189.379 => 1.208.577 bytes. Ratio 55.2%
    Compression time 3.41 secs, speed 641 kb/s. Total 4.06 secs

    Arc.exe a a -mmm:1*8+4xb
    Compressed 15 files, 2.189.379 => 1.141.624 bytes. Ratio 52.1%
    Compression time 3.90 secs, speed 562 kb/s. Total 4.39 secs

  3. #3
    Member Vacon's Avatar
    Join Date
    May 2008
    Location
    Germany
    Posts
    523
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hello everyone,

    Great work!!!
    But why GPL and not L_GPL?
    Imagine Igor wants to use it for 7-zip!?

    Best regards!

  4. #4
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    1) all my libs are gpl'ed, if Igor will ask about their use in 7zip, i will change license for him. otherwise i prefer to use gpl license
    2) it's TTA license. so changing it for TTA module will also require agreement with TTA authors

  5. #5
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    Tested mm on my testset and it successfully preprocessed BMP and PSD files:

    File: filesize w/ WinRAR -> filesize w/ mm+winRAR (ratio increase)
    PSD: 2 276 844 -> 2 032 855 (10,7%) (original filesize 6 755 163)
    BMP: 1 002 299 -> 950 609 (5,1%) (original filesize 3 932 214)

    File: filesize w/ 7-Zip -> filesize w/ mm+7-Zip (ratio increase)
    PSD: 1 152 376 -> 1 139 150 (1,1%)
    BMP: 903 276 -> 768 525 (14,9%)

  6. #6
    Member Vacon's Avatar
    Join Date
    May 2008
    Location
    Germany
    Posts
    523
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hello everyone,

    thank you Bulat for your quick answer.
    I will tell in 7-zip forum, since I don't know if Igor reads here.

    Hmmm, why should I?
    Bulat did already himself!
    Sorry for making noise
    Best regards!

  7. #7
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    Black_Fox
    according to my tests, using grzip (arc a a -m=grzip:m1:l:a) after MM preprocessor gives even better compression results

  8. #8
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    I'll try it, thanks for the tip, Bulat

    I'm not so optimistic about integrating mm(det) into 7-zip, though... Igor wants to have every part of it made by his own - at least I don't know about any part that was taken from other author

  9. #9
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    i know, so i don't tell about this to Igor directly. but he knows that he can use my libs if he want. otherwise, i will do the same in freearc

  10. #10
    Member
    Join Date
    Jan 2007
    Location
    Moscow
    Posts
    239
    Thanks
    0
    Thanked 3 Times in 1 Post
    Quote Originally Posted by Black_Fox
    at least I dont know about any part that was taken from other author
    ppmd, bzip2?

  11. #11
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    unforunately, Igor says that he prefers to use parts developed by him. in particular, instead of using ppmd for text with auto detection of file type, he will prefer if possible to use some lzma-based solution

  12. #12
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    Quote Originally Posted by nimdamsk
    ppmd, bzip2?
    Igor has made his own implemetation surely. He belongs into those compression people group that is able to made own compatible implementation of ppmonstr (closed-source) during time

  13. #13
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    i also belong to such group, theroretically but to produce something like lzma or ppmd one need several *years* of experiments

  14. #14
    Member
    Join Date
    Dec 2006
    Posts
    611
    Thanks
    0
    Thanked 1 Time in 1 Post
    I'm sure that years won't be enough for anyone to be able to produce ppmonstr compatible archives In compression circles, it seems that from every 10 compression genies 7 are Russian.

  15. #15
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    i just want to emphasize that knowledge isn't enough. you should try many ideas and spend a lot of time developing state-of-the-art library. it is why i can write something good only in areas of narrow insterests such as dict preprocessing or fast lz engine where competition is pretty small or non-existant

  16. #16
    Member Vacon's Avatar
    Join Date
    May 2008
    Location
    Germany
    Posts
    523
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hello everyone,

    Quote Originally Posted by Bulat Ziganshin
    ...one need several *years* of experiments
    maybe one should invent a time-compressor...?


    Best regards!

  17. #17
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    i work on this and hope to present 0.1 alpha version in a few years

  18. #18
    Member
    Join Date
    Jul 2008
    Posts
    54
    Thanks
    0
    Thanked 0 Times in 0 Posts
    how really it works?
    I tried and optmifrog compres more the data
    i do: mm.exe -w16 nameinput name output and file size is 12byte more
    i do: tta the same the file size it' more smaller, but not more than optimfrog

    Any examples?

  19. #19
    The Founder encode's Avatar
    Join Date
    May 2006
    Location
    Moscow, Russia
    Posts
    3,954
    Thanks
    359
    Thanked 332 Times in 131 Posts
    Quote Originally Posted by Bulat Ziganshin
    i work on this and hope to present 0.1 alpha version in a few years
    Within 20 years...

  20. #20
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    John
    1. mm is a *preprocessor* which means that you should compress its output with grzip, ppmd or some other algorithm (and compression ratio will be improved due to preprocessing)

    2. tta anyway compress audio wave data much better than mm+grzip. but it's not the best losless audio codec available and i know that optimfrog is really better. i not wrote anything yourself, i just selected among 4-5 open-source codecs one with minimum size of sources and maximum compression and then i've added a bit of code required to use it inside freearc, 7z and so on. but by itself these programs isn't interesting for *end-users*. it's just a library for programmers and *examples* of using this library

  21. #21
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    Black Fox fixed readme, thanks

  22. #22
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    New release:
    v1.1 (2007-09-11)
    TTA: fixed problems with processing 24/32-bit samples
    TTA: improved compression by 1-2%
    MM: fixed error with processing 'offset'

    http://www.haskell.org/bz/mm11.zip

  23. #23
    Member Vacon's Avatar
    Join Date
    May 2008
    Location
    Germany
    Posts
    523
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hello everyone,

    good news
    Thank you Bulat!

    Best regards!

  24. #24
    Member
    Join Date
    Jan 2007
    Location
    Moscow
    Posts
    239
    Thanks
    0
    Thanked 3 Times in 1 Post
    Thanks! Waiting for new ARC with Tornado 3 + MM

  25. #25
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    the august version includes both i don't put new MM version on the site only because there was no users who really need it

  26. #26
    Moderator

    Join Date
    May 2008
    Location
    Tristan da Cunha
    Posts
    2,034
    Thanks
    0
    Thanked 4 Times in 4 Posts
    Thanks Bulat!

  27. #27
    Member Vacon's Avatar
    Join Date
    May 2008
    Location
    Germany
    Posts
    523
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hello everyone,

    Quote Originally Posted by Bulat Ziganshin
    the august version includes both...
    Have I missed something? Is there an official final?

    Best regards!

  28. #28
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    i say about beta version you've seen

  29. #29
    Member Vacon's Avatar
    Join Date
    May 2008
    Location
    Germany
    Posts
    523
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hello everyone,

    Quote Originally Posted by Bulat Ziganshin
    i say about beta version youve seen
    Since it was a beta (and isnt officially reachable for some reasons anymore...) -> time for a stable release?

    Best regards!

  30. #30
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,497
    Thanks
    733
    Thanked 659 Times in 354 Posts
    i don't yet finished it. there are errors that i need to fix, new features that i need to make really usable, new codecs that i need to finish

Similar Threads

  1. QuickLZ ZIP - new zip/deflate library
    By Lasse Reinhold in forum Forum Archive
    Replies: 23
    Last Post: 1st October 2007, 22:08

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •