Results 1 to 24 of 24

Thread: HELP! movie compression

  1. #1
    Member Lone_Wolf236's Avatar
    Join Date
    Aug 2009
    Location
    Canada
    Posts
    13
    Thanks
    0
    Thanked 0 Times in 0 Posts

    HELP! movie compression

    hey all!
    over the past months, i've tried a lot of compressors but couldn't find any that was able to compress an .avi movie to an acceptable ratio for the time spent.
    i've been unable to gain more than 2% of the file size...

    i dont understand why because the codecs are supposed to work only with the frames and pixel blocks, not re-arrange bits so its uncompressable!
    why isn't my cpu working at 100% while playing a movie if its already compressed?

    or maybe the compressors out there dont have the required filters...?

    P.S.: i dont want to reencode my movies to other formats because i will loose quality. (and if i want to keep the same quality it will end up being the same size)


    Thanks a lot!!!

  2. #2
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    854
    Thanks
    45
    Thanked 104 Times in 82 Posts
    your data is already compressed witch makes it hard to compress it anymore.

    did you also think you could zip a zip file?

    depending on cpu codec yuo might find some special tools to enhance the compression on you file og maybe compress it sligtly more. but consider this a rare chance

  3. #3
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,253
    Thanks
    304
    Thanked 768 Times in 482 Posts
    All video formats (MPEG, MPEG2, H.264, etc) are already compressed so you can't compress them further (not much, anyway). It might be possible to write a specialized compressor to squeeze a bit more out, like stuffit or paq8 for JPEG or soundslimmer for MP3. But nobody has done that for video yet. It would likely be much slower than playing the video and I doubt you could squeeze out more than 30%.

  4. #4
    Member
    Join Date
    Jun 2009
    Location
    Puerto Rico
    Posts
    154
    Thanks
    41
    Thanked 12 Times in 8 Posts
    I've managed to compress videos and save at least 10MB using RZM Compression. 10MB is just 10MB of free space!

  5. #5
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    854
    Thanks
    45
    Thanked 104 Times in 82 Posts
    about the cpu usage.
    its the simple facts you don decompress AS FAST as possible but only as fast as needs to playback the video.

    a 2 hours movie its ok to decompress in only 2 hours thereby not needing to use the entire cpu for the two hours


    if you decompressed the movie you would se the cpu usage climb as the decompression now goes as fast as possible and it will be done faster then playback time.

  6. #6
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    854
    Thanks
    45
    Thanked 104 Times in 82 Posts
    Quote Originally Posted by moisesmcardona View Post
    I've managed to compress videos and save at least 10MB using RZM Compression. 10MB is just 10MB of free space!
    i manage to compress a video with 7-zip and save several gigabytes.
    but the input files was in tereabytes

    measuring compression gains in bytes without giving af full size doesn't really tell anything

    if you save 10mb if a 11mb file thats impressive. but if you save 10mb of a 8gb file, then its not so impressive

  7. #7
    Member
    Join Date
    Jun 2009
    Location
    Puerto Rico
    Posts
    154
    Thanks
    41
    Thanked 12 Times in 8 Posts
    Quote Originally Posted by SvenBent View Post
    i manage to compress a video with 7-zip and save several gigabytes.
    but the input files was in tereabytes

    measuring compression gains in bytes without giving af full size doesn't really tell anything

    if you save 10mb if a 11mb file thats impressive. but if you save 10mb of a 8gb file, then its not so impressive
    I saved 10MB of really low quality Xvid Movies (from 100MB to 400MB each video of 1 hour to 3 hours).

    It seems that compressing Slideshows will save a lot of MB since I compressed a 600MB Slideshow and the result was 200MB only.

  8. #8
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    2,802
    Thanks
    125
    Thanked 712 Times in 342 Posts
    > over the past months, i've tried a lot of compressors but
    > couldn't find any that was able to compress an .avi movie
    > to an acceptable ratio for the time spent.

    Yeah, obviously

    > i've been unable to gain more than 2% of the file size...

    Now, this is worse than usual

    > i dont understand why because the codecs are supposed to
    > work only with the frames and pixel blocks, not re-arrange
    > bits so its uncompressable!

    Still, all these codecs not only do the lossy media processing,
    but also apply some (inefficient, but fast) compression algorithms.

    > why isn't my cpu working at 100% while playing a movie if its already compressed?

    That's unrelated

    > or maybe the compressors out there dont have the required filters...?

    Yeah, in a way.
    The main problem is that to apply better compression we need to
    remove the existing one first - and then reapply it
    back without any loss.
    And then there're containter formats (avi in this case) and multiple video
    codecs (most of which are not open-source), and audio codecs.
    Anyway, the fashion changes much faster than one could write a working
    recompressor for a specific video format.

    > P.S.: i dont want to reencode my movies to other formats
    > because i will loose quality. (and if i want to keep the
    > same quality it will end up being the same size)

    You can try this:
    http://shelwien.googlepages.com/avip_demo_1.rar
    (its an avi parser; its output files should be more compressible
    than original file).
    Then you can additionally compress the audio track with this:
    http://soundslimmer.com/
    if its in mp3 though.

  9. #9
    Member Lone_Wolf236's Avatar
    Join Date
    Aug 2009
    Location
    Canada
    Posts
    13
    Thanks
    0
    Thanked 0 Times in 0 Posts
    i tried the avi parser, it gave me a 500mb file and 5 others that are 0kb
    i used my test file that is 25mb

    how do you put the files back together now?

    (compressing with kgb now... 1,4%)
    (winrar gave me an archive larger by a couple Kb)

  10. #10
    Member Lone_Wolf236's Avatar
    Join Date
    Aug 2009
    Location
    Canada
    Posts
    13
    Thanks
    0
    Thanked 0 Times in 0 Posts
    freearc gave me a file 40kb larger than original

  11. #11
    Member
    Join Date
    Jun 2009
    Location
    Puerto Rico
    Posts
    154
    Thanks
    41
    Thanked 12 Times in 8 Posts
    try RZM.
    for simplicity, I would recommend my software, PerfectCompress, which can compress to a lot of formats. That way, you can choose which format compresses the best for you.

    http://moises-studios.110mb.com/perfectcompress

  12. #12
    Member Lone_Wolf236's Avatar
    Join Date
    Aug 2009
    Location
    Canada
    Posts
    13
    Thanks
    0
    Thanked 0 Times in 0 Posts
    nanozip ended 202kb smaller or 0,788% smaller not worth it...
    kgb was bigger than nanozip so i stopped it

    i guess i will have to leave them as avi...
    i dont know if its the right forum for this but...
    any good codec i could use to save space without visible loss of quality?
    it also depends of the program... any suggestion?

    thanks guys!

  13. #13
    Member Fu Siyuan's Avatar
    Join Date
    Apr 2009
    Location
    Mountain View, CA, US
    Posts
    176
    Thanks
    10
    Thanked 17 Times in 2 Posts
    You are on the right forum. But I really think there hardly be software can achieve your goals, especially for the "without loose quality".

  14. #14
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    2,802
    Thanks
    125
    Thanked 712 Times in 342 Posts
    unfortunately my avi parser only works with valid avi files.
    Certainly not 25m parts of avi files or something like that.
    Its purpose is to split the avi container into video, audio,
    and index streams, which are better compressed when not
    interleaved.

    As an alternative, you can use this:
    http://www.bunkus.org/videotools/mkv...icode-2.9.8.7z
    to convert your avi (but only if that was really my avi parser bug,
    and the avi is valid) to mkv format without reencoding the media streams.

    Mkv files are usually a little more compact than avi files,
    so you can expect to shrink the size of your files by a few megabytes.
    But that difference is only caused by a more compact index
    representation in mkv, so you shouldn't expect much.

  15. #15
    Member
    Join Date
    Sep 2009
    Location
    APITRC
    Posts
    27
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Shelwien View Post
    >
    You can try this:
    http://shelwien.googlepages.com/avip_demo_1.rar
    (its an avi parser; its output files should be more compressible
    than original file).
    Then you can additionally compress the audio track with this:
    http://soundslimmer.com/
    if its in mp3 though.

    Am very much interested in putting back the splitted parts. this avip_demo does perfect withy many avi videos that I tested on, when compressed the splitted parts (.mp3 files) it produced very nice results.

    Question: Can you kindly direct how to put back these parts to an AVI file?

    Thanks

  16. #16
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    2,802
    Thanks
    125
    Thanked 712 Times in 342 Posts
    Just read the test.bat there.
    Code:
    @echo off
    rmdir /s /q temp
    mkdir temp
    cd temp
    ..\avi ..\Semi256.avi 
    ..\avi -d ..\Semi256.unp
    cd ..
    fc /b Semi256.unp Semi256.avi 
    rar a -m5 -mdg -s 1.rar Semi256.avi 
    rar m -m5 -mdg -s 2.rar temp
    del Semi256.unp
    Specifically, the line
    Code:
    ..\avi -d ..\Semi256.unp
    merges the 6 streams into single Semi256.unp

    (..\ paths there are due to temp subdirectory used
    to store the files)

  17. #17
    Member Fallon's Avatar
    Join Date
    May 2008
    Location
    Europe - The Netherlands
    Posts
    152
    Thanks
    13
    Thanked 9 Times in 4 Posts

    Post

    Avi is still popular in peer2peer, but it's on the way out, because .mkv is also up and coming for music videos.
    Mkv is flexible and not tied to old uncle Gates' rules. Microsoft is planning mediaplayer support for .mov files now.
    As usual they will have to be dragged to the blessing of supporting an open format.

    (edited short - because it's getting off topic)
    VLC-player has excellent .mkv support.
    http://www.videolan.org/
    Another cross platform free and open source player:
    http://www.getmiro.com/download/
    Last edited by Fallon; 1st November 2009 at 07:44.

  18. #18
    Member Vacon's Avatar
    Join Date
    May 2008
    Location
    Germany
    Posts
    523
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hello everyone,

    reading 7-zip's forum, I just stumbled upon that one:
    http://ck.kolivas.org/apps/lrzip/README
    Anyone tested it? Seems as if there are no windows-binaries. Possibly it can be compiled by one of our gurus

    Best regards!

  19. #19
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,475
    Thanks
    702
    Thanked 645 Times in 347 Posts
    Vacon, rzip idea lies behind my REP filter you may consider rep as improved version of rzip

    from rep docs:

    REP is an LZ77-family algorithm, i.e. it founds matches and outputs them as
    (len,offset) pairs. It is oriented toward very fast compression and small
    memory overhead (1/4 of buffer size), but limited to rather large values of
    mimimal match length (say, 32), and don't search for optimum match. It's
    intended to preprocess data before using full-fledged compressors. and in
    this area it beats RZIP and, to some degree, LZP preprocessors. Small
    memory overhead means that RZIP/LZP/REP are capable to find matches at very
    long distances and this algorithm does it much better than RZIP and LZP.
    The algorithm implemented in functions REPEncode() and REPDecode().

    Main differences comparing to RZIP:
    1) Sliding window which slides at 1/16 of buffer size each time
    2) Almost ideal hash function (see update_hash)
    3) Direct hashing without hash chains which 1.5x cuts memory requirements
    4) Tags are not saved in hashtable, which again halves memory requirements.
    Instead, a few lower bits of hash table entry are used to save a few
    bits of tag (see chksum)
    5) Hash size is proportional to buffer size (which is equal to the maximum
    search distance) and by default limited to 1/4 of buffer size
    6) In order to find strings of length >=MinLen, blocks of length L=MinLen/2
    are indexed via hash. Of all those possible blocks, only 1/sqrt(L) are
    indexed and only 1/sqrt(L) are searched. It is alternative to solution
    described in RZIP paper where 1/L of blocks are indexed and each block
    searched. This means that logb(sqrt(L)) lower bits of hash entry are
    zeroes which allows to use trick 4.


    References for RZIP algorithm explanation and implementations:
    http://samba.org/~tridge/phd_thesis.pdf
    http://rzip.samba.org/ftp/rzip/rzip-2.1.tar.gz
    http://ck.kolivas.org/apps/lrzip/lrzip-0.18.tar.bz2
    http://www.edcassa-ict.nl/lrzip.zip
    http://www.edcassa-ict.nl/rzip21.zip

    TAYLOR, R., JANA, R., AND GRIGG, M. 1997. Checksum testing of remote
    synchronisation tool. Technical Report 0627 (November), Defence Science and
    Technology Organisation, Canberra, Australia. (p.72)


    References for LZP algorithm implementations:
    http://magicssoft.ru/content/downloa...GRZipIISRC.zip
    http://www.compression.ru/ds/lzp.rar

  20. #20
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,475
    Thanks
    702
    Thanked 645 Times in 347 Posts
    well, after reading README i see that it uses no memory on decompression. we had such idea, it would be very interesting to see how it's implemented here. i will give it a try

  21. #21
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,475
    Thanks
    702
    Thanked 645 Times in 347 Posts
    1. it seems that since lrzip 0.18 nothing important was changed and i just overseen this feature
    2. make prints the following errors:

    Code:
    In file included from rzip.c:24:
    rzip.h:37:26: sys/resource.h: No such file or directory
    rzip.h:38:24: netinet/in.h: No such file or directory
    rzip.h:64:22: sys/mman.h: No such file or directory
    rzip.c: In function 'init_hash_indexes':
    rzip.c:523: warning: implicit declaration of function 'random'
    rzip.c: In function 'rzip_chunk':
    rzip.c:534: warning: implicit declaration of function 'mmap'
    rzip.c:534: error: 'PROT_READ' undeclared (first use in this function)
    rzip.c:534: error: (Each undeclared identifier is reported only once
    rzip.c:534: error: for each function it appears in.)
    rzip.c:534: error: 'MAP_SHARED' undeclared (first use in this function)
    rzip.c:547: warning: implicit declaration of function 'munmap'
    make: *** [rzip.o] Error 1
    so it just needs to support a Windows way of mmapping files in order to compile. afaik, old lrzip was compiled by someone for mfc test

  22. #22
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    681
    Thanks
    37
    Thanked 168 Times in 84 Posts
    I've became interested in compression of already compressed data some time ago, when I was playing with resources from Fallout 3 RPG. I don't mean those BSA containers but the data extracted from them. There are a lot of compressed files there. Basicly OGG files, some MP3s and some movies in BIK format. I've tried to pack them with different archivers\compressors and as far as I remember the best one was Squeez.
    Since we talking about movie compression here I decided to conduct a modest test using 2 movies and some compression tools. Both movies in AVI v2.0 (OpenDML) container.
    First one is the The Good, The Bad and The Ugly film encoded with Divx 5.2.1 (528 KBit/s) + MP3 48000 Mono VBR (61 KBit/s).
    Second one is Lost (TV-series, Season 5 Episode 1) encoded with XviD (1008 KBit/s) + MP3 48000 Stereo CBR (128 KBit/s).
    Here are the results and settings used. Results sorted from worst to best.

    The Good, The Bad and The Ugly
    Code:
                                             Original = 734 599 168
                                RAR v3.90 -m5 -md4096 = 712 107 185
       FreeARC v0.60 -mlzma:d512m:h96m:lc3:fb64:mfht4 = 712 022 813       new
      FreeARC v0.60 -mlzma:d384m:h256m:lc3:fb64:mfht4 = 711 729 518       new
      FreeARC v0.60 -mlzma:d112m:h128m:lc3:fb64:mfhc4 = 711 619 169       new
                                    7-zip v9.07 Ultra = 710 222 137
                    7-zip v9.07 -m0=LZMA2:d=64m:fb=64 = 710 098 012       new
                            NanoZip v0.07a -co -m512m = 708 650 433
                                           RZM v0.07h = 704 331 622
    WinRK v3.1.2 ROLZ3, 256MB\512\64\24\fast analysis = 704 056 954
    WinRK v3.1.2 ROLZ3, 256MB\512\64\24\full analysis = 704 038 021
            Squeez v5.63 -fmt SQX2 -m5 -MD32768 -uxx9 = 699 899 645
                                         CCM v1.30c 5 = 697 420 919
                                        CCMx v1.30c 5 = 697 329 727
                            NanoZip v0.07a -cO -m512m = 695 996 832
                                      BCM v0.09 -b140 = 687 711 810       new
                            NanoZip v0.07a -cc -m512m = 680 159 944      -7.41%
    Lost
    Code:
                                             Original = 360 011 776
      FreeARC v0.60 -mlzma:d112m:h128m:lc3:fb64:mfhc4 = 357 221 787       new
      FreeARC v0.60 -mlzma:d384m:h256m:lc3:fb64:mfht4 = 357 166 574       new
                                    7-zip v9.07 Ultra = 357 071 721
                                RAR v3.90 -m5 -md4096 = 356 699 397
                    7-zip v9.07 -m0=LZMA2:d=64m:fb=64 = 356 505 767       new
                            NanoZip v0.07a -co -m512m = 355 764 716
    WinRK v3.1.2 ROLZ3, 256MB\512\64\24\full analysis = 354 137 950
                                           RZM v0.07h = 354 097 489
    WinRK v3.1.2 ROLZ3, 256MB\512\64\24\fast analysis = 353 769 930
                                         CCM v1.30c 5 = 352 128 887
                                        CCMx v1.30c 5 = 352 122 686
            Squeez v5.63 -fmt SQX2 -m5 -MD32768 -uxx9 = 351 555 517
                            NanoZip v0.07a -cO -m512m = 350 597 110
                                      BCM v0.09 -b140 = 348 240 300       new
                            NanoZip v0.07a -cc -m512m = 344 675 819      -4.26%
    Results are talking for itself, isn't it ? But if I were you I would stick with Squeez. The reasons are simple. Squeez is very fast on decoding while NanoZIP is not (especially -cc). CCM is completely symmetric and furthermore its a compressor.

    EDIT: Added new results (strings marked with 'new' word).
    Last edited by Skymmer; 6th November 2009 at 00:34.

  23. #23
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,475
    Thanks
    702
    Thanked 645 Times in 347 Posts
    try lzma2 too

  24. #24
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    681
    Thanks
    37
    Thanked 168 Times in 84 Posts
    OK, I've added some new results to my previous post (strings marked with 'new' word). Surprisingly good results for BCM.

Similar Threads

  1. Borat Movie
    By encode in forum Forum Archive
    Replies: 2
    Last Post: 4th May 2007, 21:31

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •