Results 1 to 13 of 13

Thread: practical implementation of packJPG and packMP3

  1. #1
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    872
    Thanks
    457
    Thanked 175 Times in 85 Posts

    Question practical implementation of packJPG and packMP3

    packJPG sources are out for quite some months now, and with sources of packMP3 also being available,
    I am asking myself, why we don't see them included in p.ex. FreeARC, MCM, ZPAQ or a custom 7-Zip.
    Would such an integration be much work?

  2. #2
    Member
    Join Date
    Jun 2013
    Location
    USA
    Posts
    98
    Thanks
    4
    Thanked 14 Times in 12 Posts
    Since I am not a developer, here are my guesses:

    1: Licensing.

    packJPG is released under the LGPL(i think) while packMP3 under the GPL. This may interfere with some projects that wish to be public domain(7-Zip).
    2: Lack of Interest.

    I posted news about packMP3 to HydrogenAudio the day sources were released. 0 replies: http://www.hydrogenaudio.org/forums/...0&#entry846137
    3: Marginal compression improvement.

    packJPG is better in this regard since the compression ratio is often better. Decoding time is also much slower. Also, to get the most benefit out of both, pre-processing must be done(default jpegtran settings for packJPG and sometimes mp3packer -z depending on the encoder used as well as VBR vs. CBR).

    Actually mp3s could be compressed even further by using packJPG on the embedded image(s) prior to using packMP3 but that's a small aside.

  3. #3
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    For zpaq, there is the technical problem in that the decompression code has to be written in ZPAQL.

  4. #4
    Member
    Join Date
    Aug 2007
    Location
    Rzeczpospolita
    Posts
    10
    Thanks
    0
    Thanked 2 Times in 1 Post
    Quote Originally Posted by Mangix View Post
    I posted news about packMP3 to HydrogenAudio the day sources were released. 0 replies: http://www.hydrogenaudio.org/forums/...0&#entry846137
    Command-Line Decoder Wrapper can't use packMP3.

  5. #5
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    872
    Thanks
    457
    Thanked 175 Times in 85 Posts
    My idea would be something like 7-Zip extra that uses both compressors for their certain filetypes in a .7z or .7zx format.
    As Igor is very busy atm, would someone else implement it? As for licensing, 7-Zip is LGPL and could use packJPG easily.
    If packMP3 would be LGPL, the same would apply here. And 7-Zip already uses non-LGPL license, because BZIP2 is under
    a BSD license.

    Hydrogenaudio is a place for audiophile people; they care more about playback than compression.
    I think packMP3 is more interesting for archivers than for playback software/users.

  6. The Following User Says Thank You to Stephan Busch For This Useful Post:

    GOZARCK (8th October 2013)

  7. #6
    Member
    Join Date
    Feb 2012
    Location
    Sweden
    Posts
    59
    Thanks
    4
    Thanked 5 Times in 5 Posts
    Quote Originally Posted by Stephan Busch View Post
    My idea would be something like 7-Zip extra that uses both compressors for their certain filetypes in a .7z or .7zx format.
    As Igor is very busy atm, would someone else implement it? As for licensing, 7-Zip is LGPL and could use packJPG easily.
    If packMP3 would be LGPL, the same would apply here.
    Well like you said it's not likely to be licencing as 7-zip is LGPL, perhaps it could cause some problems with inclusion into 'libarchive' which is permissively licenced, however as I recall they don't have a problem with LGPL, atleast Con Kolivas was discussing with the original authors of lzo and rzip to see if he could re-licence lrzip under LGPL for inclusion into libarchive.

    Which leaves me with two possibilities:

    *simple lack of interest in such a specialized type of compression(s)

    *difficulty/too much work in implementing it into the respective projects.

    The latter would be the lack of a simple to use library, but you do provide one so unless it's something in the way it's constructed which makes it hard to implement then that doesn't really make sense.

    I believe that the typical ideal library is an ANSI compatible C implementation, packjpg is C++, but I really can't say if that is an actual problem with adoption in this case.

    I would love seeing your JPG and MP3 compression being implemented in other packages given what a great compression ratio they provide on already compressed data, data which in turn is quite ubiquitous.

  8. #7
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    681
    Thanks
    37
    Thanked 168 Times in 84 Posts
    Quote Originally Posted by Stephan Busch View Post
    packJPG sources are out for quite some months now, and with sources of packMP3 also being available,
    I am asking myself, why we don't see them included in p.ex. FreeARC, MCM, ZPAQ or a custom 7-Zip.
    Would such an integration be much work?
    Well, integration of PackJPG and PackMP3 into FreeArc is possible right now
    Its not the method of integration you want, but anyway...
    Add this to your ini file:
    Code:
    [External compressor:packjpg]
    packcmd   = packJPG.exe -np {options} $$arcdatafile$$.jpg
    unpackcmd = packJPG.exe -np {options} $$arcdatafile$$.pjg
    datafile   = $$arcdatafile$$.jpg
    packedfile = $$arcdatafile$$.pjg
    solid = 0
    
    [External compressor:packmp3]
    packcmd   = packMP3.exe -np {options} $$arcdatafile$$.mp3
    unpackcmd = packMP3.exe -np {options} $$arcdatafile$$.pmp
    datafile   = $$arcdatafile$$.mp3
    packedfile = $$arcdatafile$$.pmp
    solid = 0
    Modify your group file in a way that *.mp3 mask will be excluded from $compressed group and create a new group:
    Code:
    $mp3
    *.mp3
    Now you can pack with -m$jpgsolid=packjpg -m$mp3=packmp3

    As for "lack of interest" mentioned here...
    Most of a people have a poor knowledge about compression itself, so try to tell 'em about recompression
    IMHO recompression is interesting for developers who creating it, and for compression enthusiasts of course.
    For example I asked one man to compile the PackJPG 2.5f DLL with Intel Compiler. He did this and also used the profiling so resulting speedup was about 13.4% comparing to native EXE.
    I gave this DLL to another man who created multi-threaded version of PackJPG.
    http://skymmer.narod.ru/images/shots...-01-52_000.png
    On the test-set which consists of 4477 files (315 308 107 bytes) original PackJPG takes 226 sec. while optimized and multi-threaded version takes 27 seconds. So actually there is an interest. But as I said it exists in a quite underground manner.

  9. The Following 4 Users Say Thank You to Skymmer For This Useful Post:

    Bulat Ziganshin (13th October 2013),GOZARCK (13th October 2013),Jaff (18th October 2013),Matt Mahoney (14th October 2013)

  10. #8
    Member
    Join Date
    May 2013
    Location
    ARGENTINA
    Posts
    54
    Thanks
    62
    Thanked 13 Times in 10 Posts
    .................................................. ..................................
    Last edited by GOZARCK; 5th November 2013 at 19:56.

  11. #9
    Member Dimitri's Avatar
    Join Date
    Nov 2015
    Location
    Greece
    Posts
    48
    Thanks
    21
    Thanked 30 Times in 14 Posts

    Talking practical implementation of packJPG and packMP3

    Quote Originally Posted by Skymmer View Post
    Well, integration of PackJPG and PackMP3 into FreeArc is possible right now
    Its not the method of integration you want, but anyway...
    Add this to your ini file:
    Code:
    [External compressor:packjpg]
    packcmd   = packJPG.exe -np {options} $$arcdatafile$$.jpg
    unpackcmd = packJPG.exe -np {options} $$arcdatafile$$.pjg
    datafile   = $$arcdatafile$$.jpg
    packedfile = $$arcdatafile$$.pjg
    solid = 0
    
    [External compressor:packmp3]
    packcmd   = packMP3.exe -np {options} $$arcdatafile$$.mp3
    unpackcmd = packMP3.exe -np {options} $$arcdatafile$$.pmp
    datafile   = $$arcdatafile$$.mp3
    packedfile = $$arcdatafile$$.pmp
    solid = 0
    Modify your group file in a way that *.mp3 mask will be excluded from $compressed group and create a new group:
    Code:
    $mp3
    *.mp3
    Now you can pack with -m$jpgsolid=packjpg -m$mp3=packmp3

    As for "lack of interest" mentioned here...
    Most of a people have a poor knowledge about compression itself, so try to tell 'em about recompression
    IMHO recompression is interesting for developers who creating it, and for compression enthusiasts of course.
    For example I asked one man to compile the PackJPG 2.5f DLL with Intel Compiler. He did this and also used the profiling so resulting speedup was about 13.4% comparing to native EXE.
    I gave this DLL to another man who created multi-threaded version of PackJPG.
    http://skymmer.narod.ru/images/shots...-01-52_000.png
    On the test-set which consists of 4477 files (315 308 107 bytes) original PackJPG takes 226 sec. while optimized and multi-threaded version takes 27 seconds. So actually there is an interest. But as I said it exists in a quite underground manner.
    here is a more practical way to deal with mp3 and jpg a more...multithreaded one ...enjoy
    Attached Files Attached Files

  12. The Following 2 Users Say Thank You to Dimitri For This Useful Post:

    Simorq (20th August 2018),Stephan Busch (7th November 2015)

  13. #10
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    681
    Thanks
    37
    Thanked 168 Times in 84 Posts
    I'm aware of ppx2 trick but its not so practical. It doesn't allows integration with FreeArc.

  14. #11
    Member Dimitri's Avatar
    Join Date
    Nov 2015
    Location
    Greece
    Posts
    48
    Thanks
    21
    Thanked 30 Times in 14 Posts
    i know that, since i tried using it with freearc endlessly !!! beta testing and such i compiled packjpg.dll but didnt manage to get it work either so i ended up with that solution!! can you help me come to a solution with packjpg.dll i understand you ddont want to compromise your work so any advice is welcome

  15. #12
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    856
    Thanks
    45
    Thanked 104 Times in 82 Posts
    dimitri you batch script only take up to 8 coes and only even cores.
    you should use statis if settings for that since it means you have to make a new entry for every core configurations
    You want to use a for loop so its automatically fills out any arbitrary numbers of cores

    for /l %%t in (1,1,%NUMBER_OF_PROCESSORS%) do (
    What ever command lines you want to use for each cores
    )

    Thats how i did it in pngbest.bat

    it seems weird to use a variable that contains the number of cpu and put that info into another variable which is only used in the if settings check. and the if settings only function is to make static command with the the number of CPU's ( which is the very variable you started with) in a static value.

    why not just go
    start /b cmd /c dir /s /b *.jpg | ppx2 -P %NUMBER_OF_PROCESSORS% -L 1 packjpg -np "{}"
    Thats 1 line and 1 variable instead of 15 lines 2 variable and a static value AND it can even handle any number of CPU cores
    Last edited by SvenBent; 6th November 2015 at 03:21.

  16. #13
    Member Dimitri's Avatar
    Join Date
    Nov 2015
    Location
    Greece
    Posts
    48
    Thanks
    21
    Thanked 30 Times in 14 Posts
    generally i use this ~> start /b cmd /c dir /s /b *.jpg | ppx2 -P %NUMBER_OF_PROCESSORS% -L 1 packjpg -np "{}"
    but i was testing the script with freearc, and at the moment is doesnt work!! will try to use your way with loop it seems rather interesting ...

Similar Threads

  1. packMP3 v1.0d release: Open source under the GPL v3
    By packDEV in forum Data Compression
    Replies: 3
    Last Post: 2nd October 2013, 02:11
  2. Practical compression benchmarks
    By Matt Mahoney in forum Data Compression
    Replies: 36
    Last Post: 14th July 2013, 22:47
  3. packMP3 v1.0c release
    By packDEV in forum Data Compression
    Replies: 11
    Last Post: 11th October 2012, 17:48
  4. Maximum Practical Compression
    By Bulat Ziganshin in forum Forum Archive
    Replies: 5
    Last Post: 31st March 2008, 15:20
  5. Best practical archiver
    By nimdamsk in forum Forum Archive
    Replies: 34
    Last Post: 24th March 2007, 21:51

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •