Page 1 of 3 123 LastLast
Results 1 to 30 of 77

Thread: FreeArc 'Next

  1. #1
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,483
    Thanks
    719
    Thanked 653 Times in 349 Posts

    FreeArc 'Next

    Oct 08, 2016: FA 0.11 (attached)

    Release notes
    Attached Files Attached Files

  2. The Following 36 Users Say Thank You to Bulat Ziganshin For This Useful Post:

    78372 (14th April 2017),avitar (9th October 2016),Chirantan (25th May 2017),Cyan (12th October 2016),encode (8th July 2017),GOZARCK (9th October 2016),inikep (8th October 2016),Intrinsic (10th October 2016),Jarek (6th January 2017),kassane (21st December 2016),lorents17 (9th October 2016),Lucas (8th October 2016),marcelospera (21st June 2017),Matt Mahoney (17th October 2016),Mike (9th October 2016),Minimum (8th October 2016),Nania Francesco (9th October 2016),ne0n (10th October 2016),Peachill (17th October 2018),pico (13th October 2016),Piglet (9th October 2016),PrinceGupta (14th October 2016),RamiroCruzo (9th October 2016),Razor12911 (8th October 2016),samsat1024 (14th October 2016),Sergey3695 (16th January 2017),Simorq (11th October 2016),starczek (24th February 2017),Stephan Busch (9th October 2016),surfersat (11th October 2016),Turtle (11th October 2016),Vendetta (9th May 2017),xezz (9th October 2016),Zonder (11th October 2016),zryn (9th March 2018),~MAK~ (9th October 2016)

  3. #2
    Member
    Join Date
    Nov 2014
    Location
    Moscow
    Posts
    3
    Thanks
    24
    Thanked 0 Times in 0 Posts
    this day will become history!!!

  4. #3
    Member
    Join Date
    Sep 2011
    Location
    uk
    Posts
    237
    Thanks
    184
    Thanked 16 Times in 11 Posts
    Suppose fa is intended for quite big files but tried just a noddy example, nothing set up in fa.ini, only fa32.exe extracted. a.dat is a 4 Mbyte ascii file

    running on an old xp laptop with 1.5G memory - old freearc works fine.

    fa32 a b a.dat
    Scanning: 4,299,617 bytes in 0 folders and 1 files (RAM 3 MiB, I/O 0.016 sec, c
    pu 0.016 sec, real 0.095 sec)
    Compress using rep:512mb+dispack070+delta+lzma:96mb:normal:16:mc8 , $obj => rep:5
    12mb+delta+lzma:96mb:normal:16:mc8, $text => dict:64mb:80%:l8192:m400:s100+lzp:6
    4mb:90%:65:h20:d1mb:s16+ppmd:8:96mb, $compressed => rep:512mb+tor:16mb:c3, $wav
    => tta, $bmp => mm+grzip:8mb:m1:l2048:h15:a
    Compression memory 796mb + 1*1mb read-ahead buffers, decompression memory 634 Mi
    B. Prefetch 256mb using 1 thread

    At this point fa crashes with a screen saying 'utility encountered problem & needs to close' creates a 0 byte file b

    Call me old fashioned, guess I don't have enough memory or maybe I need to load a dll? But anyway, don't think it should crash like this with no message saying 'not enough memory,or dll not found or whatever!

    john.

  5. #4
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,483
    Thanks
    719
    Thanked 653 Times in 349 Posts
    can you try with smaller compression algo, f.e. by adding -mz option?

    that said, there are two problems. first, fa doesn't exit gracefully when it cannot alloc memory - it's a bug

    second - fa, like arc, has agressive compression settings. it's ok for fa64 in arc, it's mitigated by -lc/-ld options - in particular, they ensure that algorithm will be shrinked to actual memory available. fa still lacks this feature so meanwhile it may be a bad choice for 32-bit platforms.

    btw, as mentioned in docs, options like ms/mm/lc/ld/md can be implemented entirely in Lua. is there someone interested to try that?

    third - when compressing file of this size, fa reduces dictionaries to 5 MB, so memory usage is very small:
    Code:
    Working Set      :           28300 KB
    Paged Pool       :             158 KB
    Nonpaged Pool    :              12 KB
    Pagefile         :           54240 KB
    so your problem may be not memory-related at all. i have no idea yet
    Last edited by Bulat Ziganshin; 9th October 2016 at 23:08.

  6. #5
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    860
    Thanks
    441
    Thanked 169 Times in 80 Posts
    First of all I am very happy that FreeArc development is alive again )

    In my tests, FreeArc'Next 0.11 runs using -mx switch and it is about 25% faster than FreeArc 0.67
    while achieving nearly the same compression as FreeArc 0.67.

    To me, it is difficult to define new groups and define compression method to it:
    p.ex. I want to define a group $arw where I can define lzma:lc8:lp4b4
    and I do not seem to be able to edit existing groups..

    Maybe we can use compression groups of EcoZip in FreeArc? Please feel free to use them.

  7. #6
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,483
    Thanks
    719
    Thanked 653 Times in 349 Posts
    Quote Originally Posted by Stephan Busch View Post
    In my tests, FreeArc'Next 0.11 runs using -mx switch and it is about 25% faster than FreeArc 0.67 while achieving nearly the same compression as FreeArc 0.67.
    great result although i bet it used much more memory? you can use proc-profile to check that

    Quote Originally Posted by Stephan Busch View Post
    To me, it is difficult to define new groups and define compression method to it:
    p.ex. I want to define a group $arw where I can define lzma:lc8:lp4b4
    and I do not seem to be able to edit existing groups..
    unfortunately, smiles in compression methods aren't supported yet
    what's the problem with editing arc.groups/fa.ini?

    Quote Originally Posted by Stephan Busch View Post
    Maybe we can use compression groups of EcoZip in FreeArc? Please feel free to use them.
    can you provide your own version of config files? if it it's better than existing one, people will use them without asking me

    btw, while i don't plan to use FA repository for source code, it may be used for collaboration on anything else. one possibility is to move docs from wiki engine to repository itself. providing your versions of translations (when it will be implemented) and config files may be a good idea too

    meanwhile, i can add everyone interested to collabolators list, so you get write access to repository. i already sent invitation to Yann while experimenting with this feature - sorry!

  8. The Following User Says Thank You to Bulat Ziganshin For This Useful Post:

    Stephan Busch (11th October 2016)

  9. #7
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    860
    Thanks
    441
    Thanked 169 Times in 80 Posts
    I would like to define group $arw by adding this to arc.groups:

    $arw
    *.arw
    *.rw2

    In FreeArc 0.67 I could define compression for that group using commandline:

    -m$arw=lzma:273:bt4:lc8:lp4b4

    How can I define this in fa.ini?

    I want to suggest using the analyzer and filters of CsArc 3.3
    since they are able to detect multimedia inside data and they
    apply automatically - also on camera raw files like .mos, .dng, .mef,
    .srw and uncompressed .arw.

    Please let me also vote for using more codecs like LIBBSC, Lepton and packMP3..

  10. #8
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,483
    Thanks
    719
    Thanked 653 Times in 349 Posts
    C:\>fa create m:\a -m$xxx=z
    Scanning: 820,886,193 bytes in 166 folders and 1,586 files (RAM 3 MiB, I/O 0.016 sec, cpu 0.000 sec, real 0.020 sec)
    Compress using rep:512mb+dispack070+delta+4x4:lzma:96mb:normal:16 :mc8, $obj => rep:512mb+delta+4x4:lzma:96mb:normal:16:mc8, $text => grzip:8mb:m1:l32:h15, $arw => 4x4:zstd, $compressed => rep:512mb+4x4:tor:c3, $wav => tta, $bmp => mm+grzip:8mb:m1:l2048:h15:a, $xxx => zstd:1
    in fa.ini:
    [Compression methods]
    4$arw=xz
    since 4 is the default compression method. Or just look into Lua code example in fa.ini and modify it to your need:
    Code:
    [Lua code]
    onPostOption (function ()
        AddCompressionGroup("tor:7","$xx")
        --RemoveCompressionGroups("$bmp")
    end)

  11. The Following User Says Thank You to Bulat Ziganshin For This Useful Post:

    Stephan Busch (11th October 2016)

  12. #9
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    860
    Thanks
    441
    Thanked 169 Times in 80 Posts
    I have done a first draft of arc.groups and fa.ini - just with some additional lzma groups, 2 PPMd groups and 1 ZSTD group.

    How can I use external xz.exe in order to use LZMA2 and delta filter?
    How can I use external bsc.exe to use LIBBSC and external CsArc?
    Attached Files Attached Files

  13. The Following 2 Users Say Thank You to Stephan Busch For This Useful Post:

    Bulat Ziganshin (12th October 2016),RamiroCruzo (12th October 2016)

  14. #10
    Member
    Join Date
    Jan 2015
    Location
    Hungary
    Posts
    8
    Thanks
    8
    Thanked 3 Times in 1 Post
    Weird behavior. I compare FreeArc 03-15-2014. Only x64 (testing and) results :

    Code:
    1.
    
    
    x64 in folder, large size :
    
    
    F:\>fa a webpages.tar-m2.arc -m2 w.tar
    Scanning: 979,036,160 bytes in 0 folders and 1 files  (RAM 3 MiB, I/O 0.000 sec, cpu 0.000 sec, real 0.002 sec)
    Compress using rep:512mb:256:c256+exe+4x4:zstd:7:16m:h8m, $obj => rep:512mb:256:c256+4x4:zstd:7:16m:h8m, $text => grzip:8mb:m4:l32:h15, $wav => tta:m1
    , $bmp => mm+grzip:8mb:m4:l:a, $compressed => rep:512mb:256:c256+4x4:zstd:2:8mb
    Compression memory 772mb + 4*1mb read-ahead buffers, decompression memory 544..706 MiB.  Prefetch 256mb using 1 thread
    100%: 979,036,160 -> 513,775,622: 52.48%.  I/O 1.732 sec, cpu 10.8 MiB/s (86.347 sec), real 39.7 MiB/s (23.542 sec) = 367%
    Archive directory: 185 bytes
    
    
    only fa.exe , good :
    
    
    F:\t>fa a weblapok.tar-m2 -m2 w.tar
    Scanning: 979,036,160 bytes in 0 folders and 1 files  (RAM 3 MiB, I/O 0.000 sec, cpu 0.000 sec, real 0.002 sec)
    Compress using rep:512mb:256:c256+exe+4x4:tor:5, $obj => rep:512mb:256:c256+4x4:tor:5, $text => grzip:8mb:m4:l32:h15, $wav => tta:m1, $bmp => mm+grzip
    :8mb:m4:l:a, $compressed => rep:512mb:256:c256+4x4:tor:3:8mb
    Compression memory 916mb + 4*1mb read-ahead buffers, decompression memory 563..783 MiB.  Prefetch 256mb using 1 thread
    100%: 979,036,160 -> 369,303,274: 37.72%.  I/O 0.858 sec, cpu 41.8 MiB/s (22.355 sec), real 133 MiB/s (7.001 sec) = 319%
    Archive directory: 195 bytes
    
    
    2.
    
    
    x64 in folder, good :
    
    
    F:\>fa a webpages-m1.arc -m1 w
    Scanning: 960,675,938 bytes in 832 folders and 13,757 files  (RAM 3 MiB, I/O 0.047 sec, cpu 0.031 sec, real 0.078 sec)
    Compress using rep:512mb:c512+4x4:zstd:2:8mb, $compressed => rep:512mb:c512
    Compression memory 626mb + 4*1mb read-ahead buffers, decompression memory 528..609 MiB.  Prefetch 256mb using 1 thread
    100%: 960,675,938 -> 357,997,813: 37.27%.  I/O 1.778 sec, cpu 274 MiB/s (3.338 sec), real 597 MiB/s (1.536 sec) = 217%
    Archive directory: 729,720 bytes
    
    
    x64, only fa.exe, long time :
    
    
    F:\t>fa a webpages-m1.arc -m1 w
    Scanning: 960,675,938 bytes in 832 folders and 13,757 files  (RAM 3 MiB, I/O 0.062 sec, cpu 0.016 sec, real 0.079 sec)
    Compress using rep:512mb:c512+4x4:tor:3:8mb, $compressed => rep:512mb:c512
    Compression memory 696mb + 4*1mb read-ahead buffers, decompression memory 537..648 MiB.  Prefetch 256mb using 1 thread
    100%: 960,675,938 -> 362,223,298: 37.71%.  I/O 2.683 sec, cpu 199 MiB/s (4.602 sec), real 11.1 MiB/s (82.676 sec) = 6%
    Archive directory: 729,692 bytes

  15. #11
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,483
    Thanks
    719
    Thanked 653 Times in 349 Posts
    1. you can use -cfg- option to disable fa.ini processing

    2. in your first pair of examples tor has much better compression than zstd. seems that its's just unusual case, but if anyone has better zstd setting for -m2 mode - i'm all ears. i just quickly checked zstd on my model binary file before releasing fa and then wrote to the fa.ini the best zstd setting that provided about the same compression ratio as tor:5

    3. and btw, in the second pair of examples, difference between ztsd and tor compression ratio is just 2 percents. compression time (cpu) is also pretty close, I/O times are 2 seconds in both cases. in your first run data were read from disk, while in the second run they were already in disk cache. btw, if your data are on ssd, adding --prefetch:8 may greatly improve "cold data" scenario



    Here are my results - first run is on cold data, and rest are from cache:

    C:\Base\Bench-links>fa create m:\a -m1 -cfg-
    Scanning: 4,775,001,234 bytes in 6,096 folders and 106,397 files (RAM 8 MiB, I/O 0.437 sec, cpu 0.328 sec, real 1.791 sec)
    Compress using rep:512mb:c512+4x4:tor:3:8mb, $compressed => rep:512mb:c512
    Compression memory 833mb + 8*1mb read-ahead buffers, decompression memory 537..752 MiB. Prefetch 256mb using 1 thread
    100%: 4,775,001,234 -> 869,669,401: 18.21%. I/O 34.960 sec, cpu 115 MiB/s (39.655 sec), real 14.8 MiB/s (308.539 sec) = 13%
    Archive directory: 4,835,389 bytes

    C:\Base\Bench-links>fa create m:\a -m1 -cfg- -t
    Scanning: 4,775,001,234 bytes in 6,096 folders and 106,397 files (RAM 8 MiB, I/O 0.203 sec, cpu 0.234 sec, real 0.436 sec)
    Compress using rep:512mb:c512+4x4:tor:3:8mb, $compressed => rep:512mb:c512
    Compression memory 833mb + 8*1mb read-ahead buffers, decompression memory 537..752 MiB. Prefetch 256mb using 1 thread
    100%: 4,775,001,234 -> 869,669,401: 18.21%. I/O 22.698 sec, cpu 93.3 MiB/s (48.797 sec), real 246 MiB/s (18.511 sec) = 264%
    Archive directory: 4,835,389 bytes
    Testing 4,775,001,234 bytes in 6,096 folders and 106,397 files
    100%: 869,669,397 -> 4,775,001,234: 18.21%. I/O 6.131 sec, cpu 206 MiB/s (22.059 sec), real 701 MiB/s (6.498 sec) = 339%

    C:\Base\Bench-links>fa create m:\a -m1 -t
    Scanning: 4,775,001,234 bytes in 6,096 folders and 106,397 files (RAM 8 MiB, I/O 0.218 sec, cpu 0.234 sec, real 0.441 sec)
    Compress using rep:512mb:c512+4x4:zstd:2:8mb, $compressed => rep:512mb:c512
    Compression memory 693mb + 8*1mb read-ahead buffers, decompression memory 528..674 MiB. Prefetch 256mb using 1 thread
    100%: 4,775,001,234 -> 824,689,803: 17.27%. I/O 16.942 sec, cpu 120 MiB/s (38.064 sec), real 274 MiB/s (16.603 sec) = 229%
    Archive directory: 4,835,391 bytes
    Testing 4,775,001,234 bytes in 6,096 folders and 106,397 files
    100%: 824,689,799 -> 4,775,001,234: 17.27%. I/O 0.733 sec, cpu 457 MiB/s (9.968 sec), real 1013 MiB/s (4.497 sec) = 222%



    Now let's compare -m2 with tor and zstd:

    C:\Base\Bench-links>fa create m:\a -m2 -cfg- -t
    Scanning: 4,775,001,234 bytes in 6,096 folders and 106,397 files (RAM 8 MiB, I/O 0.265 sec, cpu 0.203 sec, real 0.466 sec)
    Compress using rep:512mb:256:c256+exe+4x4:tor:5, $obj => rep:512mb:256:c256+4x4:tor:5, $text => grzip:8mb:m4:l32:h15, $wav => tta:m1, $bmp => mm+grzip:8mb:m4:l:a, $compressed => rep:512mb:256:c256+4x4:tor:3:8mb
    Compression memory 1224mb + 8*1mb read-ahead buffers, decompression memory 563..988 MiB. Prefetch 256mb using 1 thread
    100%: 4,775,001,234 -> 687,320,906: 14.39%. I/O 23.151 sec, cpu 37.2 MiB/s (122.554 sec), real 157 MiB/s (29.094 sec) = 421%
    Archive directory: 4,835,822 bytes
    Testing 4,775,001,234 bytes in 6,096 folders and 106,397 files
    100%: 687,320,906 -> 4,775,001,234: 14.39%. I/O 5.101 sec, cpu 106 MiB/s (43.072 sec), real 444 MiB/s (10.263 sec) = 420%


    C:\Base\Bench-links>fa create m:\a -m2 -t
    Scanning: 4,775,001,234 bytes in 6,096 folders and 106,397 files (RAM 8 MiB, I/O 0.250 sec, cpu 0.234 sec, real 0.478 sec)
    Compress using rep:512mb:256:c256+exe+4x4:zstd:7:16m:h8m, $obj => rep:512mb:256:c256+4x4:zstd:7:16m:h8m, $text => grzip:8mb:m4:l32:h15, $wav => tta:m1, $bmp => mm+grzip:8mb:m4:l:a, $compressed => rep:512mb:256:c256+4x4:zstd:2:8mb
    Compression memory 936mb + 8*1mb read-ahead buffers, decompression memory 544..835 MiB. Prefetch 256mb using 1 thread
    100%: 4,775,001,234 -> 661,845,563: 13.86%. I/O 19.578 sec, cpu 36.5 MiB/s (124.879 sec), real 157 MiB/s (29.041 sec) = 430%
    Archive directory: 4,835,842 bytes
    Testing 4,775,001,234 bytes in 6,096 folders and 106,397 files
    100%: 661,845,563 -> 4,775,001,234: 13.86%. I/O 2.418 sec, cpu 161 MiB/s (28.330 sec), real 504 MiB/s (9.028 sec) = 314%



    And finally lzma/based -m3/m4 for comparison. Note that only these modes start to use full CPU power (i.e. cpu/real time ratio became closer to 800%):

    C:\Base\Bench-links>fa create m:\a -m3 -t
    Scanning: 4,775,001,234 bytes in 6,096 folders and 106,397 files (RAM 8 MiB, I/O 0.203 sec, cpu 0.234 sec, real 0.439 sec)
    Compress using rep:512mb+exe+delta+4x4:lzma:96mb:fast:32:mc4, $obj => rep:512mb+delta+4x4:lzma:96mb:fast:32:mc4, $text => grzip:8mb:m2:l32:h15, $bmp => mm+grzip:8mb:m1:l2048:h15:a, $compressed => rep:512mb+4x4:zstd:2:8mb, $wav => tta
    Compression memory 3024mb + 8*1mb read-ahead buffers, decompression memory 718..2496 MiB. Prefetch 256mb using 1 thread
    100%: 4,775,001,234 -> 549,478,884: 11.51%. I/O 21.185 sec, cpu 20.6 MiB/s (221.459 sec), real 90.5 MiB/s (50.315 sec) = 440%
    Archive directory: 4,835,872 bytes
    Testing 4,775,001,234 bytes in 6,096 folders and 106,397 files
    100%: 549,478,884 -> 4,775,001,234: 11.51%. I/O 2.512 sec, cpu 80.5 MiB/s (56.550 sec), real 319 MiB/s (14.293 sec) = 396%

    C:\Base\Bench-links>fa create m:\a -m4 -t
    Scanning: 4,775,001,234 bytes in 6,096 folders and 106,397 files (RAM 8 MiB, I/O 0.187 sec, cpu 0.250 sec, real 0.440 sec)
    Compress using rep:512mb+dispack070+delta+4x4:lzma:96mb:normal:16:mc8, $obj => rep:512mb+delta+4x4:lzma:96mb:normal:16:mc8, $text => grzip:8mb:m1:l32:h15, $compressed => rep:512mb+4x4:zstd:7:16m:h8m, $wav => tta, $bmp => mm+grzip:8mb:m1:l2048:h15:a
    Compression memory 3082mb + 8*1mb read-ahead buffers, decompression memory 735..2514 MiB. Prefetch 256mb using 1 thread
    100%: 4,775,001,234 -> 491,786,880: 10.30%. I/O 24.367 sec, cpu 8.74 MiB/s (521.231 sec), real 49.2 MiB/s (92.479 sec) = 564%
    Archive directory: 4,835,888 bytes
    Testing 4,775,001,234 bytes in 6,096 folders and 106,397 files
    100%: 491,786,880 -> 4,775,001,234: 10.30%. I/O 3.198 sec, cpu 67.2 MiB/s (67.814 sec), real 244 MiB/s (18.643 sec) = 364%

  16. The Following User Says Thank You to Bulat Ziganshin For This Useful Post:

    Piglet (12th October 2016)

  17. #12
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    860
    Thanks
    441
    Thanked 169 Times in 80 Posts
    Please help me: I tried to define lepton for compressing JPEG images. In fa.ini, I wrote:

    [External compressor:lepton]
    mem = 1024
    packcmd = {compressor} {options} $$arcdatafile$$.jpg
    unpackcmd = {compressor} $$arcpackedfile$$.lep
    datafile = $$arcdatafile$$.jpg
    packedfile = $$arcpackedfile$$.lep
    solid = 0

    In Lua code, I wrote:

    onPostOption (function ()
    AddCompressionGroup("lepton:skipverify:memory=1024 M:allowprogressive:threadmemory=1600M:hugepages"," $jpeg")
    end)

    But there are still errors..

  18. #13
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    860
    Thanks
    441
    Thanked 169 Times in 80 Posts
    the same is with 7z. I defined:

    [External compressor:7z]
    mem = 704
    default = -mx9
    packcmd = 7za a {-option } $$arcpackedfile$$.7z $$arcdatafile$$.tmp
    unpackcmd = 7za e $$arcpackedfile$$.7z $$arcdatafile$$.tmp
    packedfile = $$arcpackedfile$$.7z
    datafile = $$arcdatafile$$.tmp

    How can I define 7z commandline like this in AddCompressionGroup: "7z -mx9 -m0=delta:2: -m1=ppmd32:mem500mb ?
    I need to use 7z because XZ does not allow using delta filter without compression and I need the delta filter and then PPMd.

  19. #14
    Member
    Join Date
    Sep 2015
    Location
    Italy
    Posts
    201
    Thanks
    89
    Thanked 110 Times in 80 Posts
    Quote Originally Posted by Stephan Busch View Post
    In Lua code, I wrote:

    onPostOption (function ()
    AddCompressionGroup("lepton:skipverify:memory=1024 M:allowprogressive:threadmemory=1600M:hugepages"," $jpeg")
    end)
    Try without the space before $jpeg --> "$jpeg".
    I'm happy to see Lua used in a compressor.

  20. #15
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    860
    Thanks
    441
    Thanked 169 Times in 80 Posts
    @Mauro Vezzosi

    that space is not present in my fa.ini - therefore this is not causing the JPEG error.
    Lepton runs trough all JPEG but resulting filesize is equal to input filesize.

    Does anyone have an idea how to use 7z with delta filter in fa?

  21. #16
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,483
    Thanks
    719
    Thanked 653 Times in 349 Posts
    Quote Originally Posted by Mauro Vezzosi View Post
    I'm happy to see Lua used in a compressor.
    and i happy to see Lua enthusiasts here there are few features that can be implemented by anyone:
    - compression method handling, in particular -lc, -lc, -md, -mm, -ms options
    - file filtering, including -ta/tb/tn/to and rar-like -e options

    yeah, i can do it myself, but you may get fun implementing these features. i also plan to move to the Lua side two larger fearures:
    - entire UI
    - -m option parsing, inlcuding parsing of method definitions

    in last days, i added encryption support - everything available in arc except for -hp option. the entire implementation took 150 lines in Lua and ~50 lines in C++ for interfacing with Lua part (of ourse, i have used existing arc libraries, so Lua part just orchestrates them). As part of this work, i added a Lua hook that allows to construct individual compression/ecryption method for each solid block, and then tried tio imagine other situation when it may be useful. And this morning one user on russian forum asked me whether it's possible to apply different compression methods based on total solid block size. bingo!


    and now completely different topic. this transcript shows that nanozip still very competetive, even to 64-bit fa+ztsd combo (i dropped useless nz configs such as -cf -m4g and -cF):

    Code:
    C:\Base\Bench-links>fa create m:\a -m1 -t
    Compression memory 693mb + 8*1mb read-ahead buffers, decompression memory 528..674 MiB.  Prefetch 256mb using 1 thread
    4,775,002,738 -> 824,692,600: 17.27%.  I/O 16.177 sec, cpu 125 MiB/s (36.379 sec), real 286 MiB/s (15.899 sec) = 229%
    Testing: I/O 0.640 sec, cpu 470 MiB/s (9.688 sec), real 1034 MiB/s (4.403 sec) = 220%
    
    C:\Base\Bench-links>fa create m:\a -m2 -t
    Compression memory 936mb + 8*1mb read-ahead buffers, decompression memory 544..835 MiB.  Prefetch 256mb using 1 thread
    4,775,005,880 -> 661,853,280: 13.86%.  I/O 19.001 sec, cpu 37.8 MiB/s (120.495 sec), real 169 MiB/s (26.908 sec) = 448%
    Testing: I/O 1.966 sec, cpu 163 MiB/s (27.987 sec), real 535 MiB/s (8.515 sec) = 329%
    
    C:\Base\Bench-links>fa create m:\a -m2d -t
    Compression memory 936mb + 8*1mb read-ahead buffers, decompression memory 544..835 MiB.  Prefetch 256mb using 1 thread
    4,775,005,880 -> 671,663,252: 14.07%.  I/O 17.940 sec, cpu 38.0 MiB/s (119.949 sec), real 168 MiB/s (27.029 sec) = 444%
    Testing: I/O 0.905 sec, cpu 443 MiB/s (10.280 sec), real 832 MiB/s (5.475 sec) = 188%
    
    
    
    
    C:\Base\Bench-links>timer nz a m:\a -r -t8 -y -cf
    NanoZip 0.09 alpha/Win64  (C) 2008-2011 Sami Runsas  www.nanozip.net
    Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz|3395 MHz|#4+HT|21217/32698 MB
    Threads: 8, memory: 512 MB, IO-buffers: 20+4 MB
    Compressor #0: nz_lzpf [70 MB]
    Compressed 4 775 001 809 into 1 270 844 437 in 0.00s, 4447 GB/s
    IO-in: 10.76s, 423 MB/s. IO-out: 0.69s, 1756 MB/s
    Kernel Time  =    10.358 = 00:00:10.358 = 138%
    User Time    =    38.797 = 00:00:38.797 = 519%
    Process Time =    49.155 = 00:00:49.155 = 657%
    Global Time  =     7.473 = 00:00:07.473 = 100%
    
    C:\Base\Bench-links>timer nz t m:\a -t8
    Decompressed 4 775 001 809 bytes in 2.62s, 1735 MB/s.                           
    IO-in: 0.19s, 264 MB/s.
    Kernel Time  =     2.215 = 00:00:02.215 =  47%
    User Time    =    28.438 = 00:00:28.438 = 607%
    Process Time =    30.654 = 00:00:30.654 = 655%
    Global Time  =     4.680 = 00:00:04.680 = 100%
    
    
    
    
    C:\Base\Bench-links>timer nz a m:\a -r -t8 -y -cd
    Threads: 8, memory: 512 MB, IO-buffers: 20+4 MB
    Compressor #0: nz_lzhd [69 MB]
    Compressed 4 775 002 738 into 778 548 468 in 5.02s, 907 MB/s
    IO-in: 9.24s, 492 MB/s. IO-out: 0.47s, 1563 MB/s
    Kernel Time  =    11.278 = 00:00:11.278 =  71%
    User Time    =    92.820 = 00:01:32.820 = 591%
    Process Time =   104.099 = 00:01:44.099 = 663%
    Global Time  =    15.694 = 00:00:15.694 = 100%
    
    
    C:\Base\Bench-links>timer nz t m:\a -t8
    Compressor #6: nz_lzhd [46 MB]
    Decompressed 4 775 002 738 bytes in 2.44s, 1863 MB/s.  
    IO-in: 0.38s, 85 MB/s.
    Kernel Time  =     4.149 = 00:00:04.149 =  88%
    User Time    =    27.612 = 00:00:27.612 = 586%
    Process Time =    31.761 = 00:00:31.761 = 674%
    Global Time  =     4.711 = 00:00:04.711 = 100%
    
    
    
    
    
    
    C:\Base\Bench-links>timer nz a m:\a -r -t8 -y -cd -m4g
    Threads: 8, memory: 4096 MB, IO-buffers: 20+4 MB
    Compressor #0: nz_lzhd [505 MB]
    Compressed 4 775 005 066 into 648 548 550 in 0.00s, 4447 GB/s
    IO-in: 22.17s, 205 MB/s. IO-out: 0.34s, 1808 MB/s
    Kernel Time  =    11.325 = 00:00:11.325 =  58%
    User Time    =    93.007 = 00:01:33.007 = 480%
    Process Time =   104.333 = 00:01:44.333 = 539%
    Global Time  =    19.344 = 00:00:19.344 = 100%
    
    
    C:\Base\Bench-links>timer nz t m:\a -t8
    Compressor #6: nz_lzhd [354 MB]
    Decompressed 4 775 005 066 bytes in 2.43s, 1870 MB/s.              
    IO-in: 0.36s, 90 MB/s.
    Kernel Time  =     5.366 = 00:00:05.366 = 112%
    User Time    =    24.570 = 00:00:24.570 = 513%
    Process Time =    29.936 = 00:00:29.936 = 625%
    Global Time  =     4.789 = 00:00:04.789 = 100%
    
    
    
    
    
    C:\Base\Bench-links>timer nz a m:\a -r -t8 -y -cD -m4g
    Threads: 8, memory: 4096 MB, IO-buffers: 20+4 MB
    Compressor #0: nz_lzhds [507 MB]
    Compressed 4 775 007 505 into 591 674 965 in 15.21s, 299 MB/s
    IO-in: 30.05s, 152 MB/s. IO-out: 0.45s, 1229 MB/s
    Kernel Time  =    12.792 = 00:00:12.792 =  26%
    User Time    =   282.549 = 00:04:42.549 = 593%
    Process Time =   295.341 = 00:04:55.341 = 620%
    Global Time  =    47.612 = 00:00:47.612 = 100%
    
    C:\Base\Bench-links>timer nz t m:\a -t8
    Compressor #7: nz_lzhds [275 MB]
    Decompressed 4 775 007 505 bytes in 2.69s, 1688 MB/s.
    IO-in: 0.36s, 80 MB/s.
    Kernel Time  =     4.633 = 00:00:04.633 =  91%
    User Time    =    26.878 = 00:00:26.878 = 533%
    Process Time =    31.512 = 00:00:31.512 = 625%
    Global Time  =     5.038 = 00:00:05.038 = 100%

    dispack+lzma combo looks pretty competetive against nanozip lz-compression:
    Code:
    C:\Base\Bench-links>fa create m:\a -m3 -t
    Compress using rep:512mb+exe+delta+4x4:lzma:96mb:fast:32:mc4...
    Compression memory 3024mb + 8*1mb read-ahead buffers, decompression memory 718..2496 MiB. Prefetch 256mb using 1 thread
    4,775,001,234 -> 549,478,884: 11.51%. I/O 21.185 sec, cpu 20.6 MiB/s (221.459 sec), real 90.5 MiB/s (50.315 sec) = 440%
    Testing: I/O 2.512 sec, cpu 80.5 MiB/s (56.550 sec), real 319 MiB/s (14.293 sec) = 396%
    
    C:\Base\Bench-links>fa create m:\a -m4 -t
    Compress using rep:512mb+dispack070+delta+4x4:lzma:96mb:normal:16:mc8...
    Compression memory 3082mb + 8*1mb read-ahead buffers, decompression memory 735..2514 MiB. Prefetch 256mb using 1 thread
    4,775,001,234 -> 491,786,880: 10.30%. I/O 24.367 sec, cpu 8.74 MiB/s (521.231 sec), real 49.2 MiB/s (92.479 sec) = 564%
    Testing: I/O 3.198 sec, cpu 67.2 MiB/s (67.814 sec), real 244 MiB/s (18.643 sec) = 364%
    
    
    C:\Base\Bench-links>timer nz a m:\a -r -t8 -y -cdP -m2g
    Threads: 8, memory: 2048 MB, IO-buffers: 20+4 MB
    Compressor #0: nz_lzhd_parallel_extra [1995 MB]
    Compressed 4 775 009 125 into 537 931 839 in 1m 25.11s, 54 MB/s
    IO-in: 1.56s, 2915 MB/s. IO-out: 0.17s, 2948 MB/s
    Kernel Time  =    27.705 = 00:00:27.705 =  30%
    User Time    =   343.249 = 00:05:43.249 = 383%
    Process Time =   370.954 = 00:06:10.954 = 414%
    Global Time  =    89.545 = 00:01:29.545 = 100%
    
    C:\Base\Bench-links>timer nz t m:\a -t8
    Compressor #0: nz_lzhd [1410 MB]
    Decompressed 4 775 009 125 bytes in 9.88s, 461 MB/s.
    IO-in: 0.31s, 1629 MB/s.
    Kernel Time  =     3.853 = 00:00:03.853 =  36%
    User Time    =    16.582 = 00:00:16.582 = 157%
    Process Time =    20.436 = 00:00:20.436 = 193%
    Global Time  =    10.561 = 00:00:10.561 = 100%
    and, finally, best lzma compression against nz -co:
    Code:
    C:\Base\Bench-links>C:\!FreeArc\private\FB\fb011\!release\fa create m:\a -m9d -t -mcrep/rep:1g
    Compress using rep:1gb+dispack070+delta+lzma:254mb:normal:bt4:128..
    Compression memory 3735mb + 8*1mb read-ahead buffers, decompression memory 1304 MiB.  Prefetch 256mb using 1 thread
    4,775,011,602 -> 395,748,324: 8.29%.  I/O 21.700 sec, cpu 4.07 MiB/s (1120.087 sec), real 6.26 MiB/s (727.418 sec) = 154%
    Testing: I/O 1.435 sec, cpu 150 MiB/s (30.311 sec), real 142 MiB/s (31.964 sec) = 95%
    
    
    C:\Base\Bench-links>timer nz a m:\a -r -t8 -y -co -m4g
    Threads: 8, memory: 4096 MB, IO-buffers: 4+1 MB
    Compressor #0: nz_optimum1 [506 MB]
    Compressed 4 775 011 602 into 467 621 035 in 3m 16.12s, 23 MB/s
    IO-in: 21.89s, 208 MB/s. IO-out: 0.17s, 2520 MB/s
    Kernel Time  =    39.327 = 00:00:39.327 =  17%
    User Time    =  1355.258 = 00:22:35.258 = 604%
    Process Time =  1394.586 = 00:23:14.586 = 622%
    Global Time  =   224.158 = 00:03:44.158 = 100%
    
    
    C:\Base\Bench-links>timer nz t m:\a -t8
    Compressor #6: nz_optimum1 [408 MB]
    Decompressed 4 775 011 602 bytes in 28.56s, 159 MB/s.           
    IO-in: 0.21s, 106 MB/s.
    Kernel Time  =     1.684 = 00:00:01.684 =   5%
    User Time    =   161.757 = 00:02:41.757 = 515%
    Process Time =   163.442 = 00:02:43.442 = 521%
    Global Time  =    31.357 = 00:00:31.357 = 100%
    Last edited by Bulat Ziganshin; 13th October 2016 at 17:53.

  22. The Following User Says Thank You to Bulat Ziganshin For This Useful Post:

    Stephan Busch (14th October 2016)

  23. #17
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,483
    Thanks
    719
    Thanked 653 Times in 349 Posts
    Quote Originally Posted by Stephan Busch View Post
    the same is with 7z. I defined:
    Quote Originally Posted by Stephan Busch View Post
    [External compressor:7z]
    mem = 704
    default = -mx9
    packcmd = 7za a {-option } $$arcpackedfile$$.7z $$arcdatafile$$.tmp
    unpackcmd = 7za e $$arcpackedfile$$.7z $$arcdatafile$$.tmp
    packedfile = $$arcpackedfile$$.7z
    datafile = $$arcdatafile$$.tmp

    How can I define 7z commandline like this in AddCompressionGroup: "7z -mx9 -m0=delta:2: -m1=ppmd:o32:mem500mb ?
    I need to use 7z because XZ does not allow using delta filter without compression and I need the delta filter and then PPMd.
    ok, first we need to translate 7-zip compression method with parameters into freearc external method:

    Code:
    [External compressor:7delta]
    mem = 1
    default = 2
    packcmd = 7za a -m0=delta{:option} $$arcpackedfile$$.7z $$arcdatafile$$.tmp
    unpackcmd = 7za e $$arcpackedfile$$.7z $$arcdatafile$$.tmp
    packedfile = $$arcpackedfile$$.7z
    datafile = $$arcdatafile$$.tmp
    you can define 7ppmd in the same way. and then use AddCompressionGroup("7delta:2+7ppmd:o32:mem500m"," $xx") - the "{:option}" struing in command definition means that 7delta:opt1:opt2:opt3 will be replaced with :opt1:opt2:opt3 inserted here

    this way 7-zip will be executed individually for each compression method - that's because freearc can't build cmdline with -m0 and -m1 automatically, nor it can split single compression method into separate params for delta and ppmd. it may become doable in future with customary compression methods, but i have no idea yet.

    but instead of 7zip delta you can use my own mm filter, in particular delta:2 == mm:2*8, although mm:1*16 may compress even better. full list of mm params:

    Code:
    d# -- detection speed mode (1 - fastest, 9 - most accurate)
    s  -- skip WAV header detection
    f  -- floating-point data format
    c# -- channels count
    w# -- word size, in bits (8/16)
    o# -- offset of MM data in file (=header size)
    r# -- reorder data. -r1 - reorder bytes, -r2 - reorder words (unfinished!)
    c*w -- use c channels w bits each (example: 3*8)
    o+c*w -- use c channels w bits each starting from offset o

  24. The Following 2 Users Say Thank You to Bulat Ziganshin For This Useful Post:

    RamiroCruzo (14th October 2016),Stephan Busch (14th October 2016)

  25. #18
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,483
    Thanks
    719
    Thanked 653 Times in 349 Posts
    BSC:
    Code:
    [External compressor:bsc]
    packcmd = bsc e $$arcdatafile$$.tmp $$arcpackedfile$$.tmp {options}
    unpackcmd = bsc d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
    "{options}" used here is equivalent to "{-option }" so f.e. "bsc:m4:b50" is extended to "... -m4 -b50" command line



    XZ used for fast lzma2 implementation:
    Code:
    [External compressor:lzma2]
    packcmd   = xz -z {options} <stdin> <stdout>
    unpackcmd = xz -d <stdin> <stdout>


  26. The Following 2 Users Say Thank You to Bulat Ziganshin For This Useful Post:

    RamiroCruzo (14th October 2016),Stephan Busch (14th October 2016)

  27. #19
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    860
    Thanks
    441
    Thanked 169 Times in 80 Posts
    how do I define _rawzor.exe?

    I tried this and it produces an output archive that has identical size as input; it seems rawzor output was not catched up:

    [External compressor:rawzor]
    mem = 2
    packcmd = _rawzor -c -i $$arcdatafile$$.tmp -o $$arcpackedfile$$.rwz
    unpackcmd = _rawzor -d $$arcpackedfile$$.rwz
    solid = 0

    Edit: I figured out this on my own
    Last edited by Stephan Busch; 21st October 2016 at 16:10.

  28. #20
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    860
    Thanks
    441
    Thanked 169 Times in 80 Posts
    At present, I have integrated the following compressors in fa.ini:

    * 7-Zip
    * BSC
    * CsArc
    * FLIF
    * Lepton
    * Precomp 0.4.5
    * Rawzor
    * SREP
    * XZ

    I also tried to configure packPNM and PGFconsole, but they have issues:
    With packPNM, you must press enter after each compressed image;
    With PGFconsole the image cannot be decoded. I have no workaround so far.

    the curent draft is attahed to this post.

    How can I activate custom ZSTD options in fa.ini?

    I tried using them in a group like this:

    zstd:22:d27m:c27m:h25m:l9:b4:t512:s7 but the results are always the same no matter which value I changed.
    What are the maximum values for each setting?
    I also tried without level 22 (:22).
    Attached Files Attached Files

  29. The Following 2 Users Say Thank You to Stephan Busch For This Useful Post:

    avitar (21st October 2016),Bulat Ziganshin (21st October 2016)

  30. #21
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,483
    Thanks
    719
    Thanked 653 Times in 349 Posts
    zstd:22:d27m:c27m:h25m:l9:b4:t512:s7
    note that d/c/h/l values are round down to nearest power of 2. some values may be meaningless or unused with some strategies. d27m means 27 MByte, you probably want d27 without 'm'. maximum values should be asked from zstd authors

    also, you can condense your code to
    Code:
    onPostOption (function ()
        AddCompressionGroup("flif-ppm","$24bit")
        AddCompressionGroup("flif-pam","$32bit")
        AddCompressionGroup("7z-Delta1","$8bit")
    ...
        AddCompressionGroup("lzma:512m:273:lc5:lp0:pb0","$xbg")
    end)
    read a http://www.lua.org/pil/contents.html - it needs a one day and will allow to make fancier tricks with FA

  31. The Following 2 Users Say Thank You to Bulat Ziganshin For This Useful Post:

    avitar (21st October 2016),Stephan Busch (21st October 2016)

  32. #22
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    860
    Thanks
    441
    Thanked 169 Times in 80 Posts
    I am using this for compressed2 group:

    AddCompressionGroup("precomp045+zstd:22","$compres sed2")

    Which custom settings for zstd would be better from your point of view?

  33. #23
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    860
    Thanks
    441
    Thanked 169 Times in 80 Posts
    Lua is quite complex to me - so it would need much more than a day.

    If I want to replace compressor "rep" with "srep" in default group, what do I have to enter in fa.ini?

    Please help me.

    ZSTD parameters for FA are described on that page:
    https://github.com/Bulat-Ziganshin/F...s#zstd-support

    but compression is always identical - no matter which parameter I am changing.

  34. #24
    Member
    Join Date
    Oct 2016
    Location
    south africa
    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts
    What about cool gui not like the last freearc that was good but not cool...

    And what if you insert every external compressor and precompressor in fa with almost every compression command and make it advanced free arc next???
    By doing that user not have to tweak ini again and again.....
    So
    It's just an request or something like that......

    And can you tell me which is other your software in market like your profession is programmer you should be in some company... Have there is any software or program is made by you and available in market......

  35. #25
    Member
    Join Date
    Oct 2016
    Location
    south africa
    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    and this

    And how can I use freearc next with This

  36. #26
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,483
    Thanks
    719
    Thanked 653 Times in 349 Posts
    Which custom settings for zstd would be better from your point of view?
    i don't know muc about zstd except that it has fast decompression and levels up to 22. I can also point you to default settings for each compression level. You can try to ask Cyan, but you need to detail your question, may be "best compression settings for already compressed data" or so


    If I want to replace compressor "rep" with "srep" in default group, what do I have to enter in fa.ini?
    Code:
    onPostOption (function ()                          
      MapCompressionGroups (function (method, group)        
        if group == "$default" then
          return map_compressors(function(name) if name=="rep" then return "srep:m3f" end)
        end
      end)                                           
    end)
    it also can be done by option "-mc$default:rep/srep:m3f". I plan to provide some way to "execute" options from Lua code, but it's not yet supported


    but compression is always identical - no matter which parameter I am changing.
    it works for me:
    Code:
    C:\>fa.exe create m:\a z:\e8 -mz:5:d1m
    Compression memory 4mb + 8*1mb read-ahead buffers, decompression memory 1 MiB.  Prefetch 256mb using 1 thread
    100%: 100,000,000 -> 34,969,271: 34.97%.  I/O 0.047 sec, cpu 67.9 MiB/s (1.404 sec), real 59.7 MiB/s (1.598 sec) = 88%
    
    C:\>fa.exe create m:\a z:\e8 -mz:5:d2m
    Compression memory 5mb + 8*1mb read-ahead buffers, decompression memory 2 MiB.  Prefetch 256mb using 1 thread
    100%: 100,000,000 -> 34,862,204: 34.86%.  I/O 0.031 sec, cpu 64.4 MiB/s (1.482 sec), real 67.3 MiB/s (1.417 sec) = 105%
    
    C:\>fa.exe create m:\a z:\e8 -mz:5:d2m:h4m
    Compression memory 8mb + 8*1mb read-ahead buffers, decompression memory 2 MiB.  Prefetch 256mb using 1 thread
    100%: 100,000,000 -> 34,693,280: 34.69%.  I/O 0.094 sec, cpu 54.1 MiB/s (1.763 sec), real 55.4 MiB/s (1.720 sec) = 102%






    And what if you insert every external compressor and precompressor in fa with almost every compression command and make it advanced free arc next???
    it is what Stephan now does. And i work on the program itself. ASC authors probably will answer your last question, it's part of industry built around freearc





  37. The Following User Says Thank You to Bulat Ziganshin For This Useful Post:

    Stephan Busch (25th October 2016)

  38. #27
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    860
    Thanks
    441
    Thanked 169 Times in 80 Posts
    I wanted to use custom zstd settings in fa.ini and with -mx switch.

    How can I add further codecs to default group (p.ex. Precomp)?

    The code from you causes an error message:

    onPostOption (function ()
    MapCompressionGroups (function (method, group)
    if group == "$default" then
    return map_compressors(function(name) if name=="rep" then return "srep:m3f" end)
    end
    end)
    end)
    Last edited by Stephan Busch; 26th October 2016 at 10:36.

  39. #28
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,483
    Thanks
    719
    Thanked 653 Times in 349 Posts
    Code:
    onPostOption (function ()                          
      MapCompressionGroups (function (method, group)        
        if group == "$default" then
          return map_compressors(method, function(name) if name=="rep" then return "srep:m3f" end end)
        end
      end)                                           
    end)
    first error is syntax: skipped one more "end"
    second one (skipped "method") can be seen from comparison to map_compressors definition:

    -- Map each compressor in the `method` through `filter_f`, removing empty strings from result.
    function map_compressors (method, filter_f, ...)



    How can I add further codecs to default group (p.ex. Precomp)?
    Code:
    onPostOption (function ()                          
      MapCompressionGroups (function (method, group)        
        if group == "$default" then
          return "precomp+"..method
        end
      end)                                           
    end)


    I wanted to use custom zstd settings in fa.ini and with -mx switch.
    -mx is really not a switch of its own, but "-m" option selecting "x" compression method. so it's defined by the following definitions printed by --print-config:

    Code:
    [Compression methods]
    x  = 9            ;highest compression mode using only internal algorithms
    #  = #rep+#exe+#xb / $obj=#b / $text=#t
    #t  = dict:p:128m:80% + lzp:160m:145:d1m:s32:h23:92% + ppmd:16:384m
    
    #b  = #rep + #bx
    #bx = #xb
    #xb = delta + #binary
    
    9rep  = rep:2040m
    9binary = lzma:254m:max
    
    #exe = exe2
    exe2 = dispack
    dispack = dispack070
    where every line containing '#" is replaced by 9 lines with each '#' mapped to '1' .. '9' (but concrete definitions like "1rep=..." has priority over general ones like "#rep=...")

    You can replace any of these definitions using "Compression methods" section in your fa.ini. Try for example
    Code:
    [Compression methods]
    9rep  = precomp+srep:m3f
    9binary = zstd:22:d27

  40. The Following User Says Thank You to Bulat Ziganshin For This Useful Post:

    Stephan Busch (28th October 2016)

  41. #29
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    860
    Thanks
    441
    Thanked 169 Times in 80 Posts
    Speaking of GUI:

    Bulat, if you want, you can use EcoZip GUI (in whole or parts of it) for FA.
    I own the rights of it.

  42. #30
    Programmer Bulat Ziganshin's Avatar
    Join Date
    Mar 2007
    Location
    Uzbekistan
    Posts
    4,483
    Thanks
    719
    Thanked 653 Times in 349 Posts
    Version 0.11 displays a lot of info about operation performed. It was useful for myself during development. In particular, the compression progress indicator looks as:

    Code:
    100%: 41,463,554 -> 100,000,000: 41.46%.  I/O 0.250 sec, cpu 322 MiB/s (0.296 sec), real 368 MiB/s (0.259 sec) = 114%.  Remains 00:00
    Two problems is that it's too wide (this one is 133 chars long) and probably contains much more info that's interesting for the most of you. In the next version, i want to display shorter indicator by default. Ideally, it should fit in the standard 80-column display. So far, i was able to produce the following design:

    Code:
    100%: 100,000,000,000 -> 041,463,554,000: 41.46%.  368 MiB/s = 114%.  ETA 00:00
    As you see, it lacks any times and most of descriptive texts. It's exactly 79 chars long (with filesizes of hundreds of GBs, though). Well, since the entire UI is now in Lua, it's completely configurable, but i still want to have reasonable default. What you think about it? Should i drop something more, add more descriptive texts ("cpu", "real", "ratio"...), or just keep that design?
    Last edited by Bulat Ziganshin; 6th January 2017 at 15:17.

  43. The Following User Says Thank You to Bulat Ziganshin For This Useful Post:

    Cyan (3rd May 2017)

Page 1 of 3 123 LastLast

Similar Threads

  1. FreeArc
    By Bulat Ziganshin in forum Data Compression
    Replies: 3159
    Last Post: 13th October 2018, 03:23
  2. FreeArc 'Next license
    By Bulat Ziganshin in forum Data Compression
    Replies: 42
    Last Post: 19th April 2015, 21:09
  3. How freearc could be even better
    By calthax in forum Data Compression
    Replies: 4
    Last Post: 17th August 2014, 18:40
  4. How to use MM with Freearc?
    By danswano in forum Data Compression
    Replies: 3
    Last Post: 22nd September 2013, 11:19
  5. How to use 4x4 with Freearc?
    By danswano in forum Data Compression
    Replies: 9
    Last Post: 21st September 2013, 19:50

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •