FreeArc 0.40 released

[Bulat Ziganshin]

btw, arc.ini settings for rzm:

[External compressor:rzm]
cmem = 260
dmem = 130
packcmd = rzm c $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = rzm d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp

[tonifelfe]

Hi,

is there a option to pause/continue operation?
Would be a very good idea!

[JangoFatXL]

the Windows Gui has a pause option...

[Bulat Ziganshin]

our local TV heard rumours about winarc and today they captured in my room video-topic for our city news program

[LovePimple]

Excellent!

[encode]

our local TV heard rumours about winarc and today they captured in my room video-topic for our city news program

Im very curious about that...

[giorgiotani]

our local TV heard rumours about winarc and today they captured in my room video-topic for our city news program

Great!
Let us know if something about the interview is available online!

[Vacon]

Hello everyone,

some people say "TV makes stupid!"
Well, that's the proof: these people are wrong!
Give us some link to the clip if it's avaible!

Best regards!

[Black_Fox]

Congratulations Bulat!

[Vacon]

Hello everyone,

btw. what happened to wArc

Best regards!

[Bulat Ziganshin]

it was shown, but i don't made a record. anyway, it was not very "smart", but more like "interesting people around us". my friend sold me as "best programmer in the world" and you know that "best world archiver" (which also isn't quite true) will say nothing to them

but imagine: 8:00 am. phone call:
- are you Bulat?
- yes
- is it true that you are best programmer in the world?

btw, on interview i've said that best compression algorithms are written in Russia, and they showed this, but clipped out part when i said that main fa part are these algorithms

[Bulat Ziganshin]

mirror of warc site: http://mega-set.com/arc/eng/

i will update links

[Vacon]

Hello everyone,

the best programmer sometimes is nothing if noone finds a way to assamble his code to a practical suite. Take Bill for an (kind of problematic) example. Not the best programmer, but who cares nowadays...
And, please, do not try to hide your talent

Best regards!

[Bulat Ziganshin]

yes, i'm not best programmer, but best manager of algorithms. it's what i completely agree

and i don't hide myself. you should agree that Christian is best programmer in this area for the moment but he doesn' seek for a ways to make easier using of his algos

OTOH, when i said that i'm 15 years in this business, they decided that i've developed my super-algorithm all these 15 years i've tried to describe to them that only cooperation and copying of good ideas between programs allows to create best real program but they was not interested to hear and especially show this. if i will say them that i'm only good manager, they will be completely disappointed

Hi Bulat, just got error -> "CompressioLib_d7eF:interrupted" , when trying compress 9gb with FA 0.50a -mx . What is this about?

[Vacon]

Hello everyone,

yes, im not best programmer, but best manager of algorithms. its what i completely agree

Somehow I agree too, most people are not interested in difference between programmer and manager -> for them its the same.

you should agree that Christian is best programmer in this area

This forum here is a melting-pot for a real big number of really competitive TOP-programmers! Allthough I have to admit that Christian is surprising us again and again
Lets not forget about Ilia, the founder of this forum, who developed some well done little progs:
- pimple
- Pim
- Lzpm
- quad
- lzpx
and still keeps the wheel turning!
Francesco has to be mentioned too, vcore, donotdisturb, Matt, ...

Best regards!

[Bulat Ziganshin]

just got error -> "CompressioLib_d7eF:interrupted" , when trying compress 9gb with FA 0.50a -mx . What is this about?

probably its error in one compression algorithms which lead to its crash. can you lease try to find smaller chunk of data which is till fail?

in particular, run program with -di+# switch on the same data and put its *logfile* here

Error at 10% of progress.

tmp size after crash:
$$temparc$$0.tmp ~1Mb
$$arcdatafile$$.tmp ~5Gb

end of log:

 
Compressing 82 files of 3.162.776 bytes: 20.48 secs 
  Using dict:3090kb:80%:l8192:m400:s100+lzp:3090kb:92%:265:h22:d1mb+ppmd:22:1gb 
  Memory for compression 1026mb, decompression 1026mb 
  Solid block compression results (1.969 seconds) 
    dict:3090kb:80%:l8192:m400:s100: 3.162.780 bytes in 0.188 seconds 
    lzp:3090kb:92%:265:h22:d1mb: 2.788.304 bytes in 0.234 seconds 
    ppmd:22:1gb: 1.131.276 bytes in 1.547 seconds 
Compressing 713 files of 8.916.444.724 bytes: 22.84 secs 
  Using exe+rep:512mb+tempfile+lzma:96mb:max:bt4:128 
  Memory for compression 1044mb, decompression 512mb

I guess rep fails.
Compressed with option -mc-rep, all ok.

btw
tryed compressing only that file( which i see in gui + error) no error occured.

[Nania Francesco]

Hi Bulat, wanted to ask you if according to you would be possible to write the code of Fpaq0 to work in Multi-thread or not!

[Bulat Ziganshin]

oh, it seems that rep still have some, very rare bug. i should try it hard on large files, may be i will find anything.

Nania, i don't seen fpaq0 code, but believe that it's impossible. if aricoder puts serious limitation on your program performance, you may try to compress data in small independent chunks (well, it's obvious), try to use huffman coder, use separate threads for modeling and encoding...

[Nania Francesco]

Ok! Thanks Bulat! hi!

[Patchy]

Does the file format documentation exist? I'm curious why the encryption scheme details are listed twice in the end of file with different values of salt, iv and other things. (ARC 0.40 Jan 1 2008)

[Bulat Ziganshin]

no, you may only use C/haskell sources as source of info

i guess that you've used -hp -dm0 keys, in this case you see encryption parameters for the file and archive directory. add -mlzma -dm=ppmd params to differentiate between them

[Patchy]

Yes, thanks, it all makes sense now. I used "-m2 -hp" though and both parameter sets were there. By the way, what defines that two byte parameter 'c'? Is it something a cipher in CTR mode of operation use?

[Bulat Ziganshin]

1. i'm interested to know why you asking. are you develop some tool for .arc files?

2. you may look into 7-zip format description for general idea. arc contains only a small number of improvements after it

3. are you know Russian?

4. 'c' is check code of PKCS algorithm:

This two-byte value is produced as part of the process that derives the encryption and decryption keys from the password. When encrypting, a verification value is derived from the encryption password and stored with the encrypted file. Before decrypting, a verification value can be derived from the decryption password and compared to the value stored with the file, serving as a quick check that will detect most, but not all, incorrect passwords. There is a 1 in 65,536 chance that an incorrect password will yield a matching verification value; therefore, a matching verification value cannot be absolutely relied on to indicate a correct password.

[Patchy]

1. I'm not sure if I'm able to contribute as a developer to this project. I've been looking for a versatile CLI utility for convenient and secure transferring files on a flash drive between computers. Key file based strong encryption and robust archive update functions provided by FreeArc are pretty much everything I could hope for. (Not mentioning very impressive compression techniques. But that's just icing on the cake in my case.) So, when I checked what meta information is being written in plain text I found duplication of parameter sets somewhat puzzling. Trying to figure it out I ended up finding this thread.

2. Thanks.

3. Yes.

4. Got it.

[Bulat Ziganshin]

i not asked for help, your questions just let me thought that you plans to write some tool for .arc format, probably password cracker

btw, PeaZip is archive format targeted to maximum security. unlike arc, it provides archive authentication and stronger hash functions

if you want to know more about ideas behind arc format, i can send you my development document describing them (but it says nothing about encryption)

encryption in arc is pretty standard and follows all guidelines i know - first, FORTUNA cryptographic PRNG, initialized by system-wide information, used to generate salts/ivs. second, PBKDF-PKCS#2 used to derive key+checkcode from password+keyfile+salt - key generation time is ~0.1 sec. then data are encrypted using CTR streaming mode

btw, where you hear about FreeArc?

[Patchy]

password cracker

I dont think that "password cracking" is even remotely reasonable when key strengthening and proper block ciphers are used. Usually, said "crackers" exploit known weaknesses of a certain blatantly insecure encryption scheme, which is not the case with FreeArc.

if you want to know more about ideas behind arc format, i can send you my development document describing them (but it says nothing about encryption)

No, thanks. I guess, eventually you will come up with better documentation for FreeArc, and I can wait until then.

encryption in arc is pretty standard...

Glad to hear that. Thanks for such detailed explanation.

btw, where you hear about FreeArc?

http://sf.net/search/?type_of_search=soft&words=twofish

[Bulat Ziganshin]

Usually, said "crackers" exploit known weaknesses

crackers for secure formats (such as rar-aes and zip-aes256) just make bruteforce searching and dictionary search. the same may be implemented for freearc. if you use random-generated keyfile such crackers are no problem as far as they dont steal your keyfile

[Bulat Ziganshin]

being amazed by the good results of tornado due to its huge dictionary, i've added its match finder to lzma, which allows us to combine world-best lz77 modeling with a huge dictionaries. typical results (using the same amount of memory):

>Arc.exe a a dll700.dll -mlzma:fast:64m:mc8
Compressed 1 file, 690.514.620 => 208.166.986 bytes. Ratio 30.1%
Compression time 758.71 secs, speed 910 kb/s. Total 835.55 secs

>Arc.exe a a dll700.dll -mlzma:fast:256m:mc8:ht4
Compressed 1 file, 690.514.620 => 196.385.118 bytes. Ratio 28.4%
Compression time 764.25 secs, speed 904 kb/s. Total 861.14 secs

:ht4 here means my new match finder, "hash table"