Disassembled LZTurbo.exe 0.92...

[Raymond_NGhM]

Here, you are Encoders...
AND now... I say Welcome Decoders.

Download LZTturbo092.7z

[eugene]

from tornado 0.3:

LZ77_Coder.cpp:
uint extra_dbits[32] = {4,4,5,5,5,6,6,7,7,8,8,9,9,10,10,11,11,12,12,13,13 ,14,14,15,15,16,17,18,19,21,23,30};

from lzturbo asm as provided by raymond::

L0044B640:

db 04h;

db 04h;

db 05h;

db 05h;

db 05h;

db 06h;

db 06h;

db 07h;

db 07h;

db 08h;

db 08h;

db 09h;

db 09h;

db 0Ah;

db 0Ah;

db 0Bh;

db 0Bh;

db 0Ch;

db 0Ch;

db 0Dh;

db 0Dh;

db 0Eh;

db 0Eh;

db 0Fh;

db 0Fh;

db 10h;

db 11h;

db 12h;

db 13h;

db 14h;

db 16h;

db 18h;

the code look surprisingly similar, but also note:

- the last 3 bytes are different

- tornado uses int vs. lzturbo uses char as datatype

- this buffer is nothing else than a representation of the lz77 deflate 64 distance tree... but all implementations i found start with value 0 not 4... so whatever source or paper both authors based their tree on, i guess its the same and would like to know which one.

tired and lazy i give up...

bulat: please can you tell me why the buffer starts with value 4?

[Bulat Ziganshin]

<div class="jscript"><pre>extra_lbits2:
zip: {0,0,0,0,0,0,0,0,1,1,1,1,2,2,2,2,3,3,3,3,4,4,4,4,5 ,5,5,5,0}
tor: {0,0,0,0,0,0,0,1,1,2,2,3,3,4,8,30};
lzt: 00 00 01 01 02 02 03 03 04 04 05 05 06 07 08 18

extra_dbits:
zip: {0,0,0,0,1,1,2,2,3,3,4,4,5,5,6,6,
7,7,8,8,9,9,10,10,11,11,12,12,13,13}
tor: {4,4,5,5,5,6,6,7,7,8,8,9,9,10,10,11,
11,12,12,13,13,14,14,15,15,16,17,18,19,21,23,30}
lzt: 04 04 05 05 05 06 06 07 07 08 08 09 09 0A 0A 0B
0B 0C 0C 0D 0D 0E 0E 0F 0F 10 11 12 13 14 16 18
</pre></div>

[Black_Fox]

What does that mean? Is it similar as in tuned or more different?

[Bulat Ziganshin]

- the last 3 bytes are different
lzt compress data in 16mb blocks, so dists/lens are limited to 2^24. it's why he limited last values in both tables to 24, and made appropriate changes in two previous values. but note that he retained sizes of both tables - 16 and 32, respectively

it's interesting to note that using lengths up to the size of entire buffer/window is a "smell" of my programs - arjz/tornado. all other programs i've seen, limits this to 258, 273 or smth like this (well, one more exception is uc2 forget at these days)

- tornado uses int vs. lzturbo uses char as datatype
he is smarter than me!

- this buffer is nothing else than a representation of the lz77 deflate 64 distance tree... but all implementations i found start with value 0 not 4... so whatever source or paper both authors based their tree on, i guess its the same and would like to know which one.

first, read lzxfmt from http://www.haskell.org/bz/lzh.7z

i use combined dist/len code like lzx in order to make (de)compression faster. with 16 len codes and 32 dist codes there are total 512 codes for matches. this makes adding dist codes rather expensive and moreover you should add a lot of them to improve compression, like zip's {0,0,0,0,1,1,2,2,3,3}. this 10 more codes will increase total amount of match codes up to 672. moreover, unlike zip, tor uses repdist/repboth codes which significantly reduces amount of matches with small dist. moreover, unlike lzx, tor is not optimized strictly for executables/binary files. taking this all together, the resulting decision was to avoid these small codes. this slightly improves text compression and probably slightly decreases binary compression - experiment yourself

it's interesting to note that lbits table in lzt was further optimized for text files. nevertheless, the 16*32 configuration remains untouched. lzx, for example, uses 8*50 codes

[Bulat Ziganshin]

Ilya, CODE sections are absolutely unreadable now - the letters are very small

[encode]

Ilya, CODE sections are absolutely unreadable now - the letters are very small

FIXED!

[Bulat Ziganshin]

thaks, Ilya. now it's perfect

[Raymond_NGhM]

WELL WELL WELL

How do you think about it, Bulat Ziganshin...
It compiled with GCC 4
He think, ( my mean Hamid ) who is he, hah.
and now LZTurbo is opensource.
yes of course, of course 'Doctor Reactor'

AH, one another thing, LZMA is best after RAR
and igor need more and more to improve and improve IT.
specially that,in adding multimedia filters for .7z

[Raymond_NGhM]

And Good luck my friends.

[Bulat Ziganshin]

What does that mean? Is it similar as in tuned or more different?

either:
tor&lzt was written independent, but by accident
1) both have the same 16*32 match coder configuration (as opposed to 8*50 used in the only other coder of this type)
2) both start dist table with 4-bit codes (as opposed to every other coder that start with 0-bit codes)
3) both use lengths up to the size of entire buffer

third solution looks natural although not yet used anywhere except uc2. second solution is also improvement although independent develper may select to start table with 3-bit or 5-bit codes, this particular choice (4) is rather arbitrary. first solution seems rather arbitrary, although to some degree it is justified by using the same tables for bitcoder too

or:
dns may have started with tornado code and changed the following:
1) last elements of len/dist tables due to 16mb limit of blocks used
2) len codes to improve compression of enwik9 while decreasing compression of binary files


btw, ive also found cabarc-like dist table in lzt executable:
<div class=""jscript""><pre>00 00 00 00 01 01 02 02 │ 03 03 04 04 05 05 06 06
07 07 08 08 09 09 0A 0A │ 0B 0B 0C 0C 0D 0D 0E 0E
0F 0F 10 10 11 11 12 12 │ 12 12 12 12 12 12 12 12
12 12 12 12 12 12 12 12 │ 12 12 12 12 12 12 12 12
12 12 12 12 12 12 12 12 │ 12 12 12 12 12 12 12 12
12 12 12 12 12 12 12 12 │ 12 12 12 12 12 12 12 12
12 12 12 12 12</pre></div>



probably, one table is used in -4* mode and another one in -5*. and -5* mode is slower because it encodes len and dist separately

also, -59 speed seems rather close to 7zip -mx5