I wrote a small program for Win64 to demonstrate my subroutines for fast compression/uncompresion with simple hash. Download it here:
1. Your CPU should support AVX instructions set. (Yes, my code uses XMM register to calculate 4 hashes at once. But it accelerates compression on 5% ).
2. For large files you must have ~1Gb free memory for compression, and ~2Gb for uncompression. (Generally speaken its not necessary: the program can be compiled to read smaller pieces of data.)
3. The program does not calculate and store check sum.
4. The program does not save file attributes in archive.
5. The program rewrites files without prompt.
6. After (un)compress the program shows working time in ms by GetTickCount.
On a CPU 2GHz and RAM 1.6 GHz my program compresses enwik8 (the file for puzomerka) in 0.8 s., and uncompresses in 0.42 s. Compression ratio ~ 52%. My code creates hash table of 512 K cells 8 bytes each, its not so good...
lz4i64.exe enwik8 (compress enwik8 -> enwik8.lz4i)
ren enwik8 enwik8_1
lz4i64.exe enwik8.lz4i (uncompress enwik8.lz4i -> enwik8 )
fc /b enwik8 enwik8_1
No differences encountered.
My subroutines are OS independed, they do not use memory/stack, and all work data store in registers.
What do you think? May be I'm simple wasting my time?