CHK 1.01 is here! (New GUI MD5/SHA1 file checker)

[encode]

Hi fellas!

Check out the CHK! With this small GUI tool you can check/verify file contents, find duplicate files, save checksums to .md5/.sha1 files (compatible with md5sum/sha1sum and many other utilities) and more!

Have fun!

File Name: chk101.zip
Size: 437,848 bytes
MD5: BB7F5DD8AED14743F5213E1EA097F9B9

[kampaster]

encode thank

[chornobyl]

not md5 nor size wont match, is something wrong

[encode]

To compare items by hash (MD5/SHA1), simply press on MD5/SHA1 column. All items will be sorted by hash and matched ones will be marked by special icon!

[Black_Fox]

Nice When hashing some bigger files (say 50 MB each), CHK stops responding to user input at second or third file... It doesn't freeze in endless loop nor crash, just looks unresponsive.

[encode]

It's because the hash computing is done in main thread! It's BETA and proof of concept anyway... If program will become popular, I will do many improvements. If not, I'll keep it as is, since I wrote that tool for myself.

[schnaader]

Nice program. Some usability things:

- If no file is marked, "Edit->Copy" command should create a summary for the whole list instead of doing nothing.
- (Un)checking "Edit->Use SHA1" shouldn't clear the list but just switch MD5<->SHA1

not md5 nor size wont match, is something wrong

Worked for me, downloaded the ZIP file and MD5 and size matched. Are you sure you checked MD5/size of the ZIP file and not chk.exe (which is 458.240 bytes, C7A67B53B36E036EC77759E688EAB307)?

[chornobyl]

-adding same files twice, listing them two times and not replacing
-stop button, have to kill program when selected too big file
-start button, to first create list, then compare
-fixed size font for hashes, and option for lowercase
-and maybe grouping similar lines by background color
-cant select few files inside
-deleting remove file avoiding recycle bin

you sure you checked MD5/size of the ZIP

my bad, i checked executable itself

[ben]

look fine
tested on my project dir 20Go / 8826 files / 2 raptors Raid0
matches are found with md5

somes remarks :
i not find how scan a dir with menu (wildcard * not enabled)
can not manage files filters (scan only *.dat and/or don't scan files *.7z)
save as : export ascii scan list in form md5 *path\filename
should be nice if a marker exist like a + instead of * when duplicate file matches (was a bit tricky to find duplicate in ascii file with 8000+ files scaned)

another thing, not sure if in spirit of this prog (as is done to calculate signature)
turbo mode : just read file data (and calc sign) only if some others files have same length
chk : 13sec
double file scanner http://mulder.dummwiedeutsch.de/home...cts#doublescan : 8sec

[encode]

i not find how scan a dir with menu (wildcard * not enabled)

Drag&Drop a Folder or even a Drive to CHK's window! (Drop item(s) here, means drop not just a file(s) here)

should be nice if a marker exist like a + instead of * when duplicate file matches (was a bit tricky to find duplicate in ascii file with 8000+ files scaned)

MD5 and SHA1 files are in md5sum and sha1sum formats respectively - i.e. you can use these command line utilities to test integrity of files. In addition, many other utilities are compatible with such files. As example, check out the ExactFile! Also, you may sort a list by the digest (MD5/SHA1) and only then save to file - file will contain information sorted my MD5/SHA1.

another thing, not sure if in spirit of this prog (as is done to calculate signature)
turbo mode : just read file data (and calc sign) only if some others files have same length
chk : 13sec
double file scanner http://mulder.dummwiedeutsch.de/home...cts#doublescan : 8sec

CHK is not just about duplicate file search! And as a note it can be many times faster - I have my own SHA1 implementation which is 4x times faster than current Indy component based one. Also I may add a multi-threading! But at this stage of development I wrote just a small program with basic functionality. If no one needs in such kind of software I will not spend my spare time for it. If CHK will be more or less popular, as I said, I will add some further work!

[encode]

Somehow CHK 1.01 faced a false detection by some anti-viruses.

Wrote a letter to Kaspersky. The answer is:

Hello,

Sorry, it was a false detection. It will be fixed in the next update.
Thank you for your help.

-----------------
Regards, Artem Ushkov
Virus Analyst, Kaspersky Lab.

Anyway, some other AV software detects CHK 1.01 as a virus/hoax.

IN REAL WORLD, IT IS NOT!

I just can't write a letter to ALL AV makers... If someone can, please write a note to your favorite AV software vendor! Thanks a lot!

[Biozynotiker]

I have the same problem with my own programs, until I avoid UPX to compress my work.
Your are using UPX as well, try it without.

[encode]

Did a few improvements:
+ More polished GUI
+ Improved SHA1 computation - now it's my own and optimized SHA1 implementation that a few times faster than a component-based one as in current version
+ CRC computed simultaneously and kept for reference
+ Added ECR (Estimated Compression Ratio) feature. Now you can find a highly redundant files and determine already compressed files. Furthermore now you can see how good the file can be compressed.

[encode]

Any ideas are welcomed, BTW! CHK is a file analysis tool, from the data compression world. What kind of a feature you need the most?

[pat357]

What version and options did you use to UPX the chk.exe ?
UPX 3.07 can unpack it without any errors, but the unpacked file doesn't work anymore.
(something about a missing DLL)
Any idea why this is ?

[encode]

I always pack with the latest UPX and use --ultra-brute switch.

[Skymmer]

What version and options did you use to UPX the chk.exe ?
UPX 3.07 can unpack it without any errors, but the unpacked file doesn't work anymore.
(something about a missing DLL)
Any idea why this is ?

I can confirm it. The reason is that there is a shift and overlapping in reconstructed import table. I tried manually manipulate with deUPX-ed file but failed, just don't have too much experience with such things. Anyway, I have the working unpacked version of it, so if you need it then I can give it to you.
But interesting... I never seen such thing before with UPX. Seems that is some error in decompression routine. Ilia, can you provide the original non-packed file so we can submit bug-report to UPX developers ?

[encode]

Looks like I have no original EXE - only packed one. CHK v0.01 is from year 2010, probably UPX developers already fixed such issues. Anyway, we will check it in the upcoming CHK 1.02!

[encode]

+ Now if you'll add the same file(s) miltiple times the information will be updated instead of adding duplicated items

Most likely I'll switch to SHA256 since it's more secure. CRC will be kept since nearly all archivers use it for file integrity checking.

[encode]

Also, probably instead of current predictor I'll add file entropy calculator. Obviously not just an order-0 file entropy, but order-1...order-5. Anyway, current multi-order SR predictor is really accurate.

[encode]

[encode]

Marking matched files with a star:

[encode]

Now it's much more solid!

Collecting new ideas. As example I may add a "Compress" command to compress a file to .GZ/.BZ2/.Z file. In this case you will able not to just search highly redundant files, but compressing them as well. You may search and delete duplicate files already. Have an idea about authentication/signing...

And even if no one interested in CHK currently, it will be something cool and unique at the end. Believe me!

[zody]

Maybe 2 file windows side by side would be useful to compare directories easier =)
The estimated compression ratio is a really helpful feature

"Compress" command could start any archiver via command line...

[encode]

The estimated compression ratio is a really helpful feature

BTW, looks like CHK and only has precise compression prognosis - it's really close to result of real world compressors. WinRAR has such feature, but its results are far from real.