I'm looking for advice on which compression utility would be best for compressing large amounts of .VHD (Virtual hard disk) files. 10-20TB with plenty of duplication due to (Windows mostly) OS files, also lots of SQL data inside the .VHD files.
We have tried qpress and eXdupe (which is pretty good, but see below).
What we're looking for:
- reasonable compression
- can extract individual files
- supports directory recursion
- benefit from lots of memory (64GB+ free) and lots of CPU cores but at low clock speed (Quad Proc E5-4640)
eXdupe usually works quite well, but we notice that sometimes it struggles in certain places inside the VHD files where speed drops from 300-400MB/s right down to 10-20MB/s. The compression ratio on our files is between 5:1 and 10:1, which is acceptable.
So, what should we be looking at?