Results 1 to 6 of 6

Thread: How to get a reference gzip built on Windows

  1. #1
    Member SolidComp's Avatar
    Join Date
    Jun 2015
    Location
    USA
    Posts
    222
    Thanks
    89
    Thanked 46 Times in 30 Posts

    How to get a reference gzip built on Windows

    Hello everyone – For testing purposes, I'd like to have a reference gzip implementation on Windows. The last release of Gzip for Windows is 10 years old. (And I think Igor's gzipper included in 7-zip uses an improved DEFLATE, so it's not a good baseline gzip.)

    The reference gzipper in a lot of benchmarks seems to be zlib. How do I compile zlib into a program on Windows 10? If I use the included project files in Visual Studio 2017, it compiles a program called minizip. I don't know what that is, but won't run anyway. When I try to run the .exe, it complains about needing another DLL. It's strange that the included project and solution files don't take care of this DLL. The docs are many years old, so I'm not sure how to proceed.

    There is also GNU Gzip, which is a program for Unix-like OSes. Is there a way to compile the current release on Windows?

    Bonus question: Do you expect zlib and GNU Gzip to yield different results in terms of compression ratio for gzip files? Relatedly, should newer versions of these programs give better compression ratios than older versions? I was wondering about that when I noticed that the "gzip" implementation that Matt Mahoney uses in his text compression benchmark is more than 14 years old. It's an old Gzip for Windows release, and I wonder if it might be understating gzip performance.

    Anyway, any tips on how to compile zlib into a Windows executable that lets me choose levels 1-9? Or the same for GNU Gzip?

  2. #2
    Programmer
    Join Date
    May 2008
    Location
    PL
    Posts
    307
    Thanks
    68
    Thanked 166 Times in 63 Posts
    http://www.gzip.org/ is not longer maintained. To get the newest version (currently it's v1.8 ) go to http://savannah.gnu.org/projects/gzip/

    For Windows you can compile sources with gcc/MinGW, gcc/MinGW-w64, or gcc/MSYS2. MSYS2 (http://www.msys2.org/) already contains prebuilt gzip-1.8 binaries:
    https://github.com/Alexpux/MSYS2-pac.../gzip/PKGBUILD
    https://sourceforge.net/projects/msy.../MSYS2/x86_64/
    Last edited by inikep; 15th April 2017 at 11:36.

  3. #3
    Member
    Join Date
    Dec 2011
    Location
    Cambridge, UK
    Posts
    437
    Thanks
    137
    Thanked 152 Times in 100 Posts
    The deflate implementation in gzip is different to the one in zlib, although sharing a common origin (and author) it looks. I wouldn't be suprised to find the speed and ratio between the two differs slightly. Minigzip in zlib is just an interface to the zlib algorithms, implementing a simple gzip compatible command. I haven't compared the two for speed.

    It's curious that the major linux distributions (eg Ubuntu) haven't switched from zlib to zlib-ng, which is dramatically faster. Possibly because zlib-ng has never made an official release or maybe it's not as 100% compatible as we believe. It seems suprising still. The zlib implementation is very old with a lot of support for now defunct platforms (eg DOS near vs far pointers) and IIRC it's not even using Ansi C, preferring the previous K&R syntax. I guess it's just too big a risk to replace it and they're hoping long term people will move to newer formats (zstd etc).

    As for building, I'd second the choice of Mingw (any variant). It's definitely the way to go for compiling unix tools on windows without all the idiosyncraces from cygwin.

  4. #4
    Member SolidComp's Avatar
    Join Date
    Jun 2015
    Location
    USA
    Posts
    222
    Thanks
    89
    Thanked 46 Times in 30 Posts
    Quote Originally Posted by JamesB View Post
    The deflate implementation in gzip is different to the one in zlib, although sharing a common origin (and author) it looks. I wouldn't be suprised to find the speed and ratio between the two differs slightly. Minigzip in zlib is just an interface to the zlib algorithms, implementing a simple gzip compatible command. I haven't compared the two for speed.

    It's curious that the major linux distributions (eg Ubuntu) haven't switched from zlib to zlib-ng, which is dramatically faster. Possibly because zlib-ng has never made an official release or maybe it's not as 100% compatible as we believe. It seems suprising still. The zlib implementation is very old with a lot of support for now defunct platforms (eg DOS near vs far pointers) and IIRC it's not even using Ansi C, preferring the previous K&R syntax. I guess it's just too big a risk to replace it and they're hoping long term people will move to newer formats (zstd etc).

    As for building, I'd second the choice of Mingw (any variant). It's definitely the way to go for compiling unix tools on windows without all the idiosyncraces from cygwin.
    zlib-ng doesn't seem ready to use. Last time I checked, it wouldn't compile with nginx. Neither would Cloudflare's optimized zlib patch. It's a bummer that these projects don't make it easier to use them since they're significantly better than zlib. SLZ is also a big surprise, so much better performance at the lower compression levels. There was also something like minizip that was a single header and .c file, or something along those lines. I can't find it now, but it was an attempt to implement zlib or gzip compression in a super clean and concise way, with just one or two source files.

  5. #5
    Member
    Join Date
    Dec 2011
    Location
    Cambridge, UK
    Posts
    437
    Thanks
    137
    Thanked 152 Times in 100 Posts
    For amusement, don't forget the gzip and deflate stream decompressor here:

    http://www.ioccc.org/1996/rcm.c

    Compile with
    Code:
    cc -O3 -DJ='if(' -Ds='else J O==' -DU='B('     -DX='for(' -DH='unsigned' -Dn='struct x' -DA='0,0,0,0,'     -DT='<stdio.h>' -DY='<stdlib.h>' -DR='malloc(sizeof(n))' -DM='%=K' -DK=32768 -DD='getchar()' -DN=putchar rcm.c -o rcm
    .

    It's also a single source file implementation. It's not very efficient though!

  6. The Following User Says Thank You to JamesB For This Useful Post:

    SolidComp (21st April 2017)

  7. #6
    Member
    Join Date
    Mar 2013
    Location
    Worldwide
    Posts
    456
    Thanks
    46
    Thanked 164 Times in 118 Posts
    It is possible to test zlib-ng with TurboBench.
    Build libz.a with the default makefile in the zlib-ng directory and then compile TurboBench with "make ZLIB_NG=1".

    In my test only level 1 compression is fast (requires w/ sse4.2) , but with worse compression. see benchmark web-compression
    Last edited by dnd; 18th April 2017 at 23:07.

  8. The Following User Says Thank You to dnd For This Useful Post:

    SolidComp (21st April 2017)

Similar Threads

  1. Vorbis compressor - Reference
    By maadjordan in forum Data Compression
    Replies: 5
    Last Post: 18th February 2017, 16:27
  2. JPEG XT new reference software, online test updated
    By thorfdbg in forum Data Compression
    Replies: 26
    Last Post: 3rd August 2015, 20:27
  3. Reference of compression format
    By Silky in forum Data Compression
    Replies: 6
    Last Post: 24th April 2012, 04:18
  4. filesharing with built-in recompression
    By Shelwien in forum Data Compression
    Replies: 8
    Last Post: 8th December 2009, 13:42
  5. gzip-1.2.4-hack - a hacked version of gzip
    By encode in forum Forum Archive
    Replies: 63
    Last Post: 10th September 2007, 04:16

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •