Results 1 to 7 of 7

Thread: Is Zstandard eventually going to take the place of zlib?

  1. #1
    Member
    Join Date
    Sep 2016
    Location
    China
    Posts
    9
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Is Zstandard eventually going to take the place of zlib?

    Reading up on FSE/ANS I've discovered Zstandard. It seems to compare very well from the benchmarks against zlib. It also appears that it's being packaged into GNU distributions as "zstd" and "libzstd".
    So I was genuinely wondering if we can assume that, in the future, this library is going to be the default go-to choice for most compression needs (a place that is currently occupied by zlib).

  2. #2
    Member
    Join Date
    Dec 2011
    Location
    Cambridge, UK
    Posts
    437
    Thanks
    137
    Thanked 152 Times in 100 Posts
    I hope so! Zlib is a dinosaur that lasted very well. It was great for the era, but I'm hoping Zstd heralds the new era of rock solid ubiquitous compression library.

  3. #3
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    No, because compatibility with existing software is more important than speed or compression ratio. The deflate format will be around for a long time.

  4. #4
    Member
    Join Date
    Dec 2011
    Location
    Cambridge, UK
    Posts
    437
    Thanks
    137
    Thanked 152 Times in 100 Posts
    What's more peculiar is why zlib-ng, intel igzip and cloudflare's zlib haven't taken the place of the original zlib.

    All 3 of them are faster (maybe double the speed at compression) while being binary compatible. It looks simply like lack of decent packaging is maybe stopping the distributions from using them, or simply fear of change.

  5. #5
    Member
    Join Date
    Dec 2015
    Location
    US
    Posts
    57
    Thanks
    2
    Thanked 112 Times in 36 Posts
    Because zlib is everywhere, everyone knows it, because it has lots of desirable secondary characteristics such as "decades of extensive use make it pretty safe to assume there's no rare data-corruption bugs or nasty exploitable security vulnerabilities", and in a fair number of use cases, the answer to "would you rather have more speed or compression ratio?" is "honestly, I'm fine with both as they are, and what I'd really rather not have is breakage from updating a component that's completely incidental to what my app is trying to do".

    If compression is a significant part of what you do, or a significant bottleneck, then you might be motivated to use a better zlib, or Zstd, or whatever. Yes, zlib is boring. But boring is exactly what you want in many cases..

    For most of the world, DEFLATE is just part of some protocols and file formats they deal with, but completely incidental to what they're doing. They have no reason to upgrade, and really they shouldn't. The last thing these apps need is to complicate their dependencies and deployment with random "exotic" (which at least igzip and Cloudflare's zlib certainly are, and so is zstd right now) dependencies that don't contribute much value to the app and have an increased likelihood of causing trouble and extra maintenance work later on.

  6. The Following 3 Users Say Thank You to fgiesen For This Useful Post:

    jibz (28th September 2016),jman (13th April 2017),stbrumme (28th September 2016)

  7. #6
    Member
    Join Date
    Nov 2015
    Location
    ?l?nsk, PL
    Posts
    81
    Thanks
    9
    Thanked 13 Times in 11 Posts
    I expect zlib to have well more than a decade of significant use ahead. Possibly much more. But whether it will stay dominant after that time? It's hard to tell. I think that ZSTD is the first codec that has a chance of challenging zlib's dominance.

  8. #7
    Member
    Join Date
    Dec 2011
    Location
    Cambridge, UK
    Posts
    437
    Thanks
    137
    Thanked 152 Times in 100 Posts
    Matt and Fabian raise interesting points and I agree with current uses. However new tools and new file formats will gradually help the take up of Zstd instead. This is already happening. Deflate will be around for as long as those old file formats / protocols are in use, which could be decades, but percentage wise it's bound to drop off.

Similar Threads

  1. Zstandard
    By Cyan in forum Data Compression
    Replies: 330
    Last Post: 28th April 2019, 02:57
  2. ZLib Stream Checker
    By Razor12911 in forum Data Compression
    Replies: 2
    Last Post: 25th May 2016, 22:16
  3. Zlib-ng: a performance-oriented fork of zlib
    By dnd in forum Data Compression
    Replies: 0
    Last Post: 5th June 2015, 14:29
  4. Replies: 23
    Last Post: 25th February 2015, 04:56
  5. Intel's Zlib
    By JamesB in forum Data Compression
    Replies: 9
    Last Post: 21st June 2014, 03:36

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •