Results 1 to 6 of 6

Thread: Deflate stream question

  1. #1
    Member Razor12911's Avatar
    Join Date
    May 2016
    Location
    South Africa
    Posts
    31
    Thanks
    51
    Thanked 53 Times in 15 Posts

    Question Deflate stream question

    Hi all,


    I'm making a deflate or rather a zlib precompressor written in pascal, already managed to make something that can look for all streams in a file, am now dealing with the precompression part, now there is something I want to know before precompressing a certain stream which is the sliding window / window size that was used either from deflate headers or just by decompressing that stream, is there a way to know that? I want to sort out the streams with 16K and 32K window size from the other ones with different sliding window then fed them to different functions that can handle each.

    Thanks in advance.

  2. #2
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,134
    Thanks
    179
    Thanked 921 Times in 469 Posts
    For zlib there is a way - http://stackoverflow.com/a/9050274/395609
    Otherwise you just have to presume that the window is always 32k.

  3. The Following User Says Thank You to Shelwien For This Useful Post:

    Razor12911 (19th September 2016)

  4. #3
    Programmer schnaader's Avatar
    Join Date
    May 2008
    Location
    Hessen, Germany
    Posts
    539
    Thanks
    192
    Thanked 174 Times in 81 Posts
    As Shelwien said, the zlib header is the only way to detect it for sure. Anyway, one further comment to this: In other cases where you don't know window size, presume 32K like Shelwien said, but have a look at the biggest distance value used - this determines which window size settings can be used. E.g. if the biggest distance value is 5K, recompression will give the same results for 8K, 16K and 32K window size (except zlib header), but not for values <= 4K.
    http://schnaader.info
    Damn kids. They're all alike.

  5. The Following User Says Thank You to schnaader For This Useful Post:

    Razor12911 (19th September 2016)

  6. #4
    Member Razor12911's Avatar
    Join Date
    May 2016
    Location
    South Africa
    Posts
    31
    Thanks
    51
    Thanked 53 Times in 15 Posts
    Quote Originally Posted by Shelwien View Post
    For zlib there is a way - http://stackoverflow.com/a/9050274/395609
    Otherwise you just have to presume that the window is always 32k.
    Thank you for response however, I already know this much, the program I'm trying to create detects deflate streams instead of zlib streams which have the headers from the link you provided.

    Quote Originally Posted by schnaader View Post
    As Shelwien said, the zlib header is the only way to detect it for sure. Anyway, one further comment to this: In other cases where you don't know window size, presume 32K like Shelwien said, but have a look at the biggest distance value used - this determines which window size settings can be used. E.g. if the biggest distance value is 5K, recompression will give the same results for 8K, 16K and 32K window size (except zlib header), but not for values <= 4K.
    Anyways I don't know if you guys understand pascal but I found this piece of code so what you're telling me is that even if I pursue the idea that was used here to calculate window size or rather the distance, which is this part in the attachment I uploaded:
    Distance := dfc_DistanceBase[Symbol] + ((BitBuffer shr SymbolCodeLen) and AbExtractMask[ExtraBitCount]);


    where dfc_DistanceBase is


    dfc_DistanceBase : array [0..31] of word =
    (1, 2, 3, 4, 5, 7, 9, 13, 17, 25, 33, 49, 65, 97, 129, 193, 257,
    385, 513, 769, 1025, 1537, 2049, 3073, 4097, 6145, 8193, 12289,
    16385, 24577, 32769, 49153);
    Attached Files Attached Files

  7. #5
    Administrator Shelwien's Avatar
    Join Date
    May 2008
    Location
    Kharkov, Ukraine
    Posts
    3,134
    Thanks
    179
    Thanked 921 Times in 469 Posts
    As schnaader said, you can actually decode first block of deflate stream (and you kinda have to do that for detection)
    and use maximum distance from matches there to determine the window size for the block... and the min window size for the stream.
    But overall there's no way to detect the window size in advance. In fact, it may be not a good idea to do that at all, as there're streams
    made by concatenation of separately compressed blocks (eg. .docx files are like that), in which case the window size may be different
    for parts of deflate stream.

  8. #6
    Member Razor12911's Avatar
    Join Date
    May 2016
    Location
    South Africa
    Posts
    31
    Thanks
    51
    Thanked 53 Times in 15 Posts
    Ok thanks, good to know.

Similar Threads

  1. ZLib Stream Checker
    By Razor12911 in forum Data Compression
    Replies: 2
    Last Post: 25th May 2016, 22:16
  2. Replies: 3
    Last Post: 16th May 2016, 00:02
  3. Replies: 7
    Last Post: 4th January 2016, 15:06
  4. Compression for a stream that flushes often?
    By aninternet in forum Data Compression
    Replies: 11
    Last Post: 19th April 2014, 11:15
  5. Precomp stream detection
    By danswano in forum Data Compression
    Replies: 10
    Last Post: 26th September 2013, 03:10

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •