Results 1 to 5 of 5

Thread: ( almost ) unlimited compression with no data loss

  1. #1
    Member
    Join Date
    Apr 2016
    Location
    France
    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Question ( almost ) unlimited compression with no data loss

    hi everybody

    well at least it works in theory;

    here's the topic :

    any given text file usually gets compressed to 40% of original size;

    a given text file uses approximately 32 characters;

    it is easy to tranform any binary file to a text file ( that is to say it uses 32 different values ) ;

    for this we use a second file which will indicate what group of 32 characters we refer to

    7 for a value between 224 et 255
    6 pour une valeur comprise entre 192 et 223
    5 pour une valeur comprise entre 160 et 191
    4 pour une valeur comprise entre 128 et 159
    3 pour une valeur comprise entre 96 et 127
    2 pour une valeur comprise entre 64 et 95
    1 pour une valeur comprise entre 32 et 63
    0 pour une valeur comprise entre 0 et 31


    ok.
    so this file holds 8 different values
    each value can be coded with 3 bits , for a total of 3/8 = 37.5 of the original file

    let's take a file of 100ko
    as mentioned earlier, a text file is usually reduced to 40% of original size

    our output file will weight 40%+ 37.5% = 77.5%

    where is the error ??

  2. #2
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    Have you tried it? You're just moving 3 bits of each byte from one file to another. If you start with random data you won't get 40% of original size with 5 of 8 bits remaining. You'll get 62.5% with a good compressor.

  3. #3
    Member
    Join Date
    Sep 2007
    Location
    Denmark
    Posts
    856
    Thanks
    45
    Thanked 104 Times in 82 Posts
    Text is more than just having 32 different characters. Its the structure as well which you will not have when converting binary into "text" so you 40% compression rate is based on something complete different than what you are making

  4. #4
    Member
    Join Date
    Aug 2008
    Location
    Planet Earth
    Posts
    772
    Thanks
    63
    Thanked 270 Times in 190 Posts
    With http://encode.ru/threads/2480-ble-(f...ength-encoding you can see an example what can happen with two random files, bytes become smaller but included the (bookkeeping) bitmap and headers it become larger:

    ble 7 AMillionRandomDigits.bin AMillionRandomDigits.ble7

    bitmap from 75,200 to 75,199 bytes (0.00%)
    bytes from 415,313 to 364,415 bytes (-12.26%)
    encode from 415,313 to 451,365 bytes (8.68%)

    ble 15 AMillionRandomDigits.bin AMillionRandomDigits.ble15

    bitmap from 40,960 to 40,960 bytes (0.00%)
    bytes from 415,313 to 389,359 bytes (-6.25%)
    encode from 415,313 to 430,344 bytes (3.62%)

    ble 7 2012-08-30.bin 2012-08-30.ble7

    bitmap from 189,344 to 189,338 bytes (0.00%)
    bytes from 1,048,576 to 920,087 bytes (-12.25%)
    encode from 1,048,576 to 1,139,011 bytes (8.62%)

    ble 15 2012-08-30.bin 2012-08-30.ble15

    bitmap from 98,304 to 98,304 bytes (0.00%)
    bytes from 1,048,576 to 983,044 bytes (-6.25%)
    encode from 1,048,576 to 1,081,408 bytes (3.13%)

  5. #5
    Member
    Join Date
    Apr 2016
    Location
    France
    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts
    thank you for your contribution

    that's just what I was lookin for ..

Similar Threads

  1. loseless data compression method for all digital data type
    By rarkyan in forum Data Compression
    Replies: 157
    Last Post: 9th July 2019, 17:28
  2. Any money in data compression?
    By bitewing in forum The Off-Topic Lounge
    Replies: 18
    Last Post: 19th March 2019, 10:34
  3. Data Compression PC
    By encode in forum The Off-Topic Lounge
    Replies: 202
    Last Post: 3rd January 2019, 23:28
  4. lossless data compression
    By SLS in forum Data Compression
    Replies: 21
    Last Post: 15th March 2011, 11:35
  5. Sensor Data Compression
    By mmhn97 in forum Data Compression
    Replies: 11
    Last Post: 21st December 2010, 06:21

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •