Results 1 to 12 of 12

Thread: St compressor

  1. #1
    Member
    Join Date
    Oct 2014
    Location
    Bucharest
    Posts
    6
    Thanks
    0
    Thanked 2 Times in 1 Post

    St compressor

    Upon some consideration I decided to share my PPM model.
    Included is an estimator of compressed file size.
    Your observations and opinions are welcomed.
    In case you make use of the source code, I appreciate crediting it.

    Stefan

  2. The Following 2 Users Say Thank You to cloudreflection For This Useful Post:

    Gonzalo (9th October 2014),Nania Francesco (9th October 2014)

  3. #2
    Tester
    Nania Francesco's Avatar
    Join Date
    May 2008
    Location
    Italy
    Posts
    1,565
    Thanks
    220
    Thanked 146 Times in 83 Posts
    The important thing is to start with, confront and always try to improve. Looks like a good code PPM but why not create a standard compressor files, if you have trouble you can see fpaq sources, or other open source software. Just remember that a compressor that uses float numbers for the probability (float or double) in theory will not be easily compatible on all processors.

  4. #3
    Member
    Join Date
    Oct 2014
    Location
    Bucharest
    Posts
    6
    Thanks
    0
    Thanked 2 Times in 1 Post
    Not sure what are standard compressor files. Currently the code only estimates file size after compression. Added some updates and bug fixes, for anyone interested you'll find an improved version attached.

  5. #4
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    Normally a program would take command line arguments maybe like this:

    st c input compressed_file
    st d compressed_file output

    I suggest removing #include "stdafx.h", using <> instead of "", and adding <string.h> to make it portable.

  6. #5
    Member
    Join Date
    Oct 2014
    Location
    Bucharest
    Posts
    6
    Thanks
    0
    Thanked 2 Times in 1 Post
    Attached, the latest version of st.

  7. #6
    Tester
    Stephan Busch's Avatar
    Join Date
    May 2008
    Location
    Bremen, Germany
    Posts
    872
    Thanks
    457
    Thanked 175 Times in 85 Posts
    How would you compile that?

  8. #7
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    681
    Thanks
    37
    Thanked 168 Times in 84 Posts
    IntelC 11.1.072 -fp:precise compile
    Just to remind - program assumes that the file named enwik7 is nearby the exe file.
    Attached Files Attached Files

  9. The Following User Says Thank You to Skymmer For This Useful Post:

    Stephan Busch (15th December 2014)

  10. #8
    Member
    Join Date
    Oct 2014
    Location
    Bucharest
    Posts
    6
    Thanks
    0
    Thanked 2 Times in 1 Post
    Pleased to share an improved version of st.
    Last edited by cloudreflection; 29th March 2015 at 14:57.

  11. #9
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    With enwik8 in the current directory it produces the output (ignoring any command line arguments):

    Estimated filesize 44823736
    77.95

    If you decide to make this into a general purpose compressor (for any file), I will benchmark it.

  12. #10
    Member
    Join Date
    Apr 2009
    Location
    here
    Posts
    202
    Thanks
    165
    Thanked 109 Times in 65 Posts
    i'm getting this (64bit binary):

    Estimated filesize 43984826
    96.77
    and this if compiled with -O3
    Estimated filesize 43984826
    47.26
    edit:
    as 32bit binary (again with -O3) i got the same size as matt.
    Estimated filesize 44823736
    39.37
    the 32bit version is faster than the 64bit, but compression is worse obviously.
    Last edited by load; 4th April 2015 at 20:58.

  13. #11
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,255
    Thanks
    306
    Thanked 778 Times in 485 Posts
    Yes, I tested with a 32 bit compile. I didn't think 64 bit would be different.

  14. #12
    Member
    Join Date
    Oct 2014
    Location
    Bucharest
    Posts
    6
    Thanks
    0
    Thanked 2 Times in 1 Post
    Added a preprocessing stage and improved the main model.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •