Results 1 to 6 of 6

Thread: xrle-eXtreme Run Length Encoding

  1. #1
    Member
    Join Date
    Apr 2015
    Location
    Greece
    Posts
    68
    Thanks
    31
    Thanked 22 Times in 15 Posts

    xrle-eXtreme Run Length Encoding

    xrle is a compression algorithm inspired by RLE64 and lrrle.
    Source code https://github.com/algorithm314/xrle

  2. The Following User Says Thank You to algorithm For This Useful Post:

    nikkho (20th April 2015)

  3. #2
    Member nikkho's Avatar
    Join Date
    Jul 2011
    Location
    Spain
    Posts
    542
    Thanks
    214
    Thanked 163 Times in 104 Posts
    That looks pretty nice. I am glad you were able to beat RLE64 in terms of speed (and also compression ratio).
    Thanks for sharing.

  4. #3
    Member
    Join Date
    May 2008
    Location
    Germany
    Posts
    410
    Thanks
    37
    Thanked 60 Times in 37 Posts
    is there a binary for windows w32/wx64 ?

    please upload such a binary for testing

    best regards

  5. #4
    Member
    Join Date
    Apr 2015
    Location
    Greece
    Posts
    68
    Thanks
    31
    Thanked 22 Times in 15 Posts
    Quote Originally Posted by nikkho View Post
    That looks pretty nice. I am glad you were able to beat RLE64 in terms of speed (and also compression ratio).
    Thanks for sharing.
    xrle is basically RLE64 but instead of a 64bit escape value and 64bit repeat length,it has a 32bit literal length and 32bit repeat length.
    For incompressible data memcpy is faster (at least in Linux) because for copies larger than 256kbytes it uses non-temporal store instructions.
    I don't know any way to add them without using inline assembly.If you know any, let me know.

    Quote Originally Posted by joerg View Post
    is there a binary for windows w32/wx64 ?

    please upload such a binary for testing

    best regards
    Do you mean a binary that compresses on disk?xrle is very fast (many times faster than disk) so it will spent almost all time waiting for the disk.xrle is usefull for in memory compression and for low cpu usage.

  6. #5
    Member nikkho's Avatar
    Join Date
    Jul 2011
    Location
    Spain
    Posts
    542
    Thanks
    214
    Thanked 163 Times in 104 Posts
    Yes, I noticed that, but liked your optimizations, which are smart IMHO:
    - Skip small blocks (less than 16 bytes).
    - Use memcpy instead of looping as you stated.


    I am also guessing that maybe a memset would be faster than looping, like in:
    for(i = 0;i < repeat;i++)
    out_pos[i] = word;

  7. #6
    Member
    Join Date
    Apr 2015
    Location
    Greece
    Posts
    68
    Thanks
    31
    Thanked 22 Times in 15 Posts
    Quote Originally Posted by nikkho View Post
    Yes, I noticed that, but liked your optimizations, which are smart IMHO:
    - Skip small blocks (less than 16 bytes).
    - Use memcpy instead of looping as you stated.


    I am also guessing that maybe a memset would be faster than looping, like in:
    for(i = 0;i < repeat;i++)
    out_pos[i] = word;
    Where do i use memcpy instead of looping?I think it doesn't use except for trailling bytes and less than 16 bytes input.

    Compilers do preaty good job and loops like
    for(i = 0;i < lit_len;i++)                                            
    out_pos[i] = in_pos[i];

    and
    for(i = 0;i < repeat;i++)
    out_pos[i] = word;

    are auto-vectorized.
    I have tested memcpy but,memcpy comes with overhead when having many small literals and usually literals are smaller than 256KB ,so it doesn't use non-temporal memory stores.
    How can i use memset?Memset sets 1byte values but xrle 8byte values.

Similar Threads

  1. TurboRLE: Turbo Run Length Encoding
    By dnd in forum Data Compression
    Replies: 23
    Last Post: 16th June 2019, 17:39
  2. Replies: 38
    Last Post: 27th April 2016, 18:01
  3. "Extreme" compression of DNS domains
    By nickety in forum Data Compression
    Replies: 20
    Last Post: 22nd October 2011, 01:20
  4. Advanced Huffman Encoding
    By Simon Berger in forum Data Compression
    Replies: 28
    Last Post: 15th April 2009, 14:24
  5. Variable-length Offset
    By Cyan in forum Data Compression
    Replies: 8
    Last Post: 12th December 2008, 14:06

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •